Chip simulation defect generation method and device, computer equipment and storage medium

Abstract

The invention relates to a chip simulation defect generation method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring an initial model; training the initial model to obtain a simulation defect model; generating a chip simulation defect image according to the simulation defect model; wherein the simulation defect model comprises an encoder, a layer hopping connector and a decoder, and the chip simulation defect image is generated according to the simulation defect model, and the method comprises the following steps: the encoder receives an original chip image, converts an RGB color feature layer of the original chip image into a sequence feature pattern, and outputs the sequence feature pattern to the decoder; the layer-skipping connector generates a layer-skipping characteristic pattern according to the RGB color characteristic layer of the original chip image and transmits the layer-skipping characteristic pattern to the decoder in a layer-skipping manner, and the decoder generates a chip simulation defect image according to the sequence characteristic pattern, the layer-skipping characteristic pattern and the style code. Defects which do not exist in the training model can be generated, so that the types of the defects are richer.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a chip simulation defect generation method, device, computer equipment and storage medium. Background Art

[0002] Integrated circuits are the cornerstone of information technology. With the rapid development of informatization and intelligentization, integrated circuits have penetrated various fields. Chip simulation defect generation technology plays a vital role in integrated circuit design and manufacturing, especially in the generation of defects in new chip products. By simulating potential manufacturing defects, it ensures that chips have been preliminarily tested before entering actual production, thus avoiding high post-production correction costs and time delays.

[0003] Traditional chip simulation defect generation technology is gradually being surpassed by modern technologies due to its heavy reliance on manual design, low efficiency, poor adaptability, weak generalization, data limitations, difficulty simulating complex scenarios, and optimization challenges. Deep learning-driven chip simulation defect generation technology demonstrates significant advantages over traditional algorithms. Leveraging its ability to learn from large amounts of data, it can generate more complex and realistic defect models, achieve accurate simulation at the microscopic level, and automate large-scale defect simulation.

[0004] However, existing deep learning chip simulation defect generation algorithms still have limitations. Most deep learning algorithms can only perform model training and defect simulation on a certain scale of existing chip data. The generated chip defect simulation images are based on defects already in the training atlas. They cannot generate defects beyond the training atlas, nor can they generate defect images for new products. This significantly limits the application scope of existing chip simulation defect generation technology. Summary of the Invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present invention provides a chip simulation defect generation method, apparatus, computer equipment and storage medium.

[0006] In a first aspect, the present invention provides a chip simulation defect generation method, the method comprising:

[0007] Get the initial model;

[0008] Training the initial model to obtain a simulation defect model;

[0009] generating a chip simulation defect image according to the simulation defect model;

[0010] The simulation defect model includes: an encoder, a jump layer connector and a decoder.

[0011] Generating a chip simulation defect image according to the simulation defect model includes:

[0012] The encoder receives an original chip image, converts the RGB color feature layer of the original chip image into a sequential feature map, and outputs the sequential feature map to the decoder.

[0013] The layer-skipping connector generates a layer-skipping feature map according to the RGB color feature layer of the original chip image, and transmits the layer-skipping feature map to the decoder by layer-skipping.

[0014] The decoder generates the chip simulation defect image according to the sequential feature map, the skip-layer feature map and the style code.

[0015] Optionally, the style code is generated by random noise and a target domain code through a mapping network.

[0016] Optionally, the encoder includes multiple encoding layers,

[0017] The encoder receives an original chip image, converts an RGB color feature layer of the original chip image into a sequential feature map, and outputs the sequential feature map to the decoder, including:

[0018] The first encoding layer receives the original chip image, identifies a first type of feature in the RGB color feature layer of the original chip image, converts the RGB color feature layer of the original chip image into a first feature map containing the first type of feature, and outputs the first feature map to the second encoding layer;

[0019] The second encoding layer identifies the second type of features in the first feature map, converts the first feature map into a second feature map containing the second type of features, and outputs the second feature map to the third encoding layer;

[0020] The third encoding layer identifies a third type of feature in the second feature map, converts the second feature map into a third feature map containing the third type of feature, and outputs the third feature map to the fourth encoding layer;

[0021] The fourth encoding layer identifies a fourth type of feature in the third feature map, converts the third feature map into a fourth feature map including the fourth type of feature, and outputs the fourth feature map to the fifth encoding layer;

[0022] The fifth encoding layer identifies a fifth type of feature in the fourth feature map, converts the fourth feature map into a fifth feature map including the fifth type of feature, and outputs the fifth feature map to the sixth encoding layer;

[0023] The sixth encoding layer identifies a sixth type of feature in the fifth feature map, converts the fifth feature map into a sixth feature map including the sixth type of feature, and outputs the sixth feature map as the sequential feature map to the decoder.

[0024] Optionally, the jump connector includes a first jump connector, a second jump connector, a third jump connector and a fourth jump connector.

[0025] The layer-skipping connector generates a layer-skipping feature map according to the RGB color feature layer of the original chip image, and transmits the layer-skipping feature map to the decoder by layer-skipping, including:

[0026] The first jump layer connector copies the first feature map to generate a first jump layer feature map, and jumps the first jump layer feature map to the sixth decoding layer;

[0027] The second jump layer connector copies the second feature map to generate a second jump layer feature map, and jumps the second jump layer feature map to the fifth decoding layer;

[0028] The third jump layer connector copies the third feature map to generate a third jump layer feature map, and jumps the third jump layer feature map to the fourth decoding layer;

[0029] The fourth jump-layer connector copies the fourth feature map to generate a fourth jump-layer feature map, and jumps the fourth jump-layer feature map to the third decoding layer.

[0030] Optionally, the decoder includes multiple decoding layers, and the number of the decoding layers is equal to the number of the encoding layers;

[0031] The decoder generates the chip simulation defect image according to the sequential feature map, the skip-layer feature map and the style code, including:

[0032] The first decoding layer generates a first decoding feature map according to the sequential feature map and the first style code;

[0033] The second decoding layer generates a second decoding feature map based on the first decoding feature map and the second style code;

[0034] The third decoder generates a third decoding feature map according to the second decoding feature map, the third style code and the fourth skip-layer feature map;

[0035] The fourth decoder generates a fourth decoding feature map according to the third decoding feature map, the fourth style code and the third skip-layer feature map;

[0036] The fifth decoder generates a sixth decoding feature map according to the fourth decoding feature map, the fifth style code and the second skip-layer feature map;

[0037] The sixth decoder generates the chip simulation defect image according to the sixth decoding feature map, the sixth style code and the first jump-layer feature map.

[0038] Optionally, each of the coding layers includes a C2F module, and each of the C2F modules includes: a first ConvIN module, three Bottle Neck submodules, and a second ConvIN module;

[0039] The first encoding layer receives the original chip image, identifies a first type of feature in the RGB color feature layer of the original chip image, converts the RGB color feature layer of the original chip image into a first feature map containing the first type of feature, and outputs the first feature map to the second encoding layer, including:

[0040] The first ConvIN of the first encoding layer receives the original chip image and performs instance normalization on the original chip image;

[0041] The three Bottle Neck submodules of the first encoding layer decouple the image features of the RGB color feature layer of the original chip image after instance normalization through multi-channel decoupling, and extract semantic information based on the deep residual network to identify the first type of features;

[0042] The second ConvIN module of the first encoding layer converts the RGB color feature layer of the original chip image into a first feature map containing the first type of features, and outputs the first feature map to the second encoding layer.

[0043] Optionally, the training of the initial model to obtain a simulation defect model includes:

[0044] When the initial model is trained to obtain a simulation defect model, the total loss function L is:

[0045]

[0046] Among them, L adv To counter the loss function, L cyc is the cycle-consistent loss function, λ cyc For L cyc The hyperparameter, L cls is the cross entropy loss function, λ cls For L cls The hyperparameter, L ms is the MS loss function, λ ms For L ms The hyperparameter, E x,y is the expected value of x and y, D y (x) is the output of the discriminator D for x given y, For x and and the expected value of z, For about The result of the discriminator D, For the generator G, the input x and The output when For x and source domain y and target domain and the expected value of z, is the style code of the source domain y, is the relationship between x and the target domain and the expected values ​​of two different random sampling noises z1,z2 used to generate style codes, For the generator G, the input x and The output when For the generator G, the input x and The output when and For the same target domain Two different style codes, M is the mapping network.

[0047] In a second aspect, a chip simulation defect generation device is provided, wherein the device comprises:

[0048] Model unit, used to obtain the initial model;

[0049] A training unit, configured to train the initial model to obtain a simulation defect model;

[0050] A simulation unit, configured to generate a chip simulation defect image according to the simulation defect model;

[0051] The simulation defect model includes: an encoder, a jump layer connector and a decoder.

[0052] The encoder is used to receive an original chip image, convert the RGB color feature layer of the original chip image into a sequential feature map, and output the sequential feature map to the decoder.

[0053] The layer-skipping connector is used to generate a layer-skipping feature map according to the RGB color feature layer of the original chip image, and transmit the layer-skipping feature map to the decoder by layer-skipping.

[0054] The decoder is used to generate the chip simulation defect image according to the sequential feature map, the skip-layer feature map and the style code.

[0055] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above methods when executing the computer program.

[0056] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method as described in any one of the above items is implemented.

[0057] The present invention provides a method, device, computer equipment and storage medium for generating simulated defects. The method includes: obtaining an initial model; training the initial model to obtain a simulated defect model; generating a chip simulated defect image according to the simulated defect model; wherein the simulated defect model includes: an encoder, a skip-layer connector and a decoder. Generating a chip simulated defect image according to the simulated defect model includes: the encoder receiving an original chip image, converting the RGB color feature layer of the original chip image into a sequential feature map, and outputting the sequential feature map to the decoder; the skip-layer connector generating a skip-layer feature map according to the RGB color feature layer of the original chip image, and skip-layer transmitting the skip-layer feature map to the decoder; the decoder generating the chip simulated defect image according to the sequential feature map, the skip-layer feature map and a style code. In the method of the embodiment of the present invention, the skip-layer connector transmits the skip-layer feature map from the encoder to the decoder, so that the loss of effective information is less and the image details can be restored more finely; in addition, the method of the embodiment of the present invention also injects a style code when generating a chip simulation defect image, which can control the generated defect category. The skip-layer connector and the style code work together, so that the chip defects generated by the method of the present invention are not limited to the chip defects in the training model, and defects that are not in the training model can be generated, so that the types of defects are richer, and the method of the present invention has better generalization and more application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0060] Figure 1 FIG2 is a diagram showing an application environment of a chip simulation defect generation method according to an embodiment of the present invention;

[0061] Figure 2 FIG2 is a flow chart of a chip simulation defect generation method according to an embodiment of the present invention;

[0062] Figure 3 Shown is a schematic diagram of a simulation defect model according to an embodiment of the present invention;

[0063] Figure 4 FIG2 is a schematic diagram of the structure of the coding layer according to an embodiment of the present invention;

[0064] Figure 5 The figure shows a comparison between a chip edge and corner chipping defect diagram according to an embodiment of the present invention and an actual defect diagram;

[0065] Figure 6 FIG2 is a schematic diagram showing the structure of a chip simulation defect generating device according to an embodiment of the present invention;

[0066] Figure 7 Schematic diagram of generating a chip simulation defect map according to an embodiment of the present invention;

[0067] Figure 8 FIG. 1 is a diagram showing the internal structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0068] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0069] Figure 1 FIG. 1 is an application environment diagram of a chip simulation defect generation method in an embodiment. Figure 1 The chip simulation defect generation method is applied to a chip simulation defect generation system. The chip simulation defect generation method includes a terminal 110 and / or a server 120. Terminal 110 and server 120 are connected via a network. Terminal 110 can be a desktop terminal or a mobile terminal. The mobile terminal can be at least one of a mobile phone, a tablet computer, and a laptop computer. Server 120 can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0070] The chip simulation defect generation method of the present invention is applied to the terminal 110 and / or the server 120 .

[0071] like Figure 2 As shown, in one embodiment, a chip simulation defect generation method is provided. This embodiment mainly applies this method to the above Figure 1 The server 120 in FIG. 1 is used as an example. Figure 2 , the chip simulation defect generation method includes:

[0072] Step 210, obtaining an initial model;

[0073] Step 220: training the initial model to obtain a simulation defect model;

[0074] Step 230: Generate a chip simulation defect image according to the simulation defect model.

[0075] Among them, the simulation defect model includes: encoder, jump layer connector and decoder,

[0076] In step 230, generating a chip simulation defect image according to the simulation defect model includes:

[0077] The encoder receives an original chip image, converts an RGB color feature layer of the original chip image into a sequential feature map, and outputs the sequential feature map to the decoder;

[0078] The layer-hopping connector generates a layer-hopping feature map according to the RGB color feature layer of the original chip image, and transmits the layer-hopping feature map to the decoder in a layer-hopping manner;

[0079] The decoder generates the chip simulation defect image according to the sequential feature map, the skip-layer feature map and the style code.

[0080] In the embodiment of the present invention, the style code is generated by random noise and target domain code through a mapping network.

[0081] In an embodiment of the present invention, a technical solution is used to generate defect images from original chip images. The original chip images can be good images of new products. Therefore, the good images of new products can be used. According to the simulated defect model of an embodiment of the present invention, the learned defects are added to the good images of new products or specific structures are deleted, thereby generating simulated defect images with common defects or structural defects on the new products. When using the method of an embodiment of the present invention, when applying new products, there is no need to use defect images of new products to train the model, which can save time and reduce resource consumption.

[0082] Furthermore, the simulated defect model of the embodiments of the present invention can be trained by mixing multiple defect datasets, enabling the simulated defect model to learn various defect semantics, using style codes to control the type of generated defects, and improving the diversity of generated defects. Furthermore, to accurately preserve the complex texture structure of the input chip, the method of the embodiments of the present invention incorporates a skip-layer connection between the encoder and decoder, using a U-Net architecture. This allows the network to generate chip images with the original fine structure even when inputting unseen new products.

[0083] In the method of the embodiment of the present invention, a skip-layer connector transmits the skip-layer feature map from the encoder to the decoder, resulting in less loss of effective information and more precise restoration of image details. Furthermore, the method of the embodiment of the present invention also injects style codes when generating simulated chip defect images to control the generated defect categories. The combined effect of the skip-layer connection and style codes allows the chip defects generated by the method of the present invention to be not limited to those in the training model, but also to generate defects not included in the training model, enriching the defect types and providing the method of the present invention with better generalization and a wider range of applications.

[0084] In the method of the embodiment of the present invention, the style code is generated based on random noise, that is, the image style is generated only from random noise and is not extracted from images such as reference images and training atlases. This helps to remove style interference from reference images and training images, extract style information more directly, and is conducive to generating a richer variety of defects.

[0085] Figure 3 FIG. 1 is a schematic diagram of a simulation defect model according to an embodiment of the present invention. Figure 3 As shown, the simulation defect model includes an encoder, a skip-layer connector, and a decoder, wherein the encoder includes multiple encoding layers, the decoder includes multiple decoding layers, and the number of the decoding layers is equal to the number of the encoding layers.

[0086] In the embodiment of the present invention, from the input direction of the original chip image to the output direction of the chip simulation defect image, that is, from the input end to the output end of the simulation defect model, the coding layers are the first coding layer, the second coding layer, the third coding layer, the fourth coding layer, the fifth coding layer and the sixth coding layer in sequence;

[0087] From the input direction of the original chip image to the output direction of the chip simulation defect image, that is, from the input end to the output end of the simulation defect model, the decoding layers are the first decoding layer, the second decoding layer, the third decoding layer, the fourth decoding layer, the fifth decoding layer and the sixth decoding layer in sequence;

[0088] From the original chip image input direction to the chip simulation defect image output direction, that is, from the input end to the output end of the simulation defect model, they are the first jump layer connector, the first jump layer connector, the first jump layer connector and the first jump layer connector in sequence.

[0089] In an embodiment of the present invention, the encoder receives an original chip image, converts an RGB color feature layer of the original chip image into a sequential feature map, and outputs the sequential feature map to the decoder, including:

[0090] The first encoding layer receives the original chip image, identifies a first type of feature in the RGB color feature layer of the original chip image, converts the RGB color feature layer of the original chip image into a first feature map containing the first type of feature, and outputs the first feature map to the second encoding layer;

[0091] The second encoding layer identifies the second type of features in the first feature map, converts the first feature map into a second feature map containing the second type of features, and outputs the second feature map to the third encoding layer;

[0092] The third encoding layer identifies a third type of feature in the second feature map, converts the second feature map into a third feature map containing the third type of feature, and outputs the third feature map to the fourth encoding layer;

[0093] The fourth encoding layer identifies a fourth type of feature in the third feature map, converts the third feature map into a fourth feature map including the fourth type of feature, and outputs the fourth feature map to the fifth encoding layer;

[0094] The fifth encoding layer identifies a fifth type of feature in the fourth feature map, converts the fourth feature map into a fifth feature map including the fifth type of feature, and outputs the fifth feature map to the sixth encoding layer;

[0095] The sixth encoding layer identifies a sixth type of feature in the fifth feature map, converts the fifth feature map into a sixth feature map including the sixth type of feature, and outputs the sixth feature map as the sequential feature map to the decoder.

[0096] In an embodiment of the present invention, the layer-hopping connector generates a layer-hopping feature map according to the RGB color feature layer of the original chip image, and transmits the layer-hopping feature map to the decoder by layer-hopping, including:

[0097] The first jump layer connector copies the first feature map to generate a first jump layer feature map, and jumps the first jump layer feature map to the sixth decoding layer;

[0098] The second jump-layer connector copies the second feature map to generate a second jump-layer feature map, and jumps the second jump-layer feature map to the fifth decoding layer;

[0099] The third jump-layer connector copies the third feature map to generate a third jump-layer feature map, and jump-layer-transmits the third jump-layer feature map to the fourth decoding layer;

[0100] The fourth jump-layer connector copies the fourth feature map to generate a fourth jump-layer feature map, and jump-layer-transmits the fourth jump-layer feature map to the third decoding layer.

[0101] In an embodiment of the present invention, the decoder generates the chip simulation defect image according to the sequential feature map, the skip-layer feature map, and the style code, including:

[0102] The first decoding layer generates a first decoding feature map according to the sequential feature map and the first style code;

[0103] The second decoding layer generates a second decoding feature map based on the first decoding feature map and the second style code;

[0104] The third decoder generates a third decoding feature map according to the second decoding feature map, the third style code and the fourth skip-layer feature map;

[0105] The fourth decoder generates a fourth decoding feature map according to the third decoding feature map, the fourth style code and the third skip-layer feature map;

[0106] The fifth decoder generates a sixth decoding feature map according to the fourth decoding feature map, the fifth style code and the second skip-layer feature map;

[0107] The sixth decoder generates the chip simulation defect image according to the sixth decoding feature map, the sixth style code and the first jump-layer feature map.

[0108] In the embodiment of the present invention, the first jump layer connector can transmit the texture information of the chip die surface, and the third jump layer connector can transmit features such as the chip frame and bonding wires.

[0109] In the method of the embodiment of the present invention, the number of layers of the decoder, encoder and jump connector can also be other numbers, which can be set according to needs and will not be described in detail here.

[0110] In the method of the embodiment of the present invention, the skip-layer connector transmits the skip-layer feature map from the encoder to the decoder, so that the loss of effective information is less and the image details can be restored more finely; after the style code is injected, the generated defect category can be controlled, so that the generated chip defects are not limited to the chip defects in the training model, but defects that are not in the training model can be generated, so that the types of defects are richer, and the method of the present invention has better generalization and more application scenarios.

[0111] Figure 4 FIG. 1 is a schematic diagram showing the structure of the coding layer according to an embodiment of the present invention. Figure 4 As shown, in the embodiment of the present invention, each of the coding layers includes a C2F module, and each of the C2F modules includes: a first ConvIN module, three BottleNeck submodules, and a second ConvIN module;

[0112] The first encoding layer receives the original chip image, identifies a first type of feature in the RGB color feature layer of the original chip image, converts the RGB color feature layer of the original chip image into a first feature map containing the first type of feature, and outputs the first feature map to the second encoding layer, including:

[0113] The first ConvIN of the first encoding layer receives the original chip image and performs instance normalization on the original chip image;

[0114] The three Bottle Neck submodules of the first encoding layer decouple the image features of the RGB color feature layer of the original chip image after instance normalization through multi-channel decoupling, and extract semantic information based on the deep residual network to identify the first type of features;

[0115] The second ConvIN module of the first encoding layer converts the RGB color feature layer of the original chip image into a first feature map containing the first type of features, and outputs the first feature map to the second encoding layer.

[0116] The instance normalization in this embodiment of the present invention is suitable for image style transfer because it independently normalizes each channel of each sample, maintaining the independence of each image instance. For each channel of each sample, instance normalization calculates the mean and variance of its height and width, and then normalizes them. This normalizes the data within each channel to the same scale. This can accelerate model convergence, and because normalization is performed independently for each sample and each channel, it can better preserve details and features in the image.

[0117] In an embodiment of the present invention, the training of the initial model to obtain a simulation defect model includes:

[0118] When the initial model is trained to obtain a simulation defect model, the total loss function L is:

[0119]

[0120] Among them, L adv To counter the loss function, L cyc is the cycle-consistent loss function, λ cyc For L cyc The hyperparameter, L cls is the cross entropy loss function, λ cls For L cls The hyperparameter, L ms is the MS loss function, λ ms For L ms The hyperparameter, E x,y is the expected value of x and y, D y(x) is the output of the discriminator D for x given y, For x and and the expected value of z, For about The result of the discriminator D, For the generator G, the input x and The output when For x and source domain y and target domain and the expected value of z, is the style code of the source domain y, is the relationship between x and the target domain and the expected values ​​of two different random sampling noises z1,z2 used to generate style codes, For the generator G, the input x and The output when For the generator G, the input x and The output when and For the same target domain There are two different style codes, M is the mapping network.

[0121] The loss function of the embodiment of the present invention can enable the initial model to converge faster during training, and can also generate more diverse outputs.

[0122] Figure 5 The chip edge and corner chipping defect diagram of the embodiment of the present invention is shown ( Figure 5 b) and actual defects ( Figure 5 a) is a comparison chart. Figure 5 It can be seen that the chip simulation defect image generated by the chip simulation defect generation method according to the embodiment of the present invention is more natural and closer to the distribution of actual defects.

[0123] In an embodiment of the present invention, by mixing multiple defect data as training samples, the simulation defect model can learn various defect semantics, making the defects generated by the simulation defect model richer. In an embodiment of the present invention, using style codes, mapping networks, and simulation defect models with jump-layer connectors, it is possible to control the type of defects generated, generate specified defects, and generate multiple defects. In addition, the embodiment of the present invention generates specified defects through style codes, mapping networks, and simulation defect models with jump-layer connectors, eliminating the need to control defect types through traditional manually labeled mask tags, thereby improving efficiency and reducing costs.

[0124] like Figure 6 As shown, the present invention also provides a chip simulation defect generation device, the device comprising:

[0125] Model unit 610, used to obtain an initial model;

[0126] A training unit 620 is configured to train the initial model to obtain a simulation defect model;

[0127] A simulation unit 630 is configured to generate a chip simulation defect image according to the simulation defect model;

[0128] The simulation defect model includes: an encoder, a jump layer connector and a decoder.

[0129] The encoder is used to receive an original chip image, convert the RGB color feature layer of the original chip image into a sequential feature map, and output the sequential feature map to the decoder.

[0130] The layer-skipping connector is used to generate a layer-skipping feature map according to the RGB color feature layer of the original chip image, and transmit the layer-skipping feature map to the decoder by layer-skipping.

[0131] The decoder is used to generate the chip simulation defect image according to the sequential feature map, the skip-layer feature map and the style code.

[0132] The style code is generated by random noise and target domain code through a mapping network. In the embodiment of the present invention, the encoder includes multiple encoding layers,

[0133] The first encoding layer is used to receive the original chip image, identify the first type of features in the RGB color feature layer of the original chip image, convert the RGB color feature layer of the original chip image into a first feature map containing the first type of features, and output the first feature map to the second encoding layer;

[0134] The second encoding layer is used to identify the second type of features in the first feature map, convert the first feature map into a second feature map containing the second type of features, and output the second feature map to the third encoding layer;

[0135] The third encoding layer is used to identify the third type of features in the second feature map, convert the second feature map into a third feature map containing the third type of features, and output the third feature map to the fourth encoding layer;

[0136] The fourth encoding layer is used to identify the fourth type of features in the third feature map, convert the third feature map into a fourth feature map containing the fourth type of features, and output the fourth feature map to the fifth encoding layer;

[0137] The fifth encoding layer is used to identify the fifth type of features in the fourth feature map, convert the fourth feature map into a fifth feature map containing the fifth type of features, and output the fifth feature map to the sixth encoding layer;

[0138] The sixth encoding layer is used to identify the sixth type of features in the fifth feature map, convert the fifth feature map into a sixth feature map containing the sixth type of features, and output the sixth feature map as the sequential feature map to the decoder.

[0139] In the embodiment of the present invention, the first hop connector is used to copy the first feature map to generate a first hop feature map, and hop-transmit the first hop feature map to the sixth decoding layer;

[0140] The second jump-layer connector is used to copy the second feature map to generate a second jump-layer feature map, and jump-layer transmit the second jump-layer feature map to the fifth decoding layer;

[0141] The third jump-layer connector is used to copy the third feature map to generate a third jump-layer feature map, and jump-layer transmit the third jump-layer feature map to the fourth decoding layer;

[0142] The fourth jump-layer connector is used to copy the fourth feature map to generate a fourth jump-layer feature map, and jump-layer transmit the fourth jump-layer feature map to the third decoding layer.

[0143] In an embodiment of the present invention, the decoder includes a plurality of decoding layers, and the number of the decoding layers is equal to the number of the encoding layers;

[0144] The first decoding layer is used to generate a first decoding feature map according to the sequential feature map and the first style code;

[0145] The second decoding layer is used to generate a second decoding feature map according to the first decoding feature map and the second style code;

[0146] The third decoder is configured to generate a third decoding feature map according to the second decoding feature map, the third style code, and the fourth skip-layer feature map;

[0147] The fourth decoder is configured to generate a fourth decoding feature map according to the third decoding feature map, the fourth style code, and the third skip-layer feature map;

[0148] The fifth decoder is configured to generate a sixth decoding feature map based on the fourth decoding feature map, the fifth style code, and the second skip-layer feature map;

[0149] The sixth decoder is used to generate the chip simulation defect image according to the sixth decoding feature map, the sixth style code and the first jump-layer feature map.

[0150] In the embodiment of the present invention, each of the coding layers includes a C2F module, and each of the C2F modules includes: a first ConvIN module, three Bottle Neck submodules, and a second ConvIN module;

[0151] The first ConvIN of the first encoding layer is used to receive the original chip image and perform instance normalization on the original chip image;

[0152] The three Bottle Neck submodules of the first encoding layer are used to decouple the image features of the RGB color feature layer of the original chip image after instance normalization through multi-channel decomposition, and extract semantic information based on the deep residual network to identify the first type of features;

[0153] The second ConvIN module of the first encoding layer is used to convert the RGB color feature layer of the original chip image into a first feature map containing the first type of features, and output the first feature map to the second encoding layer.

[0154] In an embodiment of the present invention, the training of the initial model to obtain a simulation defect model includes:

[0155] When the initial model is trained to obtain a simulation defect model, the total loss function L is:

[0156]

[0157] Among them, L adv To counter the loss function, L cyc is the cycle-consistent loss function, λ cyc For L cyc The hyperparameter, L cla is the cross entropy loss function, λ cls For L cls The hyperparameter, L ms is the MS loss function, λ ms For L ms The hyperparameter, E x,y is the expected value of z and y, D y (x) is the output of the discriminator D for x given y, For x and and the expected value of z, For about The result of the discriminator D, For the generator G, the input x and The output when For x and source domain y and target domain and the expected value of z, is the style code of the source domain y, is the relationship between x and the target domain and the expected values ​​of two different random sampling noises z1,z2 used to generate style codes, For the generator G, the input x and The output when For the generator G, the input x and The output when and For the same target domain There are two different style codes, M is the mapping network.

[0158] The chip defects generated by the embodiment of the present invention are not limited to the chip defects in the training model, and can generate defects that the simulation defect model does not have during training, making the types of defects richer and making the method of the present invention have better generalization and more application scenarios.

[0159] Figure 7 FIG. 1 is a schematic diagram showing a method for generating a chip simulation defect map according to an embodiment of the present invention. Figure 7 As shown in the figure, by using the noise and target domain code at any time, the style code is generated through the Mapping Network. The simulation defect model receives the good image of a certain chip and controls the defect type through the style code, thereby obtaining a simulated defect image.

[0160] An embodiment of the present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following method when executing the computer program: obtaining an initial model; training the initial model to obtain a simulation defect model; generating a chip simulation defect image based on the simulation defect model; wherein the simulation defect model comprises: an encoder, a skip-layer connector, and a decoder, and generating a chip simulation defect image based on the simulation defect model comprises: the encoder receiving an original chip image, converting the RGB color feature layer of the original chip image into a sequential feature map, and outputting the sequential feature map to the decoder, the skip-layer connector generating a skip-layer feature map based on the RGB color feature layer of the original chip image, and transmitting the skip-layer feature map to the decoder by skipping layers, and the decoder generating the chip simulation defect image based on the sequential feature map, the skip-layer feature map, and a style code.

[0161] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following method: obtaining an initial model; training the initial model to obtain a simulation defect model; generating a chip simulation defect image based on the simulation defect model; wherein the simulation defect model includes: an encoder, a skip-layer connector and a decoder, and generating a chip simulation defect image based on the simulation defect model includes: the encoder receiving an original chip image, converting the RGB color feature layer of the original chip image into a sequential feature map, and outputting the sequential feature map to the decoder; the skip-layer connector generating a skip-layer feature map based on the RGB color feature layer of the original chip image, and skip-layer transmitting the skip-layer feature map to the decoder; the decoder generating the chip simulation defect image based on the sequential feature map, the skip-layer feature map and the style code.

[0162] The above chip simulation defect generation method achieves the beneficial effect of solving the technical problems raised in the background technology.

[0163] Figure 2 FIG. 1 is a flow chart of a chip simulation defect generation method in one embodiment. It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0164] Figure 8 The internal structure diagram of a computer device in one embodiment is shown. The computer device may be Figure 1 The server 120 in Figure 8As shown, the computer device includes a processor, a memory, a network interface, an input device and a display screen connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor may implement a chip simulation defect generation method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor may implement a chip simulation defect generation method. The display screen of the computer device may be a liquid crystal display or an electronic ink display screen. The input device of the computer device may be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0165] Those skilled in the art will understand that Figure 8 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0166] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-described methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0167] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0168] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A chip simulation defect generation method, characterized in that: The method comprises: Get the initial model; Training the initial model to obtain a simulation defect model; generating a chip simulation defect image according to the simulation defect model; The simulation defect model includes: an encoder, a jump layer connector and a decoder. Generating a chip simulation defect image according to the simulation defect model includes: The encoder receives an original chip image, converts the RGB color feature layer of the original chip image into a sequential feature map, and outputs the sequential feature map to the decoder. The layer-skipping connector generates a layer-skipping feature map according to the RGB color feature layer of the original chip image, and transmits the layer-skipping feature map to the decoder by layer-skipping. The decoder generates the chip simulation defect image according to the sequential feature map, the skip-layer feature map and the style code.

2. The method according to claim 1, characterized in that The style code is generated by random noise and target domain code through a mapping network.

3. The method according to claim 1, characterized in that The encoder comprises a plurality of encoding layers, The encoder receives an original chip image, converts an RGB color feature layer of the original chip image into a sequential feature map, and outputs the sequential feature map to the decoder, including: The first encoding layer receives the original chip image, identifies a first type of feature in the RGB color feature layer of the original chip image, converts the RGB color feature layer of the original chip image into a first feature map containing the first type of feature, and outputs the first feature map to the second encoding layer; The second encoding layer identifies the second type of features in the first feature map, converts the first feature map into a second feature map containing the second type of features, and outputs the second feature map to the third encoding layer; The third encoding layer identifies a third type of feature in the second feature map, converts the second feature map into a third feature map containing the third type of feature, and outputs the third feature map to the fourth encoding layer; The fourth encoding layer identifies a fourth type of feature in the third feature map, converts the third feature map into a fourth feature map including the fourth type of feature, and outputs the fourth feature map to the fifth encoding layer; The fifth encoding layer identifies a fifth type of feature in the fourth feature map, converts the fourth feature map into a fifth feature map including the fifth type of feature, and outputs the fifth feature map to the sixth encoding layer; The sixth encoding layer identifies a sixth type of feature in the fifth feature map, converts the fifth feature map into a sixth feature map including the sixth type of feature, and outputs the sixth feature map as the sequential feature map to the decoder.

4. The method according to claim 3, characterized in that The jump layer connectors include a first jump layer connector, a second jump layer connector, a third jump layer connector and a fourth jump layer connector. The layer-skipping connector generates a layer-skipping feature map according to the RGB color feature layer of the original chip image, and transmits the layer-skipping feature map to the decoder by layer-skipping, including: The first jump layer connector copies the first feature map to generate a first jump layer feature map, and jumps the first jump layer feature map to the sixth decoding layer; The second jump layer connector copies the second feature map to generate a second jump layer feature map, and jumps the second jump layer feature map to the fifth decoding layer; The third jump layer connector copies the third feature map to generate a third jump layer feature map, and jumps the third jump layer feature map to the fourth decoding layer; The fourth jump-layer connector copies the fourth feature map to generate a fourth jump-layer feature map, and jumps the fourth jump-layer feature map to the third decoding layer.

5. The method according to claim 4, characterized in that The decoder comprises a plurality of decoding layers, the number of the decoding layers being equal to the number of the encoding layers; The decoder generates the chip simulation defect image according to the sequential feature map, the skip-layer feature map and the style code, including: The first decoding layer generates a first decoding feature map according to the sequential feature map and the first style code; The second decoding layer generates a second decoding feature map based on the first decoding feature map and the second style code; The third decoder generates a third decoding feature map according to the second decoding feature map, the third style code and the fourth skip-layer feature map; The fourth decoder generates a fourth decoding feature map according to the third decoding feature map, the fourth style code and the third skip-layer feature map; The fifth decoder generates a sixth decoding feature map according to the fourth decoding feature map, the fifth style code and the second skip-layer feature map; The sixth decoder generates the chip simulation defect image according to the sixth decoding feature map, the sixth style code and the first jump-layer feature map.

6. The method according to claim 4, characterized in that Each of the coding layers includes a C2F module, and each of the C2F modules includes: a first ConvIN module, three Bottle Neck submodules, and a second ConvIN module; The first encoding layer receives the original chip image, identifies a first type of feature in the RGB color feature layer of the original chip image, converts the RGB color feature layer of the original chip image into a first feature map containing the first type of feature, and outputs the first feature map to the second encoding layer, including: The first ConvIN of the first encoding layer receives the original chip image and performs instance normalization on the original chip image; The three Bottle Neck submodules of the first encoding layer decouple the image features of the RGB color feature layer of the original chip image after instance normalization through multi-channel decoupling, and extract semantic information based on the deep residual network to identify the first type of features; The second ConvIN module of the first encoding layer converts the RGB color feature layer of the original chip image into a first feature map containing the first type of features, and outputs the first feature map to the second encoding layer.

7. The method according to claim 1, characterized in that The training of the initial model to obtain a simulation defect model includes: When the initial model is trained to obtain a simulation defect model, the total loss function L is: Among them, L adv To counter the loss function, L cyc is the cycle-consistent loss function, λ cyc For L cyc The hyperparameter, L cls is the cross entropy loss function, λ cls For L cls The hyperparameter, L ms is the MS loss function, λ ms For L ms The hyperparameter, E x,y is the expected value of x and y, D y (x) is the output of the discriminator D for x given y, For x and and the expected value of z, For about The result of the discriminator D, For the generator G, the input x and The output when For x and source domain y and target domain and the expected value of z, is the style code of the source domain y, is the relationship between x and the target domain and the expected values ​​of two different random sampling noises z1,z2 used to generate style codes, For the generator G, the input x and The output when For the generator G, the input x and The output when and For the same target domain Two different style codes, M is the mapping network.

8. A chip simulation defect generation device, characterized in that: The device comprises: Model unit, used to obtain the initial model; A training unit, configured to train the initial model to obtain a simulation defect model; A simulation unit, configured to generate a chip simulation defect image according to the simulation defect model; The simulation defect model includes: an encoder, a jump layer connector and a decoder. The encoder is used to receive an original chip image, convert the RGB color feature layer of the original chip image into a sequential feature map, and output the sequential feature map to the decoder. The layer-skipping connector is used to generate a layer-skipping feature map according to the RGB color feature layer of the original chip image, and transmit the layer-skipping feature map to the decoder by layer-skipping. The decoder is used to generate the chip simulation defect image according to the sequential feature map, the skip-layer feature map and the style code.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images