Chip layout identification method and device guided by multi-scale frequency domain and computer equipment

Abstract

The invention relates to a multi-scale frequency domain guided chip layout identification method and device and computer equipment. The method comprises the following steps: extracting an initial middle layer feature and an initial high layer feature of a to-be-identified chip layout by a backbone network; projecting the initial intermediate layer features by a self-adaptive multi-scale frequency domain learning device to obtain initial frequency domain features and initial spatial domain features, and performing multi-scale extraction on the initial frequency domain features to obtain multi-scale frequency domain features; performing adaptive enhancement on the initial spatial domain feature by using an attention mechanism to obtain an adaptive enhanced spatial domain local feature, and fusing the multi-scale frequency domain feature and the adaptive enhanced spatial domain local feature to obtain an intermediate-layer multi-scale feature; performing semantic enhancement on the initial high-level features by an adaptive task semantic enhancement module to obtain high-level semantic enhancement features; and performing classification processing on the middle-layer multi-scale features and the high-layer semantic enhancement features by a classifier to obtain a chip identifier. By adopting the method, the identification accuracy of the chip layout can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of semiconductor technology, in particular to a chip layout identification method and device based on multi-scale frequency domain guidance and a computer device. BACKGROUND

[0002] The rampant of counterfeit chips is a major challenge in the semiconductor industry, including: 1, Remarking (relabeling): polishing low-grade, old, and scrapped chips and relabeling them as high-grade and new chips; 2, Cloning: completely copying the design and function of the original chip, but using cheap or unreliable process to manufacture; 3, Counterfeiting: producing completely false chips that may not work at all; 4, Recycling: disassembling chips from waste circuit boards and washing them as new chips. Chip layout identification is used to investigate counterfeit chips, carry out intellectual property analysis, ensure that the actual model of the chip is consistent with the packaging model, and provide an effective method for strict quality control and product traceability of integrated circuits.

[0003] In the traditional technology, chip layout identification is performed by manual detection or deep learning-based integrated circuit defect detection method. However, manual detection relies on experience and may have misjudgment, and the deep learning-based integrated circuit defect detection method may introduce noise or lose information in the feature fusion process, resulting in low accuracy of chip layout identification in the traditional technology. SUMMARY

[0004] Therefore, it is necessary to provide a multi-scale frequency domain guided chip layout identification method, device, computer equipment and readable storage medium capable of improving the accuracy of chip layout identification.

[0005] In a first aspect, the present application provides a multi-scale frequency domain guided chip layout identification method, comprising:

[0006] inputting a chip layout to be identified into a layout identification model, extracting features of the chip layout to be identified by a backbone network in the layout identification model, to obtain initial intermediate layer features and initial high layer features;

[0007] projecting the initial intermediate layer features by an adaptive multi-scale frequency domain learner in the layout identification model to obtain initial frequency domain features and initial spatial domain features, performing multi-scale extraction on the initial frequency domain features to obtain multi-scale frequency domain features, and using an attention mechanism to adaptively enhance the initial spatial domain features to obtain adaptively enhanced spatial domain local features, and fusing the multi-scale frequency domain features and the adaptively enhanced spatial domain local features to obtain intermediate layer multi-scale features;

[0008] The initial high-level feature is semantically enhanced by an adaptive task semantic enhancement module in the layout recognition model to obtain a high-level semantic enhanced feature.

[0009] The intermediate layer multi-scale feature and the high-level semantic enhanced feature are classified by a classifier in the layout recognition model to obtain a chip identification corresponding to the chip layout to be recognized.

[0010] In one of the embodiments, the initial intermediate layer feature is projected by an adaptive multi-scale frequency domain learner in the layout recognition model to obtain an initial frequency domain feature and an initial spatial domain feature, including:

[0011] The initial intermediate layer feature is denoised by the adaptive multi-scale frequency domain learner in the layout recognition model according to the initial high-level feature to obtain a denoised initial intermediate layer feature, and the denoised initial intermediate layer feature is subjected to frequency domain feature extraction and spatial domain feature extraction respectively to obtain the initial frequency domain feature and the initial spatial domain feature.

[0012] In one of the embodiments, the initial intermediate layer feature is denoised by the adaptive multi-scale frequency domain learner in the layout recognition model according to the initial high-level feature to obtain a denoised initial intermediate layer feature, including:

[0013] The initial high-level feature is subjected to interpolation processing by the adaptive multi-scale frequency domain learner in the layout recognition model to obtain an interpolated initial high-level feature, the resolution of the interpolated initial high-level feature being the same as that of the initial intermediate layer feature, and the interpolated initial high-level feature and the initial intermediate layer feature are subjected to element-by-element multiplication to obtain the denoised initial intermediate layer feature.

[0014] In one of the embodiments, the above method further includes:

[0015] The initial frequency domain feature is subjected to spatial domain conversion processing to obtain a first conversion feature.

[0016] The initial spatial domain feature is subjected to frequency domain conversion processing to obtain a second conversion feature, and the initial spatial domain feature is adaptively enhanced based on the first conversion feature using an attention mechanism to obtain an adaptively enhanced spatial domain local feature.

[0017] The multi-scale frequency domain feature is subjected to self-fusion processing to obtain a self-fusion frequency domain feature.

[0018] The second conversion feature and the self-fusion frequency domain feature are fused to obtain a scale enhanced feature.

[0019] The scale enhanced feature and the adaptively enhanced spatial domain local feature are fused to obtain the intermediate layer multi-scale feature.

[0020] In one of the embodiments, the initial spatial domain feature is adaptively enhanced based on the first conversion feature using an attention mechanism to obtain an adaptively enhanced spatial domain local feature, which includes:

[0021] The initial spatial domain feature is pre-processed to obtain a pre-processed initial spatial domain feature;

[0022] The pre-processed initial spatial domain feature is fused with the first conversion feature to obtain a spatial enhancement feature;

[0023] The attention weight of the spatial enhancement feature is extracted to obtain an attention map;

[0024] The attention map and the spatial enhancement feature are weighted to obtain the adaptively enhanced spatial domain local feature.

[0025] In one of the embodiments, the initial high-level feature is semantically enhanced by an adaptive task semantic enhancement module in the layout recognition model to obtain a high-level semantic enhancement feature, which includes:

[0026] The preset chip prompt information is fused with the initial high-level feature by a task adaptive semantic enhancement module in the layout recognition model to obtain a prompt embedding feature, and the prompt embedding feature is attention enhanced and semantically enhanced to obtain the high-level semantic enhancement feature.

[0027] In one of the embodiments, the initial intermediate layer feature includes a first intermediate layer feature and a second intermediate layer feature, the adaptive multi-scale frequency domain learner includes a first adaptive multi-scale frequency domain learner and a second adaptive multi-scale frequency domain learner, the initial frequency domain feature includes a first frequency domain feature and a second frequency domain feature, the initial spatial domain feature includes a first spatial domain feature and a second spatial domain feature, the multi-scale frequency domain feature includes a first multi-scale feature and a second multi-scale feature, the adaptively enhanced spatial domain local feature includes a first enhanced spatial domain local feature and a second enhanced spatial domain local feature, and the intermediate layer multi-scale feature includes a first target feature and a second target feature;

[0028] The initial intermediate layer feature is projected by an adaptive multi-scale frequency domain learner in the layout recognition model to obtain the initial frequency domain feature and the initial spatial domain feature, the multi-scale frequency domain feature is extracted from the initial frequency domain feature in the frequency domain, the initial spatial domain feature is adaptively enhanced using an attention mechanism to obtain the adaptively enhanced spatial domain local feature, and the multi-scale frequency domain feature and the adaptively enhanced spatial domain local feature are fused to obtain the intermediate layer multi-scale feature, which includes:

[0029] projecting, by a first adaptive multi-scale frequency domain learner in the layout recognition model, the first intermediate layer feature to obtain a first frequency domain feature and a first spatial domain feature, performing multi-scale extraction on the first frequency domain feature in the frequency domain to obtain a first multi-scale feature, performing adaptive enhancement on the first spatial domain feature using an attention mechanism to obtain a first enhanced spatial domain local feature, and fusing the first multi-scale feature and the first enhanced spatial domain local feature to obtain a first target feature;

[0030] projecting, by a second adaptive multi-scale frequency domain learner in the layout recognition model, the second intermediate layer feature to obtain a second frequency domain feature and a second spatial domain feature, performing multi-scale extraction on the second frequency domain feature in the frequency domain to obtain a second multi-scale feature, performing adaptive enhancement on the second spatial domain feature using an attention mechanism to obtain a second enhanced spatial domain local feature, and fusing the second multi-scale feature and the second enhanced spatial domain local feature to obtain a second target feature.

[0031] In one of the embodiments, the intermediate layer multi-scale feature and the high layer semantic enhanced feature are classified by a classifier in the layout recognition model to obtain a chip identification corresponding to the chip layout to be identified, including:

[0032] The intermediate layer multi-scale feature and the high layer semantic enhanced feature are respectively classified by a classifier in the layout recognition model to obtain a first classification result corresponding to the intermediate layer multi-scale feature and a second classification result corresponding to the high layer semantic enhanced feature; and the first classification result and the second classification result are weighted and summed to obtain a chip identification corresponding to the chip layout to be identified.

[0033] In a second aspect, the present application further provides a multi-scale frequency domain guided chip layout recognition device, including:

[0034] An initial feature extraction module is configured to input the chip layout to be identified into a layout recognition model, and extract features of the chip layout to be identified by a backbone network in the layout recognition model to obtain initial intermediate layer features and initial high layer features.

[0035] A multi-scale feature determination module is configured to project the initial intermediate layer features by an adaptive multi-scale frequency domain learner in the layout recognition model to obtain initial frequency domain features and initial spatial domain features, perform multi-scale extraction on the initial frequency domain features in the frequency domain to obtain multi-scale frequency domain features, perform adaptive enhancement on the initial spatial domain features using an attention mechanism to obtain adaptively enhanced spatial domain local features, and fuse the multi-scale frequency domain features and the adaptively enhanced spatial domain local features to obtain intermediate layer multi-scale features.

[0036] A semantic enhancement module is configured to perform semantic enhancement on the initial high layer features by an adaptive task semantic enhancement module in the layout recognition model to obtain high layer semantic enhanced features.

[0037] a chip classification module, configured to perform classification processing on the intermediate layer multi-scale feature and the high layer semantic enhanced feature by a classifier in the layout recognition model, to obtain a chip identification corresponding to the chip layout to be recognized.

[0038] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0039] inputting the chip layout to be recognized into the layout recognition model, extracting features of the chip layout to be recognized by a backbone network in the layout recognition model, to obtain initial intermediate layer features and initial high layer features;

[0040] projecting the initial intermediate layer features by an adaptive multi-scale frequency domain learner in the layout recognition model to obtain initial frequency domain features and initial spatial domain features, performing multi-scale extraction on the initial frequency domain features to obtain multi-scale frequency domain features, adaptively enhancing the initial spatial domain features using an attention mechanism to obtain adaptively enhanced spatial domain local features, and fusing the multi-scale frequency domain features and the adaptively enhanced spatial domain local features to obtain the intermediate layer multi-scale features;

[0041] performing semantic enhancement on the initial high layer features by an adaptive task semantic enhancement module in the layout recognition model to obtain high layer semantic enhanced features;

[0042] performing classification processing on the intermediate layer multi-scale feature and the high layer semantic enhanced feature by a classifier in the layout recognition model, to obtain a chip identification corresponding to the chip layout to be recognized.

[0043] In a fourth aspect, the present application further provides a computer readable storage medium, having a computer program stored thereon, the computer program being executed by a processor to implement the following steps:

[0044] inputting the chip layout to be recognized into the layout recognition model, extracting features of the chip layout to be recognized by a backbone network in the layout recognition model, to obtain initial intermediate layer features and initial high layer features;

[0045] projecting the initial intermediate layer features by an adaptive multi-scale frequency domain learner in the layout recognition model to obtain initial frequency domain features and initial spatial domain features, performing multi-scale extraction on the initial frequency domain features to obtain multi-scale frequency domain features, adaptively enhancing the initial spatial domain features using an attention mechanism to obtain adaptively enhanced spatial domain local features, and fusing the multi-scale frequency domain features and the adaptively enhanced spatial domain local features to obtain the intermediate layer multi-scale features;

[0046] The adaptive task semantic enhancement module in the map recognition model performs semantic enhancement on the initial high-level features to obtain high-level semantically enhanced features;

[0047] The classifier in the layout recognition model classifies the intermediate multi-scale features and the high-level semantic enhancement features to obtain the chip identifier corresponding to the chip layout to be identified.

[0048] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0049] The chip layout to be identified is input into the layout recognition model, and the backbone network in the layout recognition model extracts the features of the chip layout to be identified, thus obtaining the initial intermediate layer features and the initial high layer features.

[0050] The adaptive multi-scale frequency domain learner in the map recognition model projects the initial intermediate layer features to obtain the initial frequency domain features and the initial spatial domain features. The initial frequency domain features are then extracted at multiple scales to obtain multi-scale frequency domain features. The initial spatial domain features are then adaptively enhanced using an attention mechanism to obtain adaptively enhanced local spatial domain features. Finally, the multi-scale frequency domain features and the adaptively enhanced local spatial domain features are fused to obtain the intermediate layer multi-scale features.

[0051] The adaptive task semantic enhancement module in the map recognition model performs semantic enhancement on the initial high-level features to obtain high-level semantically enhanced features;

[0052] The classifier in the layout recognition model classifies the intermediate multi-scale features and the high-level semantic enhancement features to obtain the chip identifier corresponding to the chip layout to be identified.

[0053] The aforementioned multi-scale frequency domain-guided chip layout recognition method, apparatus, computer equipment, and readable storage medium extract multi-scale frequency domain features from the chip layout to be recognized, resulting in richer feature representation. Adaptive enhancement of the initial spatial domain features using an attention mechanism can saliency, structural strengthening, and semantic equalization of the initial spatial domain features, thereby improving their feature quality and semantic expressive power, making subsequent frequency domain modeling and cross-domain fusion more accurate and stable. Fusing multi-scale frequency domain features with adaptively enhanced spatial domain features improves the accuracy of chip layout recognition, solving the problem of low accuracy caused by insufficient feature extraction. Semantic enhancement of the initial high-level features through an adaptive task semantic enhancement module can more fully utilize contextual semantic information, enhancing the model's generalization ability. Attached Figure Description

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application or the related art. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained from these drawings without creative labor.

[0055] Figure 1 A flowchart of a chip layout recognition method based on multi-scale frequency domain guidance in an embodiment is shown in the figure.

[0056] Figure 2 A flowchart of processing and fusing the initial spatial domain feature and the multi-scale frequency domain feature in an embodiment is shown in the figure.

[0057] Figure 3 A flowchart of a chip layout recognition method based on multi-scale frequency domain guidance in another embodiment is shown in the figure.

[0058] Figure 4 A block diagram of a chip layout recognition device based on multi-scale frequency domain guidance in an embodiment is shown in the figure.

[0059] Figure 5 An internal structure diagram of a computer device in an embodiment is shown in the figure. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the present application more clear, the following will further describe the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0061] It should be noted that the terms "first", "second", etc. used in the present application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "include" and "have" and any variations thereof used in the present application are intended to cover non-exclusive inclusion. The term "a plurality of" used in the present application means two or more.

[0062] In an exemplary embodiment, as shown in Figure 1 A chip layout recognition method based on multi-scale frequency domain guidance is provided, and the embodiment takes the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is realized through the interaction of the terminal and the server. In the embodiment, the method includes the following steps. Wherein:

[0063] Step 102, input the chip layout to be identified into the layout recognition model, and extract the features of the chip layout to be identified by the backbone network in the layout recognition model to obtain initial intermediate layer features and initial high layer features.

[0064] The chip layout is also referred to as an integrated circuit physical design diagram, which refers to a two-dimensional geometric graph set of transistors, resistors, capacitors, metal interconnections, vias, substrates, etc. drawn on a silicon wafer according to the circuit design schematic diagram, according to the level, size and spatial position of the manufacturing process. The layout recognition model refers to a model for recognizing the chip identification corresponding to the chip layout obtained by training a large number of chip layout recognition data sets.

[0065] In some embodiments, the terminal receives a chip layout recognition instruction, parses the chip layout recognition instruction to obtain a chip layout to be identified. The chip layout to be identified is input into a pre-constructed layout recognition model, and the features of the chip layout to be identified are extracted by the backbone network in the layout recognition model to obtain initial intermediate layer features and initial high layer features. The backbone network can be a convolutional neural network. The initial intermediate layer features refer to the features extracted by the intermediate layer of the backbone network, which include edge, corner, texture and shape information. The initial high layer features refer to the features extracted by the high layer of the backbone network, which include rich semantic information.

[0066] Further, the above method further includes a training process of the layout recognition model. A chip layout recognition data set is constructed according to the chip identification and the corresponding chip layout, the chip layout recognition data set is input into an AI (Automatic Intelligence) recognition model to obtain a model loss value, the model parameters of the AI recognition model are trained according to the model loss value until the model loss value no longer decreases, and the current AI recognition model is taken as the layout recognition model. After obtaining the trained layout recognition model, a test site is deployed to test the layout recognition model. The chip layout to be tested which does not participate in the training is input into the layout recognition model to obtain the chip identification of the chip layout to be tested, and whether the chip identification of the chip layout to be tested is consistent with the chip identification information provided by the manufacturer is identified. If consistent, it indicates that the accuracy, generalization, stability and reliability of the layout recognition model are good, which can ensure that the layout recognition model can correctly, stably and efficiently complete the chip layout recognition task in real application.

[0067] Step 104, project the initial intermediate layer features by the self-adaptive multi-scale frequency domain learner in the layout recognition model to obtain initial frequency domain features and initial spatial domain features, perform multi-scale extraction on the initial frequency domain features in the frequency domain to obtain multi-scale frequency domain features, use the attention mechanism to perform self-adaptive enhancement on the initial spatial domain features to obtain self-adaptive enhanced spatial domain local features, and fuse the multi-scale frequency domain features and the self-adaptive enhanced spatial domain local features to obtain intermediate layer multi-scale features.

[0068] wherein the adaptive multi-scale frequency learner is referred to as AMFL (Adaptive multiscale Frequency Learner).

[0069] In some embodiments, the adaptive multi-scale frequency learner comprises a first convolution module, a second convolution module, a MSST (Multi-Scale Spectral Transformer), and a frequency and spatial domain fuser. The initial intermediate layer features are projected into frequency domain by the first convolution module to obtain initial frequency domain features. The initial intermediate layer features are projected into spatial domain by the second convolution module to obtain initial spatial domain features. The initial frequency domain features are extracted into multi-scale features in a single layer by the MSST, and the initial spatial domain features are adaptively enhanced by using an attention mechanism to obtain adaptively enhanced spatial domain local features. The multi-scale frequency domain features and the adaptively enhanced spatial domain local features are fused by the frequency and spatial domain fuser to obtain intermediate layer multi-scale features.

[0070] At step 106, the initial high layer features are semantically enhanced by an adaptive task semantic enhancement module in the layout recognition model to obtain high layer semantic enhanced features.

[0071] At step 108, the intermediate layer multi-scale features and the high layer semantic enhanced features are classified by a classifier in the layout recognition model to obtain a chip identification corresponding to the chip layout to be recognized.

[0072] In some embodiments, the initial high layer features are semantically enhanced by the adaptive task semantic enhancement module to obtain high layer semantic enhanced features. The intermediate layer multi-scale features and the high layer semantic enhanced features are classified by a classifier in the layout recognition model to obtain a first classification result corresponding to the intermediate layer multi-scale features and a second classification result corresponding to the high layer semantic enhanced features. The first classification result and the second classification result are weighted and summed by the layout recognition model to obtain a chip identification corresponding to the chip layout to be recognized. For example, the chip identification can be a chip model.

[0073] In the conventional technology, chip layout recognition is performed by manual detection or a deep learning-based integrated circuit defect detection method. However, manual detection relies on experience and may have misjudgments. In the feature fusion process of the deep learning-based integrated circuit defect detection method, the feature spatial resolutions of different layers are made the same by upsampling or downsampling to realize fusion of features of different layers. However, upsampling introduces redundancy and noise, and downsampling causes information loss, resulting in the problem of low chip layout recognition accuracy in the conventional technology.

[0074] In the chip layout recognition method in the multi-scale frequency domain guided above, the multi-scale frequency domain features of the chip layout to be recognized are extracted, so that the feature expression is more abundant. The attention mechanism is used to adaptively enhance the initial spatial domain features, which can highlight the initial spatial domain features, strengthen the structure and balance the semantics, so as to improve the feature quality and semantic expression ability, and make the subsequent frequency domain modeling and cross-domain fusion more accurate and stable. The multi-scale frequency domain features and the adaptively enhanced spatial domain features are fused, which improves the chip layout recognition accuracy and solves the problem of low chip layout recognition accuracy caused by insufficient feature extraction. By performing semantic enhancement on the initial high-level features, the context semantic information can be more fully utilized, and the generalization ability of the model recognition is enhanced.

[0075] In an exemplary embodiment, the projecting, by the adaptive multi-scale frequency domain learner in the layout recognition model, the initial intermediate layer features to obtain the initial frequency domain features and the initial spatial domain features comprises: the adaptive multi-scale frequency domain learner in the layout recognition model denoises the initial intermediate layer features according to the initial high-level features to obtain denoised initial intermediate layer features, and respectively performs frequency domain feature extraction and spatial domain feature extraction on the denoised initial intermediate layer features to obtain the initial frequency domain features and the initial spatial domain features.

[0076] In some embodiments, the adaptive multi-scale frequency domain learner further includes a noise filter (Noise Fliter), and the initial intermediate layer features and the initial high-level features output by the backbone network are input to the noise filter. The noise filter denoises the initial intermediate layer features according to the initial high-level features to obtain denoised initial intermediate layer features. The second convolution module and the second convolution module in the adaptive multi-scale frequency domain learner respectively perform frequency domain feature extraction and spatial domain feature extraction on the denoised initial intermediate layer features to obtain the initial frequency domain features and the initial spatial domain features.

[0077] In the embodiments of the present application, since the initial high-level features contain rich semantic information and the initial intermediate layer features have a large amount of background noise, the initial intermediate layer features are denoised using the semantic information of the high-level to remove the background noise in the initial intermediate layer features, so that the initial intermediate layer features mainly focus on the representation of the target object in the subsequent processing process.

[0078] In an example embodiment, the denoising of the initial intermediate layer feature based on the initial high layer feature by the adaptive multi-scale frequency domain learner in the layout recognition model comprises: performing interpolation processing on the initial high layer feature by the adaptive multi-scale frequency domain learner in the layout recognition model to obtain interpolated initial high layer feature, the resolution of the interpolated initial high layer feature being the same as that of the initial intermediate layer feature, and multiplying the interpolated initial high layer feature and the initial intermediate layer feature element by element to obtain the denoised initial intermediate layer feature.

[0079] The noise filter first performs interpolation processing on the initial high layer feature to obtain interpolated initial high layer feature. The resolution of the interpolated initial high layer feature is the same as that of the initial intermediate layer feature. Then, the convolution layer in the noise filter extracts context information from the interpolated initial high layer feature to determine the noise region and the target region. Then, the sigmoid activation function (smooth, derivable S-shaped function) in the noise filter is used to normalize the context information to obtain the weight mask. Then, the weight mask and the corresponding initial intermediate layer feature are multiplied element by element to adjust the importance of each pixel in the initial intermediate layer feature through the weight mask, so as to remove the background noise.

[0080] In the embodiments of the present application, since the resolution of the initial high layer feature is smaller than that of the initial intermediate layer feature, the resolution of the initial high layer feature is made the same as that of the initial intermediate layer feature by performing interpolation processing on the initial high layer feature, so that the initial high layer feature and the initial intermediate layer feature are aligned in the spatial dimension, so as to subsequently perform pixel-level denoising on the initial intermediate layer feature. The context information is extracted from the interpolated initial high layer feature to determine the noise region and the target region, and the sigmoid activation function (smooth, derivable S-shaped function) is used to normalize the context information to obtain the weight mask, which represents the judgment of the noise filter on the target region at some spatial position. The weight mask and the corresponding initial intermediate layer feature are multiplied element by element, which can adjust the importance of each pixel in the initial intermediate layer feature through the weight mask, so as to effectively remove the background noise in the initial intermediate layer feature.

[0081] In an example embodiment, as shown in Figure 2 The above method further comprises the step of processing and fusing the initial spatial domain feature and the multi-scale frequency domain feature, which comprises:

[0082] In step 202, the initial frequency domain feature is processed in the spatial domain to obtain a first converted feature.

[0083] At step 204, the initial spatial domain feature is subjected to frequency domain conversion processing to obtain a second conversion feature, and the initial spatial domain feature is adaptively enhanced based on the first conversion feature using an attention mechanism to obtain an adaptively enhanced spatial domain local feature.

[0084] At step 206, the multi-scale frequency domain feature is subjected to self-fusion processing to obtain a self-fusion frequency domain feature.

[0085] At step 208, the second conversion feature and the self-fusion frequency domain feature are fused to obtain a scale-enhanced feature.

[0086] At step 210, the scale-enhanced feature and the adaptively enhanced spatial domain local feature are fused to obtain an intermediate layer multi-scale feature.

[0087] The adaptive multi-scale frequency domain learner further includes a frequency domain to spatial domain conversion function, an aggregation module, a spatial domain to frequency domain conversion function, and a spatial domain operation function.

[0088] While the initial frequency domain feature is subjected to multi-scale extraction by the MSST to obtain a multi-scale frequency domain feature, the initial frequency domain feature is subjected to spatial domain conversion processing by the frequency domain to spatial domain conversion function to obtain a first conversion feature. The multi-scale frequency domain feature is subjected to self-fusion processing by the aggregation module to integrate the frequency domain features of different scales into a unified fusion feature representation to obtain a self-fusion frequency domain feature.

[0089] The initial spatial domain feature is subjected to frequency domain conversion processing by the spatial domain to frequency domain conversion function to obtain a second conversion feature, and the initial spatial domain feature is adaptively enhanced based on the spatial domain operation function and the first conversion feature using an attention mechanism to obtain an adaptively enhanced spatial domain local feature. The initial spatial domain feature is adaptively enhanced based on the first conversion feature using the attention mechanism to introduce frequency domain features in the spatial domain to achieve feature complementarity.

[0090] The second conversion feature and the self-fusion frequency domain feature are fused to obtain a scale-enhanced feature. Since the self-fusion frequency domain feature only fuses the multi-scale frequency domain features of the frequency domain branch, it lacks spatial domain local information, while the features of the spatial domain branch have rich local information. By introducing the second conversion feature of the spatial domain branch into the self-fusion frequency domain feature, the local detail information of the frequency domain branch can be supplemented, thereby further enriching the scale information and increasing the capture of different scale features.

[0091] Then, the scale enhancement feature is fused with the self-adaptively enhanced spatial domain local feature to obtain an intermediate layer multi-scale feature.

[0092] In the embodiment of the present application, the initial frequency domain feature is subjected to spatial domain conversion processing to obtain a first conversion feature, so that the first conversion feature is adapted to the spatial domain branch, and the first conversion feature is introduced into the initial spatial domain feature in the subsequent process to realize feature complementation. The initial spatial domain feature is self-adaptively enhanced based on the first conversion feature using the attention mechanism, which can highlight, structure strengthen and balance the semantics of the initial spatial domain feature, so as to improve the feature quality and semantic expression ability thereof, and make the subsequent frequency domain modeling and cross-domain fusion more accurate and stable. The initial spatial domain feature is subjected to frequency domain conversion processing to obtain a second conversion feature, so that the second conversion feature is adapted to the frequency domain branch, so as to introduce the second conversion feature into the self-fused frequency domain feature in the subsequent process, to further enrich the scale information and increase the capture of different scale features. The multi-scale frequency domain features are subjected to self-fusion processing, which can realize adaptive interaction and weight balance between different scale features within the frequency domain, and finally obtain self-fused frequency domain features with consistent semantics, complete structure and noise suppression. The scale enhancement feature is fused with the self-adaptively enhanced spatial domain local feature, which can obtain more accurate and rich intermediate layer multi-scale features.

[0093] In one exemplary embodiment, the initial spatial domain feature is self-adaptively enhanced based on the first conversion feature using the attention mechanism to obtain the self-adaptively enhanced spatial domain local feature, including: performing feature preprocessing on the initial spatial domain feature to obtain a preprocessed initial spatial domain feature; fusing the preprocessed initial spatial domain feature with the first conversion feature to obtain a spatial enhancement feature; extracting attention weights from the spatial enhancement feature to obtain an attention map; and performing weighted processing on the attention map and the spatial enhancement feature to obtain the self-adaptively enhanced spatial domain local feature.

[0094] The adaptive multi-scale frequency domain learner further includes an average pooling layer, a maximum pooling layer and a third convolution module.

[0095] The initial spatial domain feature is preprocessed by a spatial domain operation function in the adaptive multi-scale frequency domain learner to obtain a preprocessed initial spatial domain feature. The feature preprocessing can include convolution / activation / weighting operations on the initial spatial domain feature to achieve significant region enhancement and noise suppression. The preprocessed initial spatial domain feature is fused with the first converted feature to introduce frequency domain features into the preprocessed initial spatial domain feature and obtain a spatial enhancement feature. The spatial enhancement feature is statistically processed by an average pooling layer and a maximum pooling layer to output an average response map and a maximum response map. The average response map and the maximum response map are spliced and processed by a third convolution module and a sigmoid activation function to obtain an attention map. The attention map and the spatial enhancement feature are weighted to obtain an adaptively enhanced spatial domain local feature.

[0096] In the embodiments of the present application, the initial spatial domain feature is preprocessed to further reduce background interference, provide clean input for subsequent fusion, and enable the model to focus more on the target region rather than noise. The preprocessed initial spatial domain feature is fused with the first converted feature to combine spatial domain details with frequency domain global structure information, achieve complementary enhancement of space and spectrum, obtain spatial enhancement features with both details and global consistency, and compensate for the lack of global semantic modeling in the spatial domain. Attention weights are extracted from the spatial enhancement feature to obtain an attention map, enabling the model to automatically focus on significant regions (such as target edges and key textures) and suppress useless regions (such as backgrounds and shadows), and improve the discriminability and semantic consistency of feature representation. Weighting the attention map and the spatial enhancement feature can use the attention map as a weight to redistribute the spatial enhancement feature, make the response of significant regions stronger and the response of non-important regions weaker, improve the contrast and saliency of the feature, and form the final adaptively enhanced spatial domain local feature, which can also improve the accuracy of subsequent classification tasks.

[0097] In one exemplary embodiment, the initial high-level feature is semantically enhanced by the adaptive task semantic enhancement module in the layout recognition model to obtain a high-level semantic enhancement feature, including: the task adaptive semantic enhancement module in the layout recognition model fuses the preset chip prompt information with the initial high-level feature to obtain prompt embedding features, and performs attention enhancement and semantic enhancement on the prompt embedding features to obtain a high-level semantic enhancement feature.

[0098] The layout recognition model further includes a fourth convolution module and a task adaptive semantic enhancement module, which is referred to as TASE (Task-Adaptive Semantic Enhancement). The task adaptive semantic enhancement module includes a base prompt module, an attention module, and a fifth convolution module.

[0099] The initial high-level feature is input to the fourth convolutional module, and the initial high-level feature is refined in semantics, smoothed in structure, and aligned in feature space by the fourth convolutional module to obtain a refined semantic feature. The preset chip prompt information output by the basic prompt module is added to the refined semantic feature element by element by a task-adaptive semantic enhancement module in the layout recognition model to obtain prompt embedded features. For example, the chip prompt information can include vendor style features, design rule templates, and the like. The prompt embedded features are enhanced in attention and semantics by an attention module and a fifth convolutional module to obtain high-level semantic enhanced features.

[0100] In the embodiments of the present application, the initial high-level feature is enhanced using a learnable prompt mechanism and an attention mechanism, so as to more fully utilize the context semantic information and enhance the generalization ability of the model recognition.

[0101] In one exemplary embodiment, the initial intermediate-level feature includes a first intermediate-level feature and a second intermediate-level feature, the adaptive multi-scale frequency domain learner includes a first adaptive multi-scale frequency domain learner and a second adaptive multi-scale frequency domain learner, the initial frequency domain feature includes a first frequency domain feature and a second frequency domain feature, the initial spatial domain feature includes a first spatial domain feature and a second spatial domain feature, the multi-scale frequency domain feature includes a first multi-scale feature and a second multi-scale feature, the adaptively enhanced spatial domain local feature includes a first adaptively enhanced spatial domain local feature and a second adaptively enhanced spatial domain local feature, and the intermediate-level multi-scale feature includes a first target feature and a second target feature.

[0102] The initial intermediate-level feature is projected by an adaptive multi-scale frequency domain learner in the layout recognition model to obtain an initial frequency domain feature and an initial spatial domain feature, the initial frequency domain feature is extracted in multi-scale in the frequency domain to obtain a multi-scale frequency domain feature, the initial spatial domain feature is adaptively enhanced using an attention mechanism to obtain an adaptively enhanced spatial domain local feature, and the multi-scale frequency domain feature and the adaptively enhanced spatial domain local feature are fused to obtain an intermediate-level multi-scale feature, including:

[0103] The first intermediate layer feature is projected by a first adaptive multi-scale frequency domain learner in the layout recognition model to obtain a first frequency domain feature and a first spatial domain feature, the first frequency domain feature is extracted in the frequency domain to obtain a first multi-scale feature, the first spatial domain feature is adaptively enhanced using an attention mechanism to obtain a first enhanced spatial domain local feature, and the first multi-scale feature and the first enhanced spatial domain local feature are fused to obtain a first target feature.

[0104] In some embodiments, the backbone network can include four network layers, and the features output by the second-to-last layer and the third-to-last layer are referred to as initial intermediate layer features. The features output by the second-to-last layer are taken as the second intermediate layer feature, and the features output by the third-to-last layer are taken as the first intermediate layer feature. The features output by the last layer are taken as initial high layer features. The adaptive multi-scale frequency domain learner includes a first adaptive multi-scale frequency domain learner and a second adaptive multi-scale frequency domain learner. The first intermediate layer feature and the initial high layer feature are input to the first adaptive multi-scale frequency domain learner, and the second intermediate layer feature and the initial high layer feature are input to the second adaptive multi-scale frequency domain learner.

[0105] The first intermediate layer feature is projected by a first adaptive multi-scale frequency domain learner in the layout recognition model to obtain a first frequency domain feature and a first spatial domain feature, the first frequency domain feature is extracted in the frequency domain to obtain a first multi-scale feature, the first spatial domain feature is adaptively enhanced using an attention mechanism to obtain a first enhanced spatial domain local feature, and the first multi-scale feature and the first enhanced spatial domain local feature are fused to obtain a first target feature.

[0106] The second intermediate layer feature is projected by a second adaptive multi-scale frequency domain learner in the layout recognition model to obtain a second frequency domain feature and a second spatial domain feature, the second frequency domain feature is extracted in the frequency domain to obtain a second multi-scale feature, the second spatial domain feature is adaptively enhanced using an attention mechanism to obtain a second enhanced spatial domain local feature, and the second multi-scale feature and the second enhanced spatial domain local feature are fused to obtain a second target feature.

[0107] In the embodiments of the present application, by inputting the intermediate layer features of different levels into different frequency domain learners for processing, the detailed information and global semantic information of different levels can be fully utilized, so that the intermediate layer multi-scale features with stronger semantic expression and structure recognition ability are obtained.

[0108] In an exemplary embodiment, the intermediate layer multi-scale features and the high-level semantic enhanced features are classified by the classifier in the layout recognition model to obtain the chip identification corresponding to the chip layout to be recognized, including: the classifier in the layout recognition model respectively classifies the intermediate layer multi-scale features and the high-level semantic enhanced features to obtain a first classification result corresponding to the intermediate layer multi-scale features and a second classification result corresponding to the high-level semantic enhanced features, and performs weighted sum processing on the first classification result and the second classification result to obtain the chip identification corresponding to the chip layout to be recognized.

[0109] In some embodiments, the intermediate layer multi-scale features include first target features and second target features. The first classification result includes a classification result corresponding to the first target features and a classification result corresponding to the second target features. The classifier in the layout recognition model includes a first classifier, a second classifier, and a third classifier. The first target features are classified by the first classifier to obtain the classification result corresponding to the first target features. The second target features are classified by the second classifier to obtain the classification result corresponding to the second target features.

[0110] The high-level semantic enhanced features are classified by the third classifier to obtain a third classification result. The first classification weight, the second classification weight, and the third classification weight are obtained, the classification result corresponding to the first target features is weighted by the first classification weight, the classification result corresponding to the second target features is weighted by the second classification weight, and the third classification result is weighted by the third classification weight, and the sum is processed to obtain the chip identification corresponding to the chip layout to be recognized.

[0111] In the embodiments of the present application, by classifying and weighting the intermediate layer multi-scale features and the high-level semantic enhanced features, the dual use of detailed information and semantic information can be realized, and the precision, stability, and generalization ability of the model in chip layout recognition can be improved.

[0112] In an exemplary embodiment, as shown in Figure 3 a multi-scale frequency domain guided chip layout recognition method is provided, including:

[0113] An input chip layout to be identified (Input) is input into a layout identification model. The layout identification model includes a backbone network, an adaptive multi-scale frequency domain learner, a task adaptive semantic enhancement module network, a classifier, and a fourth convolution module (Conv4). The adaptive multi-scale frequency domain learner includes a first adaptive multi-scale frequency domain learner and a second adaptive multi-scale frequency domain learner. The task adaptive semantic enhancement module includes a base prompt module (Base Prompt), an attention module (Attention), and a fifth convolution module (Conv5). The classifier includes a first classifier, a second classifier, and a third classifier. The first adaptive multi-scale frequency domain learner includes a first convolution module (Conv1), a second convolution module (Conv2), an MSST, a frequency domain and spatial domain fusioner, a frequency domain to spatial domain conversion function (h fs ), an aggregation module, a spatial domain to frequency domain conversion function (h sf ), a spatial domain operation function (h s ), an average pooling layer, a maximum pooling layer, and a third convolution module (Conv3). The second adaptive multi-scale frequency domain learner has the same structure as the first adaptive multi-scale frequency domain learner.

[0114] The chip layout to be identified is projected by the backbone network in the layout identification model to obtain initial middle layer features and initial high layer features (F ). The initial middle layer features include first middle layer features (F ) and second middle layer features (F ).

[0115] The first middle layer features (F ) and the initial high layer features (F ) are input into a noise filter (Noise Fliter) in the first adaptive multi-scale frequency domain learner to perform interpolation processing on the initial high layer features (F ) to obtain interpolated initial high layer features. The interpolated initial high layer features have the same resolution as the initial middle layer features, and the interpolated initial high layer features are multiplied element by element with the first middle layer features (F ) to obtain denoised first middle layer features.

[0116] The denoised first middle layer features are projected in the frequency domain by a first convolution module (Conv1) to obtain first frequency domain features (F ). The first frequency domain features (F ) are converted in the spatial domain by a frequency domain to spatial domain conversion function (h fs ) to obtain first conversion features. The first frequency domain features (F Multi-scale extraction is performed to obtain the first multi-scale feature. The aggregation module then performs self-fusion processing on the first multi-scale feature to obtain the self-fused frequency domain feature.

[0117] The spatial domain features are extracted from the denoised first intermediate layer features using the second convolutional module (Conv2) to obtain the first spatial domain features. ). Through the spatial domain to frequency domain transformation function (h sf ) for the features of the first spatial domain ( The frequency domain transformation is performed to obtain the second transformation feature, and then based on the spatial domain operation function (h) s ) for the features of the first spatial domain ( Feature preprocessing is performed to obtain the preprocessed first spatial domain features. The preprocessed first spatial domain features are then fused with the first transformed features to obtain spatial augmentation features. Average pooling and max pooling layers are used to statistically analyze the global and local response information of the spatial augmentation features, respectively, outputting an average response map and a maximum response map. These two maps are concatenated and then passed through a third convolutional module (Conv3) and a sigmoid activation function (σ) to obtain an attention map. The attention map and the spatial augmentation features are then weighted to obtain adaptively augmented first spatial domain local features.

[0118] The second transformed feature is fused with the self-fused frequency domain feature to obtain the scale-enhanced feature. The scale-enhanced feature is then fused with the enhanced first spatial domain feature to obtain the first target feature. ).

[0119] The second intermediate layer features ( ) and initial high-level features ( The input is fed into a second adaptive multi-scale frequency domain learner for processing, and the output is the second target feature. The second adaptive multi-scale frequency domain learner processes the same process as the first adaptive multi-scale frequency domain learner, and will not be described again here.

[0120] The initial high-level features ( The input is fed into the fourth convolutional module (Conv4), which processes the initial high-level features (...). Semantic refinement, structural smoothing, and feature space alignment are performed to obtain semantically refined features. The preset chip prompt information output by the Base Prompt module (…) is then used to refine these features. The elements are added together with the semantically refined features to obtain the cue embedding features. ). The prompt embedding feature (x 1) is subjected to attention enhancement and semantic enhancement by an attention module (Attention) and a fifth convolution module (Conv5) to obtain a high-level semantic enhanced feature (x 2).

[0121] The first target feature (x 1) is classified by a first classifier to obtain a classification result corresponding to the first target feature (x 1). The second target feature (x 2) is classified by a second classifier to obtain a classification result corresponding to the second target feature (x 2). The high-level semantic enhanced feature (x 2) is classified by a third classifier to obtain a third classification result. A preset first classification weight (w 1), a second classification weight (w 2) and a third classification weight (w 3) are obtained, the classification result corresponding to the first target feature (x 1) is subjected to weighted operation by the first classification weight (w 1), the classification result corresponding to the second target feature (x 2) is subjected to weighted operation by the second classification weight (w 2), and the third classification result is subjected to weighted operation by the third classification weight (w 3), and the sum is processed to obtain a chip identifier (Output) corresponding to the chip layout to be identified.

[0122] In the embodiments of the present application, the multi-scale frequency domain features of the chip layout to be identified are extracted, so that the feature expression is more rich. The initial spatial domain feature is adaptively enhanced using the attention mechanism, which can highlight, structure strengthen and balance the semantics of the initial spatial domain feature, so as to improve the feature quality and semantic expression ability, and make the subsequent frequency domain modeling and cross-domain fusion more accurate and stable. The multi-scale frequency domain features and the adaptively enhanced spatial domain features are fused, which improves the chip layout recognition accuracy and solves the problem of low chip layout recognition accuracy caused by insufficient feature extraction. The semantic enhancement of the initial high-level feature can make full use of the context semantic information and enhance the generalization ability of the model recognition.

[0123] ​​​​​​​​​​​​​​​It should be understood that although each step in the flowchart involved in the above-described embodiments is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in the above-described embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by the combination are within the scope of protection of the present application.

[0124] Based on the same inventive concept, the embodiments of the present application also provide a multi-scale frequency domain guided chip layout recognition device for implementing the above-mentioned multi-scale frequency domain guided chip layout recognition method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more multi-scale frequency domain guided chip layout recognition device embodiments provided below can refer to the limitations of the multi-scale frequency domain guided chip layout recognition method described above, and will not be repeated here.

[0125] In one exemplary embodiment, as shown in Figure 4 A multi-scale frequency domain guided chip layout recognition device is provided, comprising: an initial feature extraction module 402, a multi-scale feature determination module 404, a semantic enhancement module 406, and a chip classification module 408, wherein:

[0126] The initial feature extraction module 402 is configured to input the chip layout to be recognized into a layout recognition model, extract features of the chip layout to be recognized by a backbone network in the layout recognition model, and obtain initial intermediate layer features and initial high layer features.

[0127] The multi-scale feature determination module 404 is configured to project the initial intermediate layer features by an adaptive multi-scale frequency domain learner in the layout recognition model to obtain initial frequency domain features and initial spatial domain features, perform multi-scale extraction on the initial frequency domain features in the frequency domain to obtain multi-scale frequency domain features, use an attention mechanism to adaptively enhance the initial spatial domain features to obtain adaptively enhanced spatial domain local features, and fuse the multi-scale frequency domain features and the adaptively enhanced spatial domain local features to obtain intermediate layer multi-scale features.

[0128] The semantic enhancement module 406 is configured to perform semantic enhancement on the initial high-level feature by an adaptive task semantic enhancement module in the layout recognition model to obtain a high-level semantic enhanced feature.

[0129] The chip classification module 408 is configured to perform classification processing on the intermediate-level multi-scale feature and the initial high-level feature by a classifier in the layout recognition model to obtain a chip identification corresponding to the chip layout to be recognized.

[0130] In an exemplary embodiment, the multi-scale feature determination module 404 is further configured to perform denoising processing on the initial intermediate-level feature according to the initial high-level feature by an adaptive multi-scale frequency domain learner in the layout recognition model to obtain a denoised initial intermediate-level feature, and perform frequency domain feature extraction and spatial domain feature extraction on the denoised initial intermediate-level feature respectively to obtain an initial frequency domain feature and an initial spatial domain feature.

[0131] In an exemplary embodiment, the multi-scale feature determination module 404 is further configured to perform interpolation processing on the initial high-level feature by an adaptive multi-scale frequency domain learner in the layout recognition model to obtain an interpolated initial high-level feature, the resolution of the interpolated initial high-level feature being the same as that of the initial intermediate-level feature, and perform element-by-element multiplication on the interpolated initial high-level feature and the initial intermediate-level feature to obtain the denoised initial intermediate-level feature.

[0132] In an exemplary embodiment, the multi-scale feature determination module 404 further comprises:

[0133] The spatial domain conversion module is configured to perform spatial domain conversion processing on the initial frequency domain feature to obtain a first converted feature.

[0134] The frequency domain conversion module is configured to perform frequency domain conversion processing on the initial spatial domain feature to obtain a second converted feature.

[0135] The spatial enhancement module is configured to perform adaptive enhancement on the initial spatial domain feature based on the first converted feature using an attention mechanism to obtain an adaptively enhanced spatial domain local feature.

[0136] The self-fusion module is configured to perform self-fusion processing on the multi-scale frequency domain feature to obtain a self-fused frequency domain feature.

[0137] The first fusion module is configured to fuse the second converted feature and the self-fused frequency domain feature to obtain a scale enhanced feature.

[0138] The second fusion module is configured to fuse the scale enhanced feature and the adaptively enhanced spatial domain local feature to obtain the intermediate-level multi-scale feature.

[0139] In an example embodiment, the spatial enhancement module is further configured to perform feature preprocessing on the initial spatial domain feature to obtain a preprocessed initial spatial domain feature, fuse the preprocessed initial spatial domain feature with the first converted feature to obtain a spatial enhancement feature, perform attention weight extraction on the spatial enhancement feature to obtain an attention map, and perform weighted processing on the attention map and the spatial enhancement feature to obtain the self-adaptively enhanced spatial domain local feature.

[0140] In an example embodiment, the semantic enhancement module 406 is further configured to, by a task-adaptive semantic enhancement module in the layout recognition model, fuse the preset chip prompt information with the initial high-level feature to obtain prompt embedding features, and perform attention enhancement and semantic enhancement on the prompt embedding features to obtain high-level semantic enhancement features.

[0141] In an example embodiment, the initial intermediate layer feature includes a first intermediate layer feature and a second intermediate layer feature, the adaptive multi-scale frequency domain learner includes a first adaptive multi-scale frequency domain learner and a second adaptive multi-scale frequency domain learner, the initial frequency domain feature includes a first frequency domain feature and a second frequency domain feature, the initial spatial domain feature includes a first spatial domain feature and a second spatial domain feature, the multi-scale frequency domain feature includes a first multi-scale feature and a second multi-scale feature, the self-adaptively enhanced spatial domain local feature includes a first enhanced spatial domain local feature and a second enhanced spatial domain local feature, and the intermediate layer multi-scale feature includes a first target feature and a second target feature; the multi-scale feature determination module 404 is further configured to, by the first adaptive multi-scale frequency domain learner in the layout recognition model, project the first intermediate layer feature to obtain the first frequency domain feature and the first spatial domain feature, perform multi-scale extraction on the first frequency domain feature in the frequency domain to obtain the first multi-scale feature, use an attention mechanism to adaptively enhance the first spatial domain feature to obtain the first enhanced spatial domain local feature, and fuse the first multi-scale feature and the first enhanced spatial domain local feature to obtain the first target feature; and by the second adaptive multi-scale frequency domain learner in the layout recognition model, project the second intermediate layer feature to obtain the second frequency domain feature and the second spatial domain feature, perform multi-scale extraction on the second frequency domain feature in the frequency domain to obtain the second multi-scale feature, use an attention mechanism to adaptively enhance the second spatial domain feature to obtain the second enhanced spatial domain local feature, and fuse the second multi-scale feature and the second enhanced spatial domain local feature to obtain the second target feature.

[0142] In an exemplary embodiment, the chip classification module 406 is further configured to classify the intermediate layer multi-scale features and the high-level semantic enhancement features by the classifier in the layout recognition model, respectively, to obtain a first classification result corresponding to the intermediate layer multi-scale features and a second classification result corresponding to the high-level semantic enhancement features; and to perform a weighted summation of the first classification result and the second classification result to obtain the chip identifier corresponding to the chip layout to be identified.

[0143] Each module in the aforementioned multi-scale frequency domain guided chip layout identification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0144] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a multi-scale frequency domain guided chip layout identification method.

[0145] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0146] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0147] In an embodiment, a computer readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, implements the steps of any of the above method embodiments.

[0148] In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of any of the above method embodiments.

[0149] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0150] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0151] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0152] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A multi-scale frequency domain guided chip layout recognition method, characterized in that, The method includes: The chip layout to be identified is input into the layout recognition model, and the backbone network in the layout recognition model extracts the features of the chip layout to be identified to obtain the initial intermediate layer features and the initial high layer features. The adaptive multi-scale frequency domain learner in the map recognition model projects the initial intermediate layer features to obtain initial frequency domain features and initial spatial domain features. The initial frequency domain features are then extracted in the frequency domain at multiple scales to obtain multi-scale frequency domain features. The initial spatial domain features are then adaptively enhanced using an attention mechanism to obtain adaptively enhanced local spatial domain features. Finally, the multi-scale frequency domain features and the adaptively enhanced local spatial domain features are fused to obtain intermediate layer multi-scale features. The adaptive task semantic enhancement module in the map recognition model performs semantic enhancement on the initial high-level features to obtain high-level semantically enhanced features; The classifier in the layout recognition model classifies the intermediate layer multi-scale features and the high-level semantic enhancement features to obtain the chip identifier corresponding to the chip layout to be identified.

2. The method according to claim 1, characterized in that, The process of projecting the initial intermediate layer features onto the initial frequency domain features and initial spatial domain features using the adaptive multi-scale frequency domain learner in the layout recognition model includes: The adaptive multi-scale frequency domain learner in the map recognition model performs denoising processing on the initial intermediate layer features based on the initial high-level features to obtain denoised initial intermediate layer features. Then, frequency domain feature extraction and spatial domain feature extraction are performed on the denoised initial intermediate layer features to obtain the initial frequency domain features and the initial spatial domain features.

3. The method according to claim 2, characterized in that, The method of obtaining denoised initial intermediate layer features by the adaptive multi-scale frequency domain learner in the map recognition model based on the initial high-level features includes: The initial high-level features are interpolated by the adaptive multi-scale frequency domain learner in the map recognition model to obtain the interpolated initial high-level features. The resolution of the interpolated initial high-level features is the same as that of the initial intermediate layer features. The interpolated initial high-level features are then multiplied element-wise with the initial intermediate layer features to obtain the denoised initial intermediate layer features.

4. The method according to claim 1, characterized in that, The method further includes: The initial frequency domain features are subjected to spatial domain transformation to obtain the first transformed features; The initial spatial domain features are subjected to frequency domain transformation to obtain second transformed features, and an attention mechanism is used to adaptively enhance the initial spatial domain features based on the first transformed features to obtain adaptively enhanced local spatial domain features. The multi-scale frequency domain features are subjected to self-fusion processing to obtain self-fused frequency domain features; The second transformation feature is fused with the self-fused frequency domain feature to obtain the scale enhancement feature; The scale-enhanced features are fused with the adaptively enhanced spatial domain local features to obtain the intermediate layer multi-scale features.

5. The method according to claim 4, characterized in that, The step of using an attention mechanism to adaptively enhance the initial spatial domain features based on the first transformed features to obtain adaptively enhanced local spatial domain features includes: The initial spatial domain features are preprocessed to obtain the preprocessed initial spatial domain features; The preprocessed initial spatial domain features are fused with the first transformed features to obtain spatial enhancement features; Attention weights are extracted from the spatial enhancement features to obtain an attention map; The attention map and the spatial enhancement features are weighted to obtain the adaptively enhanced spatial domain local features.

6. The method according to claim 1, characterized in that, The step of semantically enhancing the initial high-level features to obtain high-level semantically enhanced features by the adaptive task semantic enhancement module in the map recognition model includes: The task-adaptive semantic enhancement module in the layout recognition model fuses the preset chip prompt information with the initial high-level features to obtain prompt embedding features. Attention enhancement and semantic enhancement are then performed on the prompt embedding features to obtain the high-level semantic enhancement features.

7. The method according to claim 1, characterized in that, The initial intermediate layer features include first intermediate layer features and second intermediate layer features; the adaptive multi-scale frequency domain learner includes a first adaptive multi-scale frequency domain learner and a second adaptive multi-scale frequency domain learner; the initial frequency domain features include first frequency domain features and second frequency domain features; the initial spatial domain features include first spatial domain features and second spatial domain features; the multi-scale frequency domain features include first multi-scale features and second multi-scale features; the adaptively enhanced spatial domain local features include first enhanced spatial domain local features and second enhanced spatial domain local features; and the intermediate layer multi-scale features include first target features and second target features. The adaptive multi-scale frequency domain learner in the map recognition model projects the initial intermediate layer features to obtain initial frequency domain features and initial spatial domain features. It then performs multi-scale extraction on the initial frequency domain features in the frequency domain to obtain multi-scale frequency domain features. Finally, it uses an attention mechanism to adaptively enhance the initial spatial domain features to obtain adaptively enhanced local spatial domain features. The multi-scale frequency domain features and the adaptively enhanced local spatial domain features are then fused to obtain intermediate layer multi-scale features, including: The first adaptive multi-scale frequency domain learner in the map recognition model projects the first intermediate layer features to obtain the first frequency domain features and the first spatial domain features. The first frequency domain features are extracted in the frequency domain at multiple scales to obtain the first multi-scale features. The first spatial domain features are adaptively enhanced using an attention mechanism to obtain the first enhanced spatial domain local features. The first multi-scale features and the first enhanced spatial domain local features are then fused to obtain the first target features. The second adaptive multi-scale frequency domain learner in the map recognition model projects the second intermediate layer features to obtain the second frequency domain features and the second spatial domain features. The second frequency domain features are extracted in the frequency domain at multiple scales to obtain the second multi-scale features. The second spatial domain features are adaptively enhanced using an attention mechanism to obtain the second enhanced spatial domain local features. The second multi-scale features and the second enhanced spatial domain local features are then fused to obtain the second target features.

8. The method according to claim 1, characterized in that, The step of classifying the intermediate-layer multi-scale features and the high-level semantic enhancement features by the classifier in the layout recognition model to obtain the chip identifier corresponding to the chip layout to be identified includes: The classifier in the layout recognition model classifies the intermediate layer multi-scale features and the high-level semantic enhancement features respectively, to obtain a first classification result corresponding to the intermediate layer multi-scale features and a second classification result corresponding to the high-level semantic enhancement features; and performs a weighted summation on the first classification result and the second classification result to obtain the chip identifier corresponding to the chip layout to be identified.

9. A chip layout recognition device guided by multi-scale frequency domain, characterized in that, The device includes: The initial feature extraction module is used to input the chip layout to be identified into the layout recognition model, and the backbone network in the layout recognition model extracts the features of the chip layout to be identified to obtain the initial intermediate layer features and the initial high layer features. The multi-scale feature determination module is used to project the initial intermediate layer features into the adaptive multi-scale frequency domain learner in the map recognition model to obtain initial frequency domain features and initial spatial domain features, extract the initial frequency domain features in the frequency domain at multiple scales to obtain multi-scale frequency domain features, adaptively enhance the initial spatial domain features using an attention mechanism to obtain adaptively enhanced local spatial domain features, and fuse the multi-scale frequency domain features and the adaptively enhanced local spatial domain features to obtain intermediate layer multi-scale features. The semantic enhancement module is used to semantically enhance the initial high-level features by the adaptive task semantic enhancement module in the layout recognition model to obtain high-level semantically enhanced features; The chip classification module is used to classify the intermediate layer multi-scale features and the high-level semantic enhancement features by the classifier in the layout recognition model to obtain the chip identifier corresponding to the chip layout to be identified.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images