File auditing method and system and computer equipment

Abstract

The invention relates to a file auditing method and system and computer equipment, and the method comprises the steps: obtaining a to-be-audited file, inputting the to-be-audited file into a pre-stored recognition model for recognition, and extracting auditing features in the to-be-audited file; based on a preset rule, auditing the auditing features to determine whether the auditing features accord with the preset rule; and when at least part of the audit features in the to-be-audited file meet a first preset condition, generating a processing file based on the at least part of the audit features, and processing the semiconductor structure by using the processing file. The pre-stored recognition model is used for recognizing the to-be-audited file, various features can be rapidly extracted from various different types of files input into the recognition model, then the standard processing file is generated, the auditing efficiency and the auditing precision are improved, and the processing progress and the processing reliability are accelerated.

Description

Technical Field

[0001] The present disclosure relates to the field of information technology, and in particular to a file review method, system and computer equipment. Background Art

[0002] The mask factory involves a variety of technical documents, such as equipment manuals and tooling (process equipment) documents provided by customers. These documents stipulate the specifications and parameters for mask production, etc., and contain a lot of complex information. It is very important to accurately understand and identify the requirements.

[0003] The existing document review methods include manual review and semi-automatic review. Manual review of the information in the document is time-consuming and prone to errors. In the semi-automatic review process, an audit program will be set up for review, but such programs are generally fixed codes and can only process documents with fixed formats. This audit method is not universal and has high update costs. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a file review method, system and computer device.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for reviewing a file is provided, comprising: obtaining a file to be reviewed, wherein the file to be reviewed is used to record processing parameters of a semiconductor structure, and the semiconductor structure includes a mask; inputting the file to be reviewed into a pre-stored recognition model for recognition, and extracting review features in the file to be reviewed; based on preset rules, reviewing the review features to determine whether the review features comply with the preset rules; when at least part of the review features in the file to be reviewed meets a first preset condition, generating a processing file based on at least part of the review features, and using the processing file to process the semiconductor structure.

[0006] In some exemplary embodiments of the present disclosure, the method for obtaining the recognition model includes: obtaining a sample file; converting the format of the sample file to obtain a standard sample file; inputting the standard sample file into a preset training model for training, and extracting sample information from the standard sample file; evaluating the sample information based on pre-stored preset information, and obtaining the recognition model when the sample information meets a second preset condition.

[0007] In some exemplary embodiments of the present disclosure, the sample information includes sample text information, the preset information includes preset text information, and the sample information is evaluated based on the pre-stored preset information, and the recognition model is obtained when the sample information meets a second preset condition, including: when a first evaluation index of the sample text information relative to the preset text information reaches a first preset threshold, the recognition model is obtained, wherein the first evaluation index is used to characterize the accuracy of the sample text information relative to the preset text information.

[0008] In some exemplary embodiments of the present disclosure, the sample information includes sample image information, the preset information includes preset image information, and the sample information is evaluated based on the pre-stored preset information, and the recognition model is obtained when the sample information satisfies a second preset condition, including: when a second evaluation indicator of the sample image information relative to the preset image information reaches a second preset threshold, the recognition model is obtained, wherein the second evaluation indicator is used to characterize the similarity between the sample image information and the preset image information.

[0009] In some exemplary embodiments of the present disclosure, the sample information includes sample text information and sample image information, and the sample information is evaluated based on pre-stored preset information to obtain the recognition model when the sample information meets a second preset condition, including: performing feature fusion on the sample text information and the sample image information to obtain a sample fusion feature; when a third evaluation indicator of the sample text information in the sample fusion feature and the sample image information reaches a third preset threshold, the recognition model is obtained; wherein the third evaluation indicator is used to characterize the degree of matching between the sample text information and the sample image information in the sample fusion feature.

[0010] In some exemplary embodiments of the present disclosure, when the sample information does not meet the second preset condition, the audit method includes: finding the sample file with errors and re-marking it to obtain a corrected sample file; updating the sample file based on the corrected sample file; re-inputting the updated sample file into the preset training model for training, and extracting the sample information in the updated sample file; evaluating the sample information based on the pre-stored preset information; if the sample information does not meet the second preset condition, repeating the above steps until the sample information meets the second preset condition.

[0011] In some exemplary embodiments of the present disclosure, the preset training model includes a preset sub-model for text information and a preset sub-model for image information, and the method for obtaining a recognition model further includes: based on a multimodal fusion strategy, fusing the sample text information extracted by the preset sub-model for text information and the sample image information extracted by the preset sub-model for image information through a self-attention mechanism; during the fusion process, processing the sample text information and the sample image information based on a nonlinear activation function and a normalization layer to obtain the sample information.

[0012] In some exemplary embodiments of the present disclosure, the audit feature includes only text information, and the auditing of the audit feature based on preset rules includes: comparing the text information in the audit feature with the preset rules; the audit feature includes text information and image information, and the auditing of the audit feature based on preset rules includes: comparing the text information in the audit feature with the preset rules; cross-comparing the text information in the audit feature with the image information in the audit feature.

[0013] According to a second aspect of an embodiment of the present disclosure, a file review system is provided, comprising: a file acquisition module, configured to acquire a file to be reviewed, wherein the file to be reviewed is used to record processing parameters of a semiconductor structure, and the semiconductor structure includes a mask; a file recognition module, configured to input the file to be reviewed into a pre-stored recognition model for recognition, and extract review features in the file to be reviewed; a file review module, configured to review the review features based on preset rules to determine whether the review features comply with the preset rules; and a file output module, configured to generate a processing file based on at least some of the review features in the file to be reviewed when at least some of the review features meet a first preset condition, and use the processing file to process the semiconductor structure.

[0014] According to a third aspect of an embodiment of the present disclosure, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect are implemented.

[0015] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0016] The file audit method disclosed in the present invention recognizes semiconductor files by using a pre-stored recognition model and audits the recognized semiconductor files based on preset rules, thereby achieving efficient auditing of files of different formats and improving the accuracy of file auditing. In addition, since a recognition model is set for recognition, the file audit method disclosed in the present invention can continuously learn from new data, improve its auditing performance and adaptability over time, and reduce the update cost of the auditing system.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0019] Figure 1 The following is a flowchart of reviewing a Tooling file according to an exemplary embodiment.

[0020] Figure 2 The present invention is a flowchart of a method for reviewing a file according to an exemplary embodiment.

[0021] Figure 3 The present invention is a flowchart of a method for reviewing a file according to an exemplary embodiment.

[0022] Figure 4 It is a schematic diagram of a preset training model architecture according to an exemplary embodiment.

[0023] Figure 5 The present invention is a flowchart of a method for reviewing a file according to an exemplary embodiment.

[0024] Figure 6 The figure is a schematic diagram of a file review architecture according to an exemplary embodiment.

[0025] Figure 7 The present invention is a flowchart of a method for reviewing a file according to an exemplary embodiment.

[0026] Figure 8 The present invention is a block diagram of a file review system according to an exemplary embodiment.

[0027] Fig. 9 The present invention is a block diagram of a file review system according to an exemplary embodiment.

[0028] Fig.10 It is a block diagram of a computer device according to an exemplary embodiment. DETAILED DESCRIPTION

[0029] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of systems and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0030] The mask factory involves a variety of technical documents, such as equipment manuals and tooling files provided by customers. These documents stipulate the specifications and parameters for mask production and contain a lot of complex information. It is very important to accurately understand and identify the requirements.

[0031] Taking the Tooling file as an example, the Tooling file provided by the customer contains complex information, as shown in Table 1. Table 1 lists some of the information that the Tooling file must contain, which reflects the customer's specific requirements for semiconductor processing. The Tooling file can also contain other information. Before processing the data in the Tooling file provided by the customer, the information in the Tooling file and the design drawing must be reviewed, and the consistency between the Tooling file and the design layout must be checked to ensure the accuracy of the MDP (Mask Data Preparation) execution. Figure 1 As shown in the figure, the process of reviewing Tooling files includes:

[0032] S100. Obtain tooling files and design drawings.

[0033] S110, determine whether the Tooling file complies with the audit rules, if so, execute S130, if not, execute S120.

[0034] S120. Update Tooling file.

[0035] S130, determine whether the design drawings comply with the review rules, if so, execute S150, if not, execute S140.

[0036] S140. Update design drawings.

[0037] S150, processing the data in the tooling file and the design drawing.

[0038] By reviewing the information in the tooling files and design drawings, we ensure that the data processing is efficient and accurate. Because the file types, file formats, and term usage of tooling files provided by different customers are different, and the format and design habits of the design layout are also different, it usually takes a lot of time and manpower.

[0039] Table 1

[0040]

[0041]

[0042] Existing document review methods include manual review and semi-automatic review. Among them, manual verification of information in documents is time-consuming and prone to errors. In the semi-automatic review process, an audit program will be set up for review. This type of program is generally a fixed code. By limiting the data type of a specific field (for example, numbers, fixed formats, etc.), the fields that violate the preset rules are highlighted to achieve the audit effect. However, this technology can only process documents with fixed formats (for example, Excel documents), and this type of program needs to be customized according to the customer's file format. Even small format changes later require maintenance and update of the program. This audit method is not universal and has a high update cost.

[0043] In order to solve the above problems, the present disclosure provides a file review method, system and computer device. The file review method includes: obtaining a file to be reviewed, wherein the file to be reviewed is used to record the processing parameters of a semiconductor structure, and the semiconductor structure includes a mask, inputting the file to be reviewed into a pre-stored recognition model for recognition, extracting the review features in the file to be reviewed, and reviewing the review features based on preset rules to determine whether the review features meet the preset rules. When at least part of the review features in the file to be reviewed meet the first preset condition, a processing file is generated based on at least part of the review features, and the semiconductor structure is processed using the processing file. The file review method of the present disclosure is adopted, and the semiconductor file is recognized by using a pre-stored recognition model, and the recognized semiconductor file is reviewed based on the preset rules, so that the efficient review of files of different formats is realized, and the accuracy of the file review is improved. In addition, since the recognition model is set for recognition, the file review method in the present disclosure can continuously learn from new data, and its review performance and adaptability can be improved over time, reducing the update cost of the review system.

[0044] The exemplary embodiment of the present disclosure provides a method for reviewing a file, such as Figure 2 As shown, the document review method includes:

[0045] S200, obtaining a file to be reviewed, wherein the file to be reviewed is used to record processing parameters of a semiconductor structure, and the semiconductor structure includes a mask.

[0046] S210: Input the document to be reviewed into a pre-stored recognition model for recognition, and extract the review features in the document to be reviewed.

[0047] S220: Based on the preset rules, the audit features are audited to determine whether the audit features comply with the preset rules.

[0048] S230, determining whether at least part of the audit features in the to-be-audited file meet a first preset condition, if so, executing S240; if not, executing S250.

[0049] S240. Generate a processing file based on at least part of the audited features, and use the processing file to process the semiconductor structure.

[0050] S250. Output the review report.

[0051] In step S200, the document to be reviewed is a document recording the processing parameters of the semiconductor structure, for example, the processing design document of the semiconductor structure, including relevant product specifications, equipment manuals, tooling files, factory engineering files, etc. The semiconductor structure includes a mask. Specifically, the document to be reviewed can be the processing design document on the mask provided to the customer.

[0052] In step S210, the file to be reviewed is input into a pre-stored recognition model for recognition. The file to be reviewed needs to be in a format that can be recognized by the recognition model. For example, if the recognition model is trained in a pytorch (an open source Python machine learning library) environment, the file to be reviewed can be in a pytorch object format. This embodiment does not make any specific limitations.

[0053] Therefore, in step S200, before obtaining the file to be reviewed, it also includes converting the original file into a file to be reviewed that can be recognized by the recognition model. Converting the original file into a file to be reviewed that can be recognized by the recognition model can include processing the text information and image information in the original file separately. For example, data cleaning, conversion, and standardization can be adopted, and the data in the original file can be read through specific program development. By extracting text information, the format of the text information is adjusted to a format that can be recognized by the recognition model. For example, synonyms are uniformly replaced with standard words (for example, synonyms such as "mask", "mask", and "mask (mask)" are unified into "mask"), meaningless words are removed, and the information of each item is expressed in the form of keywords. By processing image information, the image is adjusted to a specific size, the contrast is adjusted, or the image is binarized, etc. Other methods can also be used to convert the original file into a format that can be recognized by the recognition model. This disclosure is not specifically limited.

[0054] In step S210, the file to be reviewed may include text information and / or image information. The review features in the file to be reviewed identified by the recognition model include text information and / or image information. That is, the file to be reviewed may be a text file, an image file, or a file combining text and image. Regardless of the type of file, the pre-stored recognition model may be used for recognition.

[0055] In some exemplary embodiments of the present disclosure, Figure 3 As shown, the method for obtaining the recognition model includes:

[0056] S300: Obtain a sample file.

[0057] S310: Convert the format of the sample file to obtain a standard sample file.

[0058] S320: Input the standard sample file into a preset training model for training, and extract sample information from the standard sample file.

[0059] S330: Evaluate the sample information based on pre-stored preset information.

[0060] S340, determine whether the sample information meets the second preset condition, if so, execute S350; if not, execute S360.

[0061] S350: Obtain a recognition model.

[0062] S360, searching for sample files with errors and re-marking them to obtain corrected sample files.

[0063] S370: Update the sample file based on the revised sample file.

[0064] S380, re-input the updated sample file into the preset training model for training, extract the sample information in the updated sample file, and execute S330.

[0065] In step S300, obtaining sample files includes: obtaining a large number of technical files in the semiconductor field, such as semiconductor technical manuals, process flow charts, equipment manuals, and patent documents. The sample files include text information and / or image information. In addition, these sample files will be marked to facilitate the learning of the preset training model, understand the technical content, text features, and graphic features in the semiconductor field, and improve the accuracy and reliability of the trained model.

[0066] In step S310, the standard sample file is in a format that can be recognized by the preset training model, such as a pytorch object format, which is not limited in this embodiment. The sample file is format converted, including: format conversion of text information and image information in the sample file respectively.

[0067] Exemplarily, converting the format of text information in the sample file may include:

[0068] Word segmentation: Use a customized word segmenter to segment the text information in the sample file. By segmenting the text information in the sample file, professional terms and complex technical expressions can be better processed.

[0069] Terminology annotation: The domain-enhanced BERT (Bidirectional Encoder Representations from Transformers) model is used to identify and annotate key technical terms (such as "exposure", "etching", "wafer", etc.) in sample files.

[0070] Data augmentation: Add synonym replacement (such as "mask" and "reticle"), and insert descriptions of specific process steps to enrich the sample file dataset. Data augmentation is used to ensure that the preset training model can handle the same technical content in different expressions.

[0071] Exemplarily, converting the format of the image information in the sample file may include:

[0072] Image normalization: Normalize the image information in the sample file. Image normalization ensures that the preset training model can process different types of images.

[0073] Image enhancement: Randomly crop and rotate the image information in the sample file. Image enhancement is used to improve the robustness of the preset training model in processing image information with angle and position changes.

[0074] It is understandable that other methods may be used to convert the format of text information and image information in the sample file, and the present disclosure does not limit this.

[0075] By performing format conversion on the text information and image information in the sample file respectively in this embodiment, not only the sample data set is enriched, but also the sample file is converted into a standard sample file that can be recognized by a preset training model.

[0076] In step S320, the preset training model includes a text information preset sub-model and an image information preset sub-model. The text information preset sub-model can select the BERT model, which is used to identify semantics and extract features from the text information in the file. The text information preset sub-model can also use other models, which is not specifically limited in the present disclosure. The image information preset sub-model can select a CNN (Convolutional Neural Networks) architecture model, such as ResNet (Residual Network), etc., which is not specifically limited in the present disclosure.

[0077] In some embodiments, the method for obtaining a recognition model further includes:

[0078] Based on the multimodal fusion strategy, the sample text information extracted by the text information preset sub-model and the sample image information extracted by the image information preset sub-model are fused through the self-attention mechanism.

[0079] During the fusion process, the sample text information and the sample image information are processed based on the nonlinear activation function and the normalization layer to obtain the sample information.

[0080] The preset training model disclosed in the present invention adds a fully connected layer in the text information preset sub-model and the image information preset sub-model, which is used in the semiconductor industry, especially the semantic and image recognition of mask technical documents. A feature fusion module is also added, which uses a self-attention mechanism to fuse the sample text information extracted by the text information preset sub-model and the sample image information extracted by the image information preset sub-model, thereby improving the system's ability to understand and process semiconductor technical documents containing graphic information. In the fusion process, the sample text information and sample image information are processed through nonlinear activation functions and normalization layers to enhance the expressive power of the preset training model, so that the model can accurately identify and match graphic information.

[0081] An example is given below to describe the design case of the preset training model disclosed in the present invention. The following example is only for ease of understanding and does not constitute a limitation of this embodiment.

[0082] Preset training model You can select a pre-trained model and fine-tune the pre-trained model to enable it to understand the process terminology and process parameters of the semiconductor industry.

[0083] For example, the text information preset sub-model can choose the BERT-Base (the original version of the BERT model) pre-trained model, and the image information preset sub-model can choose the ResNet-50 (ResNet containing 50 convolutional layers) pre-trained model. During the fine-tuning process, a fully connected layer is added to the BERT-Base pre-trained model architecture and the ResNet-50 pre-trained model architecture, and a feature fusion module is added. The self-attention mechanism is used to fuse the text features extracted by the BERT-Base pre-trained model and the image features extracted by the ResNet-50 pre-trained model. The preset training model architecture is as follows: Figure 4 shown.

[0084] The BERT-Base pre-trained model architecture includes:

[0085] Input layer: used to receive and process text data, especially semiconductor technical documents.

[0086] Embedding layer: used to convert the input vocabulary into word vectors to capture the basic semantic information in the text.

[0087] Transformer encoder (12 layers): used to deeply understand the context and semantic relationships of text.

[0088] CLS (Classification) output layer: used to generate a global semantic representation of the text as the input of the classification task.

[0089] New fully connected layer: added during fine-tuning, used for classification of specific file review tasks.

[0090] The ResNet-50 pre-trained model architecture includes:

[0091] Input layer: used to receive and process image data, such as process flow charts and equipment structure diagrams.

[0092] Residual block (50 layers): used to extract image features and maintain key structural information of the image.

[0093] Global average pooling layer: used to map high-dimensional features to low-dimensional space for easy classification.

[0094] Original fully connected layer: used to output the preliminary classification results of the image.

[0095] New fully connected layers: added during fine-tuning for a specific semiconductor image classification task.

[0096] The image and text joint processing module includes:

[0097] New feature fusion module: A Transformer-based multimodal fusion strategy is adopted to fuse the text features extracted by the BERT-Base pre-trained model with the image features extracted by the ResNet-50 pre-trained model through a self-attention mechanism. In the feature fusion process, nonlinear activation functions (for example, ReLU (Rectified Linear Unit)) and normalization layers are added to enhance the expressiveness of the model and ensure that the model can accurately identify and match text and image information.

[0098] In this example, both the text information preset sub-model and the image information preset sub-model use pre-trained models. Pre-trained models are universal and only need to be fine-tuned according to actual technical features, which can save training time, reduce data requirements, and improve model performance and stability. The text information preset sub-model uses the BERT-Base pre-trained model, which uses the Transformer architecture to deeply understand the semantic information in the file. The image information preset sub-model uses the ResNet-50 pre-trained model to extract visual features from the image of the file. By designing a feature fusion module and using the self-attention mechanism to fuse the text features and image features extracted by the model, the system's ability to understand and process files containing graphic information is improved, and a comprehensive understanding and recognition of semiconductor files is achieved.

[0099] The following is the process of fine-tuning the pre-trained models used by the text information preset sub-model and the image information preset sub-model.

[0100] In some embodiments, in step S320, Figure 5 As shown, the standard sample file is input into the preset training model for training, including:

[0101] S500: Preliminary pre-training of a preset sub-model for text information and a preset sub-model for image information.

[0102] An example is given below to specifically describe the process of preliminary pre-training of the text information preset sub-model and the image information preset sub-model in step S500. The following example is only for ease of understanding and does not constitute a limitation of this embodiment.

[0103] Initial pre-training of the BERT model: The BERT-Base pre-trained model is used and trained on a specialized database containing semiconductor industry terminology and technical documents. Self-supervised tasks such as term prediction and process step reordering are added to enhance the BERT-Base pre-trained model's understanding of technical processes and document structures.

[0104] Preliminary pre-training of the ResNet model: The ResNet-50 pre-trained model is used and initialized using the model weights pre-trained on industrial image datasets (such as semiconductor equipment image libraries and process route flow charts). In the preliminary pre-training, multi-view images of semiconductor structures are combined to enhance the ResNet-50 pre-trained model’s ability to recognize device morphology and structure.

[0105] S510: Perform domain adaptation training on the preset sub-model for text information and the preset sub-model for image information.

[0106] An example is given below to specifically describe the process of performing domain adaptation training on the preset sub-model for text information and the preset sub-model for image information in step S510. The following example is only for ease of understanding and does not constitute a limitation of this embodiment.

[0107] Domain adaptation training of the BERT model: By expanding the semiconductor process document dataset and using self-training technology to generate pseudo-label data, combined with manually annotated data for continuous pre-training, the model is ensured to adapt to the specific language style of the semiconductor field. The model is optimized using the attention mechanism so that it can give priority to the key technical steps and parameter descriptions in the document, improving the ability to parse process flow documents.

[0108] Domain adaptation training of the ResNet model: Freeze the feature extraction layer of the ResNet model and fine-tune the domain image data (such as images of lithography machines, etching machines, and testing equipment, process roadmaps, chip layout diagrams, etc.). Through knowledge distillation technology, the knowledge of complex models containing a large number of semiconductor-related images is transferred to this model to adapt to the recognition task of industry-related images.

[0109] S520: Perform multi-task fine-tuning on the preset sub-model for text information and the preset sub-model for image information.

[0110] An example is given below to specifically describe the process of performing multi-task fine-tuning on the text information preset sub-model and the image information preset sub-model in step S520. The following example is only for ease of understanding and does not constitute a limitation of this embodiment.

[0111] Multi-task fine-tuning of the BERT model:

[0112] Document classification task: Fine-tune the BERT model to classify different types of documents such as process flow, equipment manuals, and technical specifications. Use SMOTE (Synthetic Minority Over-sampling Technique) technology to deal with the class imbalance problem, and combine it with Focal Loss to optimize the model, so that the model's accuracy in classifying process steps and equipment instructions is improved.

[0113] Terminology explanation task: Combine the bidirectional LSTM (Long Short-Term Memory) network with the BERT model output to generate contextual explanations of semiconductor terms, such as explaining the different technical steps involved in the "lithography" process. Introduce a multi-head attention mechanism to improve the understanding of complex technical descriptions.

[0114] Process relationship judgment task: Use the graph convolution network (GCN) to model the process flow as a graph structure to capture the dependencies between process steps, such as determining the process sequence between lithography and etching, and determining the data processing flow corresponding to the process specified by the customer.

[0115] Multi-task fine-tuning of the ResNet model: For image classification, a multi-task learning architecture is designed to support classification of different devices and different image types, while improving the performance of the overall model through feature sharing mechanism. The learning rate is adjusted through the cosine annealing scheduler to ensure the stability and accuracy of model training.

[0116] S530, joint fine-tuning of graphics and text.

[0117] An example is given below to specifically describe the process of fine-tuning the image and text in step S530. The following example is only for ease of understanding and does not constitute a limitation on this embodiment.

[0118] Joint fine-tuning of feature fusion modules: Through end-to-end optimization, we train the joint text-image task. For the audit task of sample files, we use a weighted strategy to balance the loss functions of text and image tasks to ensure that the multimodal model can effectively fuse and analyze text-image information. Through distributed training, we improve training efficiency and ensure that the model is efficient and real-time when processing large-scale technical document data.

[0119] A large number of sample files are used to train the preset training model, and the text information preset sub-model and the image information preset sub-model are trained separately. A feature fusion module is designed to fuse the sample text information extracted by the text information preset sub-model and the sample image information extracted by the image information preset sub-model, thereby enhancing the preset training model's ability to understand and process files containing text information and image information.

[0120] In step S320, the standard sample file is input into a preset training model for training, and sample information in the training result is obtained, where the sample information includes sample text information and sample image information.

[0121] In some embodiments, in steps S330, S340, and S350, the sample information includes sample text information, the preset information includes preset text information, and the sample information is evaluated based on the pre-stored preset information. When the sample information satisfies the second preset condition, a recognition model is obtained, including:

[0122] When a first evaluation index of the sample text information relative to the preset text information reaches a first preset threshold, a recognition model is obtained, wherein the first evaluation index is used to characterize the accuracy of the sample text information relative to the preset text information.

[0123] The preset text information is the correct recognition result after the preset training model recognizes the text information in the sample file. It can be understood that the sample information in this embodiment satisfies the second preset condition including: the sample text information reaches the first preset threshold relative to the first evaluation index of the preset text information. Among them, the first evaluation index may include accuracy and F1 score (F1Score), which is not limited in this embodiment. The value of the first preset threshold can be an empirical value, or it can be determined through experimental data of multiple experiments. This embodiment does not limit this. For example, the first preset threshold of accuracy can be 90%, 95%, etc., and the first preset threshold of F1 score can be 0.85, 0.9, etc.

[0124] In some embodiments, in steps S330, S340, and S350, the sample information includes sample image information, the preset information includes preset image information, and the sample information is evaluated based on the pre-stored preset information. When the sample information satisfies the second preset condition, a recognition model is obtained, including:

[0125] When a second evaluation index of the sample image information relative to the preset image information reaches a second preset threshold, a recognition model is obtained, wherein the second evaluation index is used to characterize the similarity between the sample image information and the preset image information.

[0126] The preset image information is the correct recognition result after the preset training model recognizes the image information in the sample file. It can be understood that the sample information in this embodiment satisfies the second preset condition including: the second evaluation index of the sample image information relative to the preset image information reaches the second preset threshold. Among them, the second evaluation index may include accuracy, recall rate and precision rate, etc., which are not limited in this embodiment. The value of the second preset threshold can be an empirical value, or it can be determined by experimental data from multiple experiments. This embodiment does not limit this. For example, the second preset threshold of accuracy can be 95%, 98%, etc., the second preset threshold of recall rate can be 0.9, 0.95, etc., and the second preset threshold of precision rate can be 0.9, 0.95, etc.

[0127] In some embodiments, in steps S330, S340, and S350, the sample information includes sample text information and sample image information, and the sample information is evaluated based on pre-stored preset information. When the sample information meets the second preset condition, a recognition model is obtained, including:

[0128] The sample text information and the sample image information are subjected to feature fusion to obtain a sample fusion feature, and when the third evaluation index of the sample text information and the sample image information in the sample fusion feature reaches a third preset threshold, a recognition model is obtained. The third evaluation index is used to characterize the matching degree of the sample text information and the sample image information in the sample fusion feature.

[0129] It can be understood that the sample information in this embodiment satisfies the second preset condition including: the third evaluation index of the sample text information and the sample image information in the sample fusion feature reaches the third preset threshold. Among them, the third evaluation index may include accuracy and image-text matching accuracy, which is not limited in this embodiment. The value of the third preset threshold may be an empirical value, or it may be determined by experimental data from multiple experiments. This embodiment does not limit this. For example, the third preset threshold of accuracy may be 90%, 95%, etc., and the P@K value in the first 10 results is required to reach 0.8 or above (that is, the Precision at K indicator is used for evaluation, indicating that among the first 10 returned results, at least 8 have an accuracy rate above the third preset threshold).

[0130] In this embodiment, the training results in step S320 are evaluated, including evaluating the sample text information recognized by the preset training model, evaluating the sample image information recognized by the preset training model, and evaluating the matching degree between the sample text information and the sample image information. Only when the preset training model reaches the set threshold value on all evaluation indicators and performs stably in actual application (for example, the indicators are achieved in the trial operation for more than half a year), the preset training model is put into use as the recognition model.

[0131] In step S360, the sample files with errors are searched and re-marked, including: searching for sample files corresponding to sample information that does not meet the second preset condition, modifying and re-marking the weights of the sample files to obtain corrected sample files.

[0132] In steps S370 and S380, the sample file is updated based on the modified sample file, and the preset training model is retrained using the updated sample file to form a data closed loop, thereby improving the accuracy of model training. The recognition model is obtained by repeating the training until the sample information meets the second preset condition.

[0133] In this embodiment, deep learning technology is used to train a model that can recognize text information and image information, and to fuse text information and image information so that the model can understand the semantics of the file, thereby realizing the automated review of the file and improving the accuracy and efficiency of the file review. The model can recognize text information and image information, and can be used for the recognition and understanding of all types of file materials, saving time for manual review and reducing the error rate. In addition, since a recognition model is set for recognition, the file review method in the present disclosure can also continuously learn from new data, and can improve its review performance and adaptability over time, reducing the update cost of the review system.

[0134] In step S220, the preset rules are rules for pre-defining key process information, processing parameters, etc. related to semiconductor processing and design. Based on the preset rules, the audit features identified by the recognition model are audited, that is, the text information and / or image information included in the audit features are audited to determine whether they meet the preset rules.

[0135] In some embodiments, the audit feature only includes text information. In step 220, the audit feature is audited based on preset rules, including: comparing the text information in the audit feature with the preset rules.

[0136] In some embodiments, the audit feature includes text information and image information. In step 220, based on the preset rules, the audit feature is audited, including: comparing the text information in the audit feature with the preset rules, and cross-comparing the text information in the audit feature with the image information in the audit feature. By cross-comparing the text information with the image information, the rigor of the audit process is enhanced.

[0137] In step S230, the first preset condition includes: the audit features related to the design and processing of the semiconductor all meet the preset rules. It can be understood that the document to be audited may include information or parameters that are not related to the design and processing of the semiconductor.

[0138] In step S240, at least some of the audit features in the to-be-audited document meet the first preset condition, including: the audit features related to the design and processing of the semiconductor in the to-be-audited document all meet the preset rules, or all the audit features in the to-be-audited document all meet the preset rules. It can be understood that the audit features in the to-be-audited document that are not related to the design and production of the semiconductor and the audit features that are not defined in the preset rules do not meet the preset rules and have no impact on the design and processing of the semiconductor.

[0139] In step S250, if at least some of the audit features in the audited document do not meet the first preset condition, a review report is output. In the review report, only the text information and image information in the audit features related to the semiconductor design and processing process in the audited document that do not meet the preset rules can be annotated and feedbacked; the review report can also annotate and feedback all the text information and image information in the audit features of the audited document that do not meet the preset rules. This disclosure does not make specific limitations.

[0140] In some embodiments, Figure 6 As shown in Figure 1, an audit architecture is shown. Figure 4 Based on the preset model architecture shown, a final classification layer is added, which is used to review the sample information identified by the recognition model and output the review results.

[0141] The file audit method disclosed in the present invention uses a pre-stored recognition model to identify semiconductor files, and audits the identified semiconductor files based on preset rules, thereby solving the technical problems of long time consumption and high update cost of manual and semi-automatic audit in the prior art. By adopting deep learning technology, using models to understand file semantics, and performing automated file audits, efficient audits of files of different formats are achieved, and the accuracy of file audits is improved. In addition, since a recognition model is set for recognition, the file audit method disclosed in the present invention can continuously learn from new data, and its audit performance and adaptability can be improved over time, reducing the update cost of the audit system.

[0142] The following is a description of the file review method in the present disclosure through a specific example. The following example is only for ease of understanding and does not constitute a limitation of the present disclosure.

[0143] Example

[0144] 1. Prepare sample files: Collect a large number of technical documents in the semiconductor field, such as semiconductor technical manuals, process flow charts, equipment manuals, and patent documents, and convert the format of the sample files.

[0145] 2. Train the preset training model: Train in the pytorch environment, use sample files to fine-tune the BERT-Base pre-trained model and the ResNet-50 pre-trained model, and add a fully connected layer and a feature fusion module to the BERT-Base pre-trained model architecture and the ResNet-50 pre-trained model architecture during the fine-tuning process.

[0146] 3. Obtain the recognition model: Evaluate the training results of the BERT-Base pre-trained model. If the accuracy is above 90% and the F1 score is above 0.85, the BERT-Base pre-trained model is qualified. Evaluate the training results of the ResNet-50 pre-trained model. If the accuracy is above 95% and the recall and precision are both above 0.9, the ResNet-50 pre-trained model is qualified. Evaluate the training results of the feature fusion module: If at least 8 of the first 10 returned results have an accuracy of more than 90%, the feature fusion module is qualified and the recognition model is obtained.

[0147] 4. Document recognition: Use the recognition model to identify the document to be reviewed, which includes at least one of text information and image information.

[0148] 5. Set preset rules: limit key process information, processing parameters, etc. related to semiconductor processing and design, and build a rule base.

[0149] 6. Document review: If the document to be reviewed only includes text information, the results of the recognition model will be compared with the preset rules. If the document to be reviewed includes text information and image information, the text information recognized by the recognition model will be compared with the preset rules, and the text information recognized by the recognition model will be cross-compared with the image information recognized by the recognition model.

[0150] 7. Output results: If the information related to the design and processing of semiconductors in the audited document complies with the preset rules, or if all the information in the audited document complies with the preset rules, the data in the audited document can be processed for semiconductor processing. If the information related to the design and processing of semiconductors in the audited document does not comply with the preset rules, the content that does not comply with the preset rules will be automatically marked and the review report will be output.

[0151] like Figure 7 As shown, for ease of understanding, a specific embodiment is given below to describe the audit method of the document disclosed in the present invention.

[0152] S700: Obtain a sample file.

[0153] S701. Convert the format of a sample file to obtain a standard sample file.

[0154] S702: Perform preliminary pre-training on the preset sub-model for text information and the preset sub-model for image information.

[0155] S703: Perform domain adaptation training on the preset sub-model for text information and the preset sub-model for image information.

[0156] S704: Perform multi-task fine-tuning on the text information preset sub-model and the image information preset sub-model.

[0157] S705, fine-tuning the image and text together to extract sample information from the standard sample file.

[0158] S706, determining whether a first evaluation index of the sample text information relative to the preset text information reaches a first preset threshold value, if so, executing S707; if not, executing S710.

[0159] S707 , determining whether the second evaluation index of the sample image information relative to the preset image information reaches a second preset threshold value, if so, executing S708 ; if not, executing S710 .

[0160] S708, determining whether the third evaluation index of the sample text information and the sample image information in the sample fusion feature reaches a third preset threshold, if so, executing S709; if not, executing S710.

[0161] S709: Obtain a recognition model and execute S712.

[0162] S710, searching for sample files with errors and re-marking them to obtain corrected sample files.

[0163] S711. Update the sample file based on the revised sample file and execute S702.

[0164] S712. Obtain the files to be reviewed.

[0165] S713: Input the document to be reviewed into a pre-stored recognition model for recognition, and extract the review features in the document to be reviewed.

[0166] S714. Based on the preset rules, the audit features are audited to determine whether the audit features comply with the preset rules.

[0167] S715: Determine whether at least part of the audit features in the to-be-audited file meets the first preset condition. If so, execute S716; if not, execute S717.

[0168] S716. Generate a processing file based on at least part of the audited features, and use the processing file to process the semiconductor structure.

[0169] S717. Output the review report.

[0170] The file audit method disclosed in the present invention recognizes semiconductor files by using a pre-stored recognition model and audits the recognized semiconductor files based on preset rules, thereby achieving efficient auditing of files of different formats and improving the accuracy of file auditing. In addition, since a recognition model is set for recognition, the file audit method disclosed in the present invention can continuously learn from new data, improve its auditing performance and adaptability over time, and reduce the update cost of the auditing system.

[0171] The exemplary embodiment of the present disclosure provides a file review system. Figure 8 As shown, a block diagram of a file review system shown in the present disclosure includes a file acquisition module 800, a file identification module 810, a file review module 820, and a file output module 830.

[0172] The file acquisition module 800 is configured to acquire a file to be reviewed, wherein the file to be reviewed is used to record processing parameters of a semiconductor structure, and the semiconductor structure includes a mask;

[0173] The file identification module 810 is configured to input the to-be-reviewed file into a pre-stored identification model for identification, and extract the audit features in the to-be-reviewed file;

[0174] The file review module 820 is configured to review the review feature based on the preset rules to determine whether the review feature complies with the preset rules;

[0175] The file output module 830 is configured to generate a processing file based on at least some of the audit features in the audited file when at least some of the audit features in the audited file meet a first preset condition, and use the processing file to process the semiconductor structure.

[0176] In some exemplary embodiments of the present disclosure, the file identification module 810 is further configured to:

[0177] Get the sample file;

[0178] Convert the format of the sample file to obtain a standard sample file;

[0179] Input the standard sample file into the preset training model for training, and extract the sample information in the standard sample file;

[0180] The sample information is evaluated based on the pre-stored preset information, and when the sample information meets the second preset condition, a recognition model is obtained.

[0181] In some exemplary embodiments of the present disclosure, the sample information includes sample text information, and the file identification module 810 is further configured to:

[0182] When a first evaluation index of the sample text information relative to the preset text information reaches a first preset threshold, a recognition model is obtained, wherein the first evaluation index is used to characterize the accuracy of the sample text information relative to the preset text information.

[0183] In some exemplary embodiments of the present disclosure, the sample information includes sample image information, and the file identification module 810 is further configured to:

[0184] When a second evaluation index of the sample image information relative to the preset image information reaches a second preset threshold, a recognition model is obtained, wherein the second evaluation index is used to characterize the similarity between the sample image information and the preset image information.

[0185] In some exemplary embodiments of the present disclosure, the sample information includes sample text information and sample image information, and the file identification module 810 is further configured to:

[0186] Perform feature fusion on sample text information and sample image information to obtain sample fusion features;

[0187] When the third evaluation index of the sample text information and the sample image information in the sample fusion feature reaches a third preset threshold, obtaining a recognition model;

[0188] The third evaluation index is used to characterize the matching degree between the sample text information and the sample image information in the sample fusion feature.

[0189] In some exemplary embodiments of the present disclosure, the file identification module 810 is further configured to:

[0190] Find sample files with errors and re-mark them to obtain corrected sample files;

[0191] updating the sample file based on the revised sample file;

[0192] Re-input the updated sample file into the preset training model for training, and extract the sample information in the updated sample file;

[0193] Evaluating sample information based on pre-stored preset information;

[0194] If the sample information does not meet the second preset condition, the above steps are repeated until the sample information meets the second preset condition.

[0195] In some exemplary embodiments of the present disclosure, Fig. 9 As shown, the present disclosure shows a block diagram of a file review system, showing that the preset training model includes a text information preset sub-model, an image information preset sub-model and the feature fusion module described in the above embodiment, and the file recognition module 810 is also configured as follows:

[0196] Based on the multimodal fusion strategy, the sample text information extracted by the text information preset sub-model and the sample image information extracted by the image information preset sub-model are fused through the self-attention mechanism;

[0197] During the fusion process, the sample text information and the sample image information are processed based on the nonlinear activation function and the normalization layer to obtain the sample information.

[0198] In some exemplary embodiments of the present disclosure, the audit feature only includes text information, and the file audit module 820 is further configured to:

[0199] Compare the text information in the audit feature with the preset rules;

[0200] The audit features include text information and image information. The file audit module 820 is also configured to:

[0201] Compare the text information in the audit feature with the preset rules;

[0202] Cross-compare the text information in the audit feature with the image information in the audit feature.

[0203] Each module in the above-mentioned file review system can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0204] In an exemplary embodiment, a computer device is provided, including a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any of the above-mentioned file auditing methods are implemented.

[0205] refer to Fig.10 , a block diagram of a computer device that can be used as an audit system for the documents disclosed herein will now be described, the computer device comprising a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the computer device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0206] A plurality of components in the computer device 1000 are connected to the I / O interface 1005, including: an input unit 1006, an output unit 1007, a storage unit 1008, and a communication unit 1009. The input unit 1006 may be any type of device capable of inputting information to the computer device 1000, the input unit 1006 may receive input digital or character information, and generate key signal input related to user settings and / or function control of the computer device 1000, and may include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 1007 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1008 may include but is not limited to a disk and an optical disk. The communication unit 1009 allows the computer device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device and / or the like.

[0207] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above, such as the audit method of the file. For example, in some embodiments, the audit method of the file may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the computer device 1000 via ROM 1002 and / or the communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the audit method of the file described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute the audit method of the file by any other appropriate means (e.g., by means of firmware).

[0208] The computer device 1000 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned file auditing method.

[0209] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0210] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for reviewing a document, characterized in that: The audit methods include: Obtaining a file to be reviewed, wherein the file to be reviewed is used to record processing parameters of a semiconductor structure, wherein the semiconductor structure includes a mask; Input the document to be reviewed into a pre-stored recognition model for recognition, and extract the review features in the document to be reviewed; Based on the preset rules, the audit feature is audited to determine whether the audit feature complies with the preset rules; When at least part of the audit features in the to-be-audited file meets a first preset condition, a processing file is generated based on at least part of the audit features, and the semiconductor structure is processed using the processing file.

2. The method for reviewing a document according to claim 1, characterized in that: The method for obtaining the recognition model includes: Get the sample file; Convert the sample file into a standard file. Input the standard sample file into a preset training model for training, and extract sample information from the standard sample file; The sample information is evaluated based on pre-stored preset information, and when the sample information meets a second preset condition, the recognition model is obtained.

3. The method for reviewing a document according to claim 2, characterized in that: The sample information includes sample text information, the preset information includes preset text information, and the sample information is evaluated based on the pre-stored preset information, and when the sample information meets a second preset condition, the recognition model is obtained, including: When a first evaluation index of the sample text information relative to the preset text information reaches a first preset threshold, the recognition model is obtained, wherein the first evaluation index is used to characterize the accuracy of the sample text information relative to the preset text information.

4. The method for reviewing a document according to claim 3, characterized in that: The sample information includes sample image information, the preset information includes preset image information, and the sample information is evaluated based on the pre-stored preset information, and when the sample information meets a second preset condition, the recognition model is obtained, including: When a second evaluation index of the sample image information relative to the preset image information reaches a second preset threshold, the recognition model is obtained, wherein the second evaluation index is used to characterize the similarity between the sample image information and the preset image information.

5. The method for reviewing a document according to claim 4, characterized in that: The sample information includes sample text information and sample image information, and the sample information is evaluated based on pre-stored preset information, and when the sample information meets a second preset condition, the recognition model is obtained, including: Performing feature fusion on the sample text information and the sample image information to obtain sample fusion features; When the third evaluation index of the sample text information and the sample image information in the sample fusion feature reaches a third preset threshold, obtaining the recognition model; The third evaluation index is used to characterize the matching degree between the sample text information and the sample image information in the sample fusion feature.

6. The method for reviewing a document according to claim 2, characterized in that: When the sample information does not meet the second preset condition, the audit method includes: Find the sample file with errors and re-mark it to obtain a corrected sample file; updating the sample file based on the revised sample file; Re-inputting the updated sample file into the preset training model for training, and extracting sample information from the updated sample file; Evaluating the sample information based on pre-stored preset information; If the sample information does not meet the second preset condition, repeat the above steps until the sample information meets the second preset condition.

7. The method for reviewing a document according to claim 2, characterized in that: The preset training model includes a text information preset sub-model and an image information preset sub-model, and the method for obtaining a recognition model further includes: Based on the multimodal fusion strategy, the sample text information extracted by the preset sub-model of the text information and the sample image information extracted by the preset sub-model of the image information are fused through a self-attention mechanism; During the fusion process, the sample text information and the sample image information are processed based on a nonlinear activation function and a normalization layer to obtain the sample information.

8. The method for reviewing a document according to claim 1, characterized in that: The audit feature only includes text information, and the audit feature is audited based on a preset rule, including: Comparing the text information in the audit feature with the preset rules; The audit feature includes text information and image information, and the auditing of the audit feature based on the preset rules includes: Comparing the text information in the audit feature with the preset rules; The text information in the audit feature is cross-compared with the image information in the audit feature.

9. A document review system, characterized in that: The audit system includes: A file acquisition module is configured to acquire a file to be reviewed, wherein the file to be reviewed is used to record processing parameters of a semiconductor structure, and the semiconductor structure includes a mask; A file recognition module is configured to input the to-be-reviewed file into a pre-stored recognition model for recognition, and extract the review features in the to-be-reviewed file; A file review module, configured to review the review feature based on a preset rule to determine whether the review feature complies with the preset rule; The file output module is configured to generate a processing file based on at least part of the audit features in the to-be-audited file when at least part of the audit features meet a first preset condition, and use the processing file to process the semiconductor structure.

10. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 8 when executing the computer program.

Images