Data cleaning method, device, equipment, medium and product
By measuring and clustering the lithography modeling data, selecting representative elements for lithography modeling, the problem that traditional methods cannot effectively clean high-dimensional lithography modeling data is solved, and the accuracy and efficiency of modeling are improved, and the promotion of the modeling is promoted. The development of lithography technology.
Patent Information
- Application Number
- CN202411455547.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2025-02-14
AI Technical Summary
Traditional lithography modeling data cleaning methods cannot effectively identify and remove repetitive information and useless information from massive, high-dimensional and complex lithography modeling data, resulting in error accumulation, waste of computing resources, and model accuracy during the modeling process. Stability decreases.
By measuring the distance based on the difference between the lithographic modeling data, the distance values between each data point are determined, and the data is clustered based on these distance values to obtain multiple clustering sets. Then, for each clustering set, a preset number of data is selected as representative elements, and the remaining data except the representative elements are filtered for lithography modeling.
This method effectively removes the repetitive and useless data in lithography modeling data, improves the accuracy and efficiency of modeling, provides high-quality, refined and representative data for lithography technology, and promotes lithography technology optimization and development.
Smart Images
Figure CN119441720A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of photolithography technology, and in particular, relates to a data cleaning method, device, equipment, medium and product. Background Art
[0002] In the lithography process of semiconductor manufacturing, accurate modeling is crucial to optimizing the lithography process and improving the quality and yield of chip production. With the continuous development of lithography technology, the number of measurement points has increased dramatically. When processing lithography modeling data containing a large number of measurement points, the data generated is not only large in scale, but also multi-dimensional. These lithography modeling data contain rich information, but also contain a lot of noise, repetitive data and other parts that are useless for modeling, and this part of useless information needs to be cleaned.
[0003] Traditional lithography modeling data cleaning methods usually rely on manual removal of bad points and random screening when preparing modeling data to complete data cleaning. This method is often inadequate when dealing with massive, high-dimensional and complex lithography modeling data. It is unable to effectively identify and remove repetitive and useless information hidden deep in the data, resulting in error accumulation, waste of computing resources, and reduced model accuracy and stability during the modeling process. Summary of the invention
[0004] The embodiments of the present application provide a data cleaning method, device, equipment, medium and product, which can effectively identify and remove repetitive information and useless information hidden deep in the data, and ensure correct and stable lithography modeling.
[0005] On the one hand, an embodiment of the present application provides a data cleaning method, including:
[0006] Based on each measurement point of the lithography modeling, corresponding lithography modeling data is obtained;
[0007] Performing distance measurement based on the difference between each of the photolithography modeling data to determine the distance value corresponding to each of the photolithography modeling data;
[0008] Based on the distance value, clustering is performed on each of the lithography modeling data to obtain a plurality of cluster sets;
[0009] For each of the cluster sets, a preset number of the photolithography modeling data are selected as representative elements, so as to perform photolithography modeling based on the representative elements.
[0010] On the other hand, clustering is performed on each of the lithography modeling data based on the distance value to obtain multiple cluster sets, including:
[0011] Determine each of the photolithography modeling data as an initial cluster set;
[0012] Based on the distance value, the initial cluster sets with the smallest current distance are gradually merged until a clustering termination condition is met, so as to obtain a plurality of cluster sets.
[0013] On the other hand, based on the distance value, the initial cluster sets with the smallest current distance are gradually merged until the clustering termination condition is met to obtain multiple cluster sets, including:
[0014] Merging the initial cluster sets with the smallest current distances to obtain multiple cluster sets to be determined;
[0015] Determining whether the pending cluster set satisfies the cluster termination condition;
[0016] In the case that the pending cluster set does not satisfy the cluster termination condition, merging the pending cluster sets with the closest current distances to obtain a new pending cluster set, and returning to the step of determining whether the pending cluster set satisfies the cluster termination condition;
[0017] In the case that the pending cluster sets satisfy the cluster termination condition, each of the pending cluster sets is determined as the cluster set.
[0018] On the other hand, the clustering termination condition includes any one or any combination of the following:
[0019] The number of cluster sets is less than a cluster number threshold;
[0020] The distance between the pending cluster sets to be merged next time is less than the distance threshold;
[0021] For each of the initial clustering sets, the similarities between the photolithography modeling data within the initial clustering sets are all greater than a similarity threshold;
[0022] For each of the initial clustering sets, the variance of each of the lithography modeling data within the initial clustering set is less than a variance threshold;
[0023] The number of merging of each of the initial clustering sets is greater than a number threshold.
[0024] On the other hand, for each cluster set, selecting a preset number of the lithography modeling data as representative elements includes:
[0025] For each of the cluster sets, the preset number of the lithography modeling data are randomly selected as the representative elements.
[0026] On the other hand, the step of obtaining corresponding lithography modeling data at each measurement point based on lithography modeling includes:
[0027] Based on the measurement points of photolithography modeling, the corresponding original modeling data is obtained;
[0028] Based on preset rules, the original modeling data is cleaned to obtain the required lithography modeling data.
[0029] On the other hand, the distance measurement is performed based on the difference between each of the lithography modeling data to determine the distance value corresponding to each of the lithography modeling data, including:
[0030] Performing standardization processing on each of the photolithography modeling data to obtain each of the photolithography modeling data having a mean of 0 and a variance of 1;
[0031] Based on the standardized photolithography modeling data, a corresponding covariance matrix is calculated;
[0032] Determining eigenvalues and eigenvectors of the covariance matrix;
[0033] Based on the eigenvalues and the eigenvectors, mapping each of the lithography modeling data into a two-dimensional space;
[0034] Based on the two-dimensional space, the distance between each of the photolithography modeling data is determined.
[0035] On the other hand, an embodiment of the present application provides a data cleaning device, including:
[0036] An acquisition module, used for acquiring corresponding lithography modeling data based on each measurement point of lithography modeling;
[0037] A determination module, configured to perform distance measurement based on the difference between each of the photolithography modeling data, and determine the distance value corresponding to each of the photolithography modeling data;
[0038] A clustering module, used for performing clustering processing on each of the photolithography modeling data based on the distance value to obtain a plurality of cluster sets;
[0039] The selection module is used to select a preset number of the photolithography modeling data as representative elements for each cluster set, so as to perform photolithography modeling based on the representative elements.
[0040] In another aspect, an embodiment of the present application provides a data cleaning device, comprising: a processor and a memory storing computer program instructions;
[0041] When the processor executes the computer program instructions, the data cleaning method as described above is implemented.
[0042] On the other hand, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the data cleaning method as described above is implemented.
[0043] On the other hand, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the data cleaning method as described above.
[0044] A data cleaning method provided in an embodiment of the present application first obtains a plurality of lithography modeling data, and uses a distance measurement method suitable for lithography data to determine the distance between each lithography modeling data, and then clusters each lithography modeling data based on the distance between each lithography modeling data to obtain a plurality of cluster sets. For each cluster set, only a preset number of lithography modeling data are selected as representative elements, and the remaining non-representative elements except the representative elements are filtered out to perform lithography modeling based on the representative elements. Compared with the traditional manual screening method, this scheme only selects a small number of representative representative elements from multiple similar elements for lithography modeling, accurately removes the duplication and useless data in the lithography modeling data, and greatly improves the accuracy and efficiency of modeling; it provides high-quality, refined and representative data for computational lithography modeling, and effectively promotes the optimization and development of lithography processes. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0046] Figure 1 A schematic diagram of a data cleaning method according to an embodiment of the present invention is shown;
[0047] Figure 2 A schematic diagram of two-dimensional data points after dimensionality reduction is shown;
[0048] Figure 3 A schematic diagram of a clustering tree without setting a clustering termination condition is shown;
[0049] Figure 4 A schematic diagram of the structure of a data cleaning device provided in an embodiment of the present application is shown;
[0050] Figure 5 A schematic diagram of the hardware structure of a data cleaning device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0051] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0052] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "include..." does not exclude the presence of other identical elements in the process, method, article or device including the element.
[0053] In computational lithography modeling, when it is necessary to process lithography measurement data at a large number of measurement points, the accuracy and efficiency of modeling are seriously affected due to the large scale of data and the presence of repetitive and unhelpful data. Too much useless data may even cause overfitting of the modeling. Traditional data cleaning methods perform poorly when dealing with such complex lithography data and cannot effectively remove redundant and useless information.
[0054] In order to solve the problems of the traditional solution, the embodiment of the present application provides a data cleaning method, device, equipment, medium and product. The data cleaning method provided by the embodiment of the present application is first introduced below. Figure 1 FIG. 1 is a flow chart of a data cleaning method provided by an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:
[0055] S101: Based on each measurement point of the lithography modeling, corresponding lithography modeling data is obtained.
[0056] In the process of photolithography modeling, it is necessary to model based on the graphics on the mask. Each position of the graphics on the mask is a measurement point, and each measurement point corresponds to photolithography modeling data. These photolithography modeling data can be obtained in various ways, such as obtaining gauges data through actual measurement, obtaining simulation data through simulation, or obtaining historical data from a database, etc. The photolithography modeling data corresponding to a measurement point includes the period Pitch, the critical dimension (Critical Dimention, CD) measurement value (such as width), the maximum light intensity Imax of the area corresponding to the measurement point, the minimum light intensity Imin of the area corresponding to the measurement point, and the imaging contrast ils, etc.
[0057] First, data collection and preprocessing are performed to collect a large amount of raw lithography modeling data from the measurement points of computational lithography modeling. And according to the characteristics of lithography data, preliminary cleaning is performed to remove missing values and outliers that are not related to the lithography process. A matrix that can characterize lithography modeling data is established, which reflects the characteristics of lithography modeling data in its corresponding group in multiple dimensions. The row of the matrix can be set as the directory index of the lithography modeling data, and the column of the matrix represents the various characteristics of the lithography modeling data, including but not limited to Pitch, CD, Imax, Imin, and ils.
[0058] In addition, in order to facilitate the subsequent distance measurement, the lithography modeling data can be first subjected to principal component analysis (PCA) dimensionality reduction. First, the numerical range and distribution of the lithography data are considered, and the pre-processed lithography modeling data is standardized so that its mean is 0 and its variance is 1. Then the covariance matrix of the data is accurately calculated, and the eigenvalues and eigenvectors of the covariance matrix are solved. Finally, according to the requirements of lithography modeling, two principal components with larger eigenvalues are selected to map the data into a two-dimensional space.
[0059] S102: Perform distance measurement based on the differences between the photolithography modeling data to determine the distance values corresponding to the photolithography modeling data.
[0060] The embodiment of the present application may select bottom-up hierarchical clustering and adopt a distance measurement method suitable for lithography data to calculate the distance between the lithography modeling data corresponding to each data point. The distance value between two lithography modeling data represents the difference between the two lithography modeling data.
[0061] S103: performing clustering processing on each lithography modeling data based on the distance value to obtain a plurality of cluster sets.
[0062] After determining the distance between two lithography modeling data, clustering is performed on each lithography modeling data. Initially, each data point is regarded as a separate initial cluster set, and then the sets with the closest distance are gradually merged until a specific termination condition is met (for example, the number of clusters that meet the lithography modeling requirements is reached).
[0063] S104: For each cluster set, select a preset number of lithography modeling data as representative elements, so as to perform lithography modeling based on the representative elements.
[0064] Finally, representative elements are selected. For each cluster, according to the distribution characteristics of the lithography data, an element can be randomly selected as the representative of the cluster, and other non-representative elements can be removed to complete data cleaning suitable for lithography modeling.
[0065] The present application scheme is described below with a specific implementation method.
[0066] Step 1: Collect data based on a large number of measurement points, including lithography modeling data of different CDs, different pitches, and different pattern types. After preliminary cleaning of abnormal points, obtain the data pool and establish the feature matrix of the data (Pitch, CD, Imax, Imin, ils).
[0067] Step 2: Perform PCA dimensionality reduction on the feature matrix of the data by group, and display it through the matplotlib.pylot editor. Figure 2 Schematic diagram of a two-dimensional data point after dimensionality reduction is shown; Figure 2 As shown, one group of two-dimensional data points after dimensionality reduction is displayed, which contains a total of 39 gauges data points.
[0068] Step 3: Perform bottom-up hierarchical clustering on the obtained two-dimensional data points. The clustering termination conditions can be selected according to actual needs. They may include the following: 1. Predetermined cluster number threshold: The number of clusters you want to obtain can be specified in advance. When the clustering process reaches the specified number, the clustering is stopped. 2. Distance threshold: Set a distance threshold. When the distance between two clusters to be merged is less than this threshold, the clustering is stopped. 3. Intra-cluster (each data in the set) similarity threshold: Define a threshold for intra-cluster similarity. When the clustering reaches a certain stage and the similarity between the data points in all clusters exceeds this threshold, the clustering is stopped. 4. Based on variance: Calculate the variance of each cluster. When the variance of all clusters is less than a predetermined value, it is considered that the clustering has achieved a good effect and the clustering is stopped. 5. Based on the height or depth of the hierarchical tree: Set a height or depth limit for the hierarchical tree. When the clustering process reaches this limit, the clustering is stopped.
[0069] Figure 3 A schematic diagram of a clustering tree without setting a clustering termination condition is shown; Figure 3 As shown in the figure, clustering is performed on each set in turn to obtain a cluster tree, but the final result is only one cluster set, which does not meet the actual needs. Therefore, it is necessary to set the cluster termination condition. If the termination condition is set to: the cluster number threshold is 20, the iteration will stop after 20 different clusters are obtained. Finally, 20 different cluster sets can be obtained and output in a list according to the specific gauge number: [[16,17],[19,20,21,22,23,24,25,26],
[18] ,
[28] ,
[30] ,[31,32],[33,34,35,36,37,38],
[29] ,[7,11],[0],[6],
[13] ,
[10] ,[4,8],[1],
[27] ,[2,5],[12,14],[3,9],
[15] ].
[0070] Step 4: Select representative elements based on the cluster sets obtained in step 3. For example, you can randomly select a gauge number from each cluster set in the cluster list as the representative element of the class. Since the characteristics of the class are similar, this representative element can be used as the final gauge for modeling data, thereby discarding the remaining duplicate elements.
[0071] The embodiment of the present application combines the dimensionality reduction capability of PCA with the bottom-up hierarchical clustering method to form a unique data processing flow, which effectively copes with the high dimensionality and complexity of gauge data in lithography modeling.
[0072] By adopting bottom-up hierarchical clustering, we can adaptively discover hidden patterns and structures in lithography data and accurately distinguish useful and useless data.
[0073] A data cleaning method provided in an embodiment of the present application first obtains a plurality of lithography modeling data, and uses a distance measurement method suitable for lithography data to determine the distance between each lithography modeling data, and then clusters each lithography modeling data based on the distance between each lithography modeling data to obtain a plurality of cluster sets. For each cluster set, only a preset number of lithography modeling data are selected as representative elements, and the remaining non-representative elements except the representative elements are filtered out to perform lithography modeling based on the representative elements. Compared with the traditional manual screening method, this scheme only selects a small number of representative representative elements from multiple similar elements for lithography modeling, accurately removes the duplication and useless data in the lithography modeling data, and greatly improves the accuracy and efficiency of modeling; it provides high-quality, refined and representative data for computational lithography modeling, and effectively promotes the optimization and development of lithography processes.
[0074] After clustering, multiple cluster sets are finally obtained, and multiple photolithography modeling data are initially obtained, so each photolithography modeling data can be determined as an initial cluster set, and then based on the distance value, the initial cluster sets with the smallest current distance are gradually merged until the clustering termination condition is met to obtain multiple cluster sets. By gradually merging, each cluster set that meets the actual needs can be quickly and accurately obtained, and excessive merging can be avoided.
[0075] For example, the pairwise initial clustering sets with the smallest current distances may be merged to obtain multiple pending clustering sets; and then it may be determined whether the pending clustering sets satisfy the clustering termination condition.
[0076] When the pending cluster set does not meet the cluster termination condition, the pairwise pending cluster sets with the closest current distances are merged to obtain a new pending cluster set, and the process returns to the step of determining whether the pending cluster set meets the cluster termination condition.
[0077] When the undetermined cluster sets satisfy the clustering termination condition, each undetermined cluster set is determined as a cluster set.
[0078] The embodiment of the present application provides a specific method for gradually merging initial cluster sets. Through this method, the merging of cluster sets can be completed efficiently, and the merging result conforms to the actual situation.
[0079] The above embodiment does not limit the specific clustering termination condition. The embodiment of the present application provides some specific implementation methods. The clustering termination condition includes any one or any combination of the following:
[0080] 1. The number of cluster sets is less than the cluster number threshold. That is, a predetermined cluster number threshold is set, and the number of clusters that are expected to be obtained is specified in advance. When the clustering process reaches the specified number, the clustering process stops.
[0081] 2. The distance between the next set of pending clusters to be merged is less than the distance threshold; that is, a distance threshold is set, and when the distance between two clusters to be merged is less than this threshold, clustering is stopped.
[0082] 3. For each initial cluster set, the similarity between each lithography modeling data within is greater than the similarity threshold. That is, set the similarity threshold within the cluster (each data within the set): define a threshold for the similarity within the cluster. When clustering reaches a certain stage, the similarity between all data points in the cluster exceeds this threshold, stop clustering.
[0083] 4. For each initial cluster set, the variance of each lithography modeling data is less than the variance threshold. That is, the variance of each cluster is calculated. When the variance of all clusters is less than a predetermined value, it is considered that clustering has achieved a good effect and clustering is stopped.
[0084] 5. The number of merges of each initial clustering set is greater than the threshold number. That is, a height or depth limit of the hierarchical tree is set, and when the clustering process reaches this limit, the clustering is stopped.
[0085] The several clustering termination conditions provided in the embodiments of the present application can obtain a clustering set that meets actual needs.
[0086] In an embodiment of the present application, for each cluster set, a preset number of lithography modeling data needs to be selected as representative elements. In actual application, the number of representative elements selected for each cluster set and the selection method are not limited. Generally, a cluster set selects a representative element set. In order to ensure that the representative element can represent the entire cluster set, a preset number of lithography modeling data can be randomly selected as the representative element for each cluster set.
[0087] The embodiment of the present application randomly selects representative elements, and can efficiently randomly select an element of a class of elements as a representative in the clustering results, which not only ensures the representativeness of the data but also reduces data redundancy.
[0088] In practical applications, when obtaining corresponding lithography modeling data based on each measurement point of lithography modeling, some useless data may be obtained. It is necessary to clean and filter out this part of the data in advance to avoid affecting subsequent steps.
[0089] Specifically, firstly, the corresponding original modeling data is obtained based on the measurement points of the lithography modeling; then, the original modeling data is cleaned based on the preset rules to obtain the required lithography modeling data. Here, the cleaning can be performed according to the characteristics of the lithography data to remove missing values and abnormal values that are not related to the lithography process.
[0090] In the embodiment of the present application, the data is preliminarily cleaned before using the lithography modeling data, so as to prevent some irrelevant data from affecting the subsequent processing process.
[0091] In a specific implementation, when performing distance measurement, it is necessary to first process the acquired lithography modeling data in order to better perform distance measurement. The present application embodiment provides a specific solution, including:
[0092] Standardizing each lithography modeling data to obtain each lithography modeling data with a mean of 0 and a variance of 1;
[0093] Based on the standardized photolithography modeling data, the corresponding covariance matrix is calculated;
[0094] Determine the eigenvalues and eigenvectors of the covariance matrix;
[0095] Based on the eigenvalues and eigenvectors, each lithography modeling data is mapped into a two-dimensional space;
[0096] Based on the two-dimensional space, the distance between each lithography modeling data is determined.
[0097] The embodiment of the present application performs precise dimensionality reduction processing and accurately reduces the multi-dimensional lithography data to two dimensions through PCA, thereby greatly reducing the complexity of data processing while retaining key information.
[0098] In order to solve the above technical problems, the present application also provides a data cleaning device. Figure 4 The structure diagram of the data cleaning device provided by the embodiment of the present application is shown. Figure 4 As shown, the device includes the following modules:
[0099] An acquisition module 401 is used to acquire corresponding lithography modeling data based on each measurement point of the lithography modeling;
[0100] A determination module 402 is used to perform distance measurement based on the difference between each lithography modeling data, and determine the distance value corresponding to each lithography modeling data;
[0101] A clustering module 403 is used to perform clustering processing on each lithography modeling data based on the distance value to obtain multiple cluster sets;
[0102] The selection module 404 is used to select a preset number of lithography modeling data as representative elements for each cluster set, so as to perform lithography modeling based on the representative elements.
[0103] In some embodiments, the clustering module is specifically configured to:
[0104] Determine each lithography modeling data as an initial cluster set;
[0105] Based on the distance value, the initial cluster sets with the smallest current distance are gradually merged until the clustering termination condition is met to obtain multiple cluster sets.
[0106] In some embodiments, the clustering module is specifically configured to:
[0107] Merge the pairwise initial clustering sets with the smallest current distance to obtain multiple pending clustering sets;
[0108] Determine whether the pending clustering set meets the clustering termination condition;
[0109] In the case that the pending cluster set does not meet the cluster termination condition, the pairwise pending cluster sets with the closest current distance are merged to obtain a new pending cluster set, and the step of determining whether the pending cluster set meets the cluster termination condition is returned;
[0110] When the undetermined cluster sets satisfy the clustering termination condition, each undetermined cluster set is determined as a cluster set.
[0111] In some embodiments, the clustering termination condition includes any one or any combination of the following:
[0112] The number of cluster sets is less than the cluster number threshold;
[0113] The distance between the pending cluster sets to be merged next time is less than the distance threshold;
[0114] For each initial clustering set, the similarity between each lithography modeling data within it is greater than the similarity threshold;
[0115] For each initial clustering set, the variance of each lithography modeling data within it is less than the variance threshold;
[0116] The number of merging of each initial clustering set is greater than the number threshold.
[0117] In some embodiments, the selection module is specifically used to:
[0118] For each cluster set, a preset number of lithography modeling data are randomly selected as representative elements.
[0119] In some embodiments, the acquisition module is specifically configured to:
[0120] Based on the measurement points of photolithography modeling, the corresponding original modeling data is obtained;
[0121] Based on preset rules, the original modeling data is cleaned to obtain the required lithography modeling data.
[0122] In some embodiments, the determination module is specifically configured to:
[0123] Standardizing each lithography modeling data to obtain each lithography modeling data with a mean of 0 and a variance of 1;
[0124] Based on the standardized photolithography modeling data, the corresponding covariance matrix is calculated;
[0125] Determine the eigenvalues and eigenvectors of the covariance matrix;
[0126] Based on the eigenvalues and eigenvectors, each lithography modeling data is mapped into a two-dimensional space;
[0127] Based on the two-dimensional space, the distance between each lithography modeling data is determined.
[0128] The device provided in the embodiment of the present application is the same as the method in the above embodiment, so both have the same embodiments and beneficial effects, which will not be repeated here.
[0129] Figure 5The hardware structure diagram of the data cleaning device provided in the embodiment of the present application is shown. Figure 5 As shown, the data cleaning device may include a processor 501 and a memory 502 storing computer program instructions.
[0130] Specifically, the processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0131] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 502 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory.
[0132] The memory 502 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present disclosure.
[0133] The processor 501 implements any one of the data cleaning methods in the above embodiments by reading and executing computer program instructions stored in the memory 502 .
[0134] In one example, the data cleaning device may further include a communication interface 503 and a bus 504. The processor 501, the memory 502, and the communication interface 503 are connected via the bus 504 and communicate with each other.
[0135] The communication interface 503 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.
[0136] The bus 504 includes hardware, software or both, coupling the components of the data cleaning device to each other. For example, but not limitation, the bus may include an Accelerate Graphical Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus or other suitable buses or a combination of two or more of these. Where appropriate, the bus 504 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.
[0137] In addition, in combination with the data cleaning method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the data cleaning methods in the above embodiment is implemented.
[0138] An embodiment of the present application also provides a computer program product, including a computer program, which implements any one of the data cleaning methods in the above embodiments when the computer program is processed and executed.
[0139] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0140] The functional blocks shown in the block diagram above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an ASIC, appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (Erasable ROM, EROM), floppy disks, compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical discs, hard disks, optical fiber media, radio frequency (Radio Frequency, RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0141] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0142] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of a data cleaning method, device, equipment, medium and product according to an embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.
[0143] The above contents are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application.
Claims
1. A data cleaning method, characterized in that: include: Based on each measurement point of the lithography modeling, corresponding lithography modeling data is obtained; Performing distance measurement based on the difference between each of the photolithography modeling data to determine the distance value corresponding to each of the photolithography modeling data; Based on the distance value, clustering is performed on each of the lithography modeling data to obtain a plurality of cluster sets; For each of the cluster sets, a preset number of the photolithography modeling data are selected as representative elements, so as to perform photolithography modeling based on the representative elements.
2. The data cleaning method according to claim 1, characterized in that: The clustering process is performed on each of the lithography modeling data based on the distance value to obtain multiple cluster sets, including: Determine each of the photolithography modeling data as an initial cluster set; Based on the distance value, the initial cluster sets with the smallest current distance are gradually merged until a clustering termination condition is met, so as to obtain a plurality of cluster sets.
3. The data cleaning method according to claim 2, characterized in that: The step of gradually merging the initial cluster sets with the smallest current distance based on the distance value until a clustering termination condition is met to obtain a plurality of cluster sets includes: Merging the initial cluster sets with the smallest current distances to obtain multiple cluster sets to be determined; Determining whether the pending cluster set satisfies the cluster termination condition; In the case that the pending cluster set does not satisfy the cluster termination condition, merging the pending cluster sets with the closest current distances to obtain a new pending cluster set, and returning to the step of determining whether the pending cluster set satisfies the cluster termination condition; In the case that the pending cluster sets satisfy the cluster termination condition, each of the pending cluster sets is determined as the cluster set.
4. The data cleaning method according to claim 3, characterized in that: The clustering termination condition includes any one or any combination of the following: The number of cluster sets is less than a cluster number threshold; The distance between the pending cluster sets to be merged next time is less than the distance threshold; For each of the initial clustering sets, the similarities between the photolithography modeling data within the initial clustering sets are all greater than a similarity threshold; For each of the initial clustering sets, the variance of each of the lithography modeling data within the initial clustering set is less than a variance threshold; The number of merging of each of the initial clustering sets is greater than a number threshold.
5. The data cleaning method according to claim 1, characterized in that: The step of selecting a preset number of the photolithography modeling data as representative elements for each cluster set includes: For each of the cluster sets, the preset number of the lithography modeling data are randomly selected as the representative elements.
6. The data cleaning method according to any one of claims 1 to 5, characterized in that: The step of obtaining corresponding lithography modeling data at each measurement point based on lithography modeling includes: Based on the measurement points of photolithography modeling, the corresponding original modeling data is obtained; Based on preset rules, the original modeling data is cleaned to obtain the required lithography modeling data.
7. The data cleaning method according to claim 1, characterized in that: The performing distance measurement based on the difference between each of the lithography modeling data to determine the distance value corresponding to each of the lithography modeling data includes: Performing standardization processing on each of the photolithography modeling data to obtain each of the photolithography modeling data having a mean of 0 and a variance of 1; Based on the standardized photolithography modeling data, a corresponding covariance matrix is calculated; Determining eigenvalues and eigenvectors of the covariance matrix; Based on the eigenvalues and the eigenvectors, mapping each of the lithography modeling data into a two-dimensional space; Based on the two-dimensional space, the distance between each of the photolithography modeling data is determined.
8. A data cleaning device, characterized in that: include: An acquisition module, used for acquiring corresponding lithography modeling data based on each measurement point of lithography modeling; A determination module, configured to perform distance measurement based on the difference between each of the photolithography modeling data, and determine the distance value corresponding to each of the photolithography modeling data; A clustering module, used for performing clustering processing on each of the photolithography modeling data based on the distance value to obtain a plurality of cluster sets; The selection module is used to select a preset number of the photolithography modeling data as representative elements for each cluster set, so as to perform photolithography modeling based on the representative elements.
9. A data cleaning device, characterized in that: include: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the data cleaning method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the data cleaning method according to any one of claims 1 to 7 is implemented.
11. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the data cleaning method as described in any one of claims 1 to 7.