Template extraction method and system based on sparse coding

Abstract

The invention provides a template extraction method and system based on sparse coding. The method comprises the following steps: shooting at least one complete image of a to-be-detected crystal grain; in the at least one complete image, determining a group of basic images as a dictionary by optimizing and solving a target function, and carrying out training learning on the dictionary; coding the grain image by using the learned dictionary to obtain a sparse coefficient vector; and combining the dictionary, the non-zero numerical values in the sparse coefficient vector and the positions of the non-zero numerical values to form a comprehensive crystal grain template. According to the method, the storage space required by the template is greatly reduced, and the universality and flexibility of the template are also improved. In the detection process, if a new crystal grain is encountered, a whole set of new templates do not need to be regenerated, and only a new coefficient vector needs to be calculated according to the existing dictionary to represent the uniqueness of the crystal grain.

Description

Technical Field

[0001] The present application relates to the technical field of industrial visual inspection, and in particular to a template extraction method and system based on sparse coding. Background Art

[0002] In the field of industrial visual inspection, especially in grain defect detection, the template comparison-based method is a common strategy for defect detection. The core of this method is to identify any abnormalities on the surface or inside of the object by comparing it with a pre-defined defect-free standard template. In order to further improve the efficiency and accuracy of detection, the current optimization practice tends to complete the extraction of the complete template of the object to be inspected at one time before the start of the inspection process.

[0003] Specifically, this template extraction process usually includes the following steps: First, a series of high-resolution images are obtained from multiple objects of the same type. This process is designed to ensure that the collected data can cover all possible changes, thereby improving the representativeness and reliability of subsequent templates. Then, these images are accurately aligned using advanced image processing technology to ensure that each pixel corresponds to each other in spatial position. This step is crucial to ensuring the quality of the final template because it directly affects the consistency of features between different images.

[0004] Finally, the mean image or median image of the aligned image set is calculated as the final template. The choice of using the mean image or the median image depends on the specific application scenario and its requirements for noise sensitivity. The mean image can effectively smooth random noise, but it is more sensitive to extreme values; while the median image performs better in removing outliers and is suitable for environments with more interference factors. Through such a series of carefully designed steps, a standard template that is both accurate and robust can be constructed, providing a solid foundation for subsequent defect detection. This method not only improves the speed of detection, but also enhances the reliability and repeatability of the results.

[0005] When extracting templates, existing methods only use pixel information but not structural information, which means that the template must record the value of each pixel, which takes up a lot of storage space. In particular, in wafer inspection applications, the pattern of a single die may be composed of repeated combinations of patterns of several modules. In this case, recording each pixel will waste a lot of space. Summary of the invention

[0006] In view of this, the purpose of the present application is to propose a template extraction method and system based on sparse coding, which can solve the existing problems in a targeted manner.

[0007] Based on the above purpose, the present application proposes a template extraction method based on sparse coding, comprising: Taking at least one complete image of the grain to be inspected; In the at least one complete image, a set of basic images is determined as a dictionary by optimizing and solving an objective function, and the dictionary is trained and learned; Encode the grain image using the learned dictionary to obtain a sparse coefficient vector; The dictionary and the non-zero values ​​and their positions in the sparse coefficient vector are combined to form a comprehensive grain template.

[0008] Furthermore, the relationship between the number of images and computational efficiency is balanced according to specific application requirements, acceptable processing time, and available computing resources; Ensure that all images have a preset high resolution when shooting to capture the subtle features of the grain surface; Control the consistency of shooting conditions to reduce errors caused by changes in external factors and ensure the authenticity and comparability of the acquired images.

[0009] Furthermore, the objective function is used to maximize sparsity while minimizing the reconstruction error, and the objective function is in the following form: , Where: represents the input grain image; is the dictionary that attempts to determine the base image composition; represents the coefficient vector, which describes how to approximate the input image by combining the basis images in the dictionary; It is a parameter used to balance reconstruction error and sparsity, and adjust the model's preference for sparsity; It is a parameter used to balance reconstruction error and periodicity, and adjust the model's preference for periodicity; Represents the L2 norm, which is used to measure the difference between matrices; represents the L1 norm, which is used to promote the sparsity of the solution; Indicates Fourier transform of X for calculating the spectrum; Indicates modulus, which is used to calculate the spectrum amplitude; Entroy() means calculating the entropy value of the spectrum. The lower the entropy, the stronger the periodicity. The calculation formula is as follows: , , in and Indicates the frequency position in the spectrum.

[0010] Furthermore, the algorithm for optimizing and solving the objective function is a K-SVD algorithm or a gradient descent method.

[0011] Furthermore, the training and learning of the dictionary includes: Perform noise reduction on the input image; Strictly align the die patterns on the wafer; Choose the appropriate dictionary size based on the complexity of the wafer design; The training and learning are performed based on normal grain images covering different batches and under different process conditions.

[0012] Furthermore, the specific process of the encoding is: (1) Preparation stage: First, obtain a dictionary that has been learned through dictionary learning ; (2) Input image processing: The complete image of the grain to be encoded is used as input ; (3) Select algorithm: Select a preset algorithm for sparse coding according to the specific situation; (4) Execute the code: Run the preset algorithm and use the dictionary For the input image Encode and get the sparse coefficient vector ; (5) Verification and optimization: Check the obtained sparse coefficient vector quality, adjust parameters or use different algorithms to optimize the results.

[0013] Furthermore, the non-zero values ​​and positions in the sparse coefficient vector are used to indicate which base images are used to reconstruct the original image, and the weight of each base image's contribution to the final image.

[0014] Based on the above purpose, the present application also proposes a template extraction system based on sparse coding, including: A data preparation module, used for taking at least one complete image of the grain to be inspected; A dictionary learning module, used to determine a set of basic images as a dictionary by optimizing and solving an objective function in the at least one complete image, and to train and learn the dictionary; A coefficient encoding module, for encoding the grain image using the learned dictionary to obtain a sparse coefficient vector; The template combination module is used to combine the non-zero values ​​and their positions in the dictionary and the sparse coefficient vector to form a comprehensive grain template.

[0015] In general, the advantages of this application and the experience it brings to users are: For wafers with similar patterns, they can share the same dictionary. This means that different wafers or dies can build their templates by using a common base pattern (i.e., dictionary), and the differences are reflected by their respective sparse coefficient vectors. This approach not only greatly reduces the storage space required for the template, but also improves the versatility and flexibility of the template. For example, during the inspection process, if a new die is encountered, we do not need to regenerate a whole new set of templates, but can only calculate a new coefficient vector based on the existing dictionary to represent the uniqueness of this die.

[0016] The comprehensive effects of this application are as follows: 1. Reduce storage requirements: Through the above two methods, the present application effectively reduces the space required for template storage, especially when processing a large number of grains with similar patterns.

[0017] 2. Improved efficiency: Due to the existence of a shared dictionary, the algorithm can process new input grain images faster because a lot of preprocessing work (such as dictionary learning) only needs to be performed once.

[0018] 3. Enhanced adaptability: This approach enables the system to be more flexible in dealing with different types of wafers and dies, and can adapt to new situations by adjusting the coefficient vector without major changes to the underlying model.

[0019] In summary, the present application provides a more efficient and economical way to deal with wafer defect detection problems, which is particularly suitable for applications in large-scale industrial production environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in the present application and should not be regarded as limiting the scope of the present application.

[0021] Figure 1 A flowchart of a template extraction method based on sparse coding according to an embodiment of the present application is shown.

[0022] Figure 2 A schematic diagram of a grain image collected according to an embodiment of the present application is shown.

[0023] Figure 3A schematic diagram of a dictionary according to an embodiment of the present application is shown.

[0024] Figure 4 A schematic diagram of a template combination according to an embodiment of the present application is shown.

[0025] Figure 5 A diagram showing the structure of a template extraction system based on sparse coding according to an embodiment of the present application is shown.

[0026] Figure 6 A schematic structural diagram of an electronic device provided in one embodiment of the present application is shown.

[0027] Figure 7 A schematic diagram of a storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0029] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] When extracting templates, the existing technology mainly relies on pixel-level information for processing, and fails to effectively utilize the rich information provided by the internal structural features of the object. This template extraction method based only on pixel information requires the template to accurately record the specific value of each pixel, which takes up a lot of storage space. Especially in the application scenario of wafer inspection, the pattern on a single grain is often formed by the repeated combination of several basic module patterns. In this case, the method of recording the value of each pixel is not only inefficient, but also greatly wastes storage resources.

[0031] More specifically, during the wafer defect detection process, if the structural information of these recurring basic module patterns can be identified and utilized, the amount of data required to be stored can be greatly reduced. For example, by individually defining and encoding the basic modules and then building the entire grain pattern based on these modules, the storage capacity required for the final template can be significantly reduced. This method not only effectively compresses data, but also improves the speed and accuracy of subsequent image processing because it reduces unnecessary redundant information, allowing the algorithm to focus more on the actual structural features, thereby improving the overall detection performance and efficiency. In addition, the use of structural information can also enhance the system's robustness to local changes or noise, because compared to pure pixel-level comparisons, structural features can better reflect the essential properties of objects.

[0032] like Figure 1 As shown, a template extraction method based on sparse coding of the present application includes the following steps.

[0033] S1. Data preparation In this step, during the data preparation stage for grain defect detection, this application first needs to capture a complete image of the grain to be detected. In order to ensure the accuracy and representativeness of the template, it is recommended to collect as many image samples as possible. Although the minimum requirement is to provide one image as a basic template, increasing the number of images can significantly improve the quality of the template and the accuracy of detection. This is because more image samples can better reflect the various variations that may occur in the grain during the production process, making the final generated template more comprehensive and reliable.

[0034] However, it is worth noting that as the number of images increases, the computational time and resource consumption will also increase accordingly. This is because each additional image requires additional computational steps to align these images and integrate them into the template, whether by calculating the mean image or the median image. Therefore, in actual operation, it is necessary to balance the relationship between the number of images and computational efficiency based on specific application requirements, acceptable processing time, and available computing resources.

[0035] In addition, in order to ensure the accuracy of subsequent processing, all images should be taken with a high enough resolution to capture the subtle features of the grain surface. At the same time, attention should be paid to controlling the consistency of shooting conditions (such as lighting, angle, etc.) to reduce errors caused by changes in external factors and ensure the authenticity and comparability of the acquired images. This step is crucial to building an accurate and efficient template, which directly affects the final defect detection effect.

[0036] like Figure 2 As shown, it is a schematic diagram of the grain image collected in this application.

[0037] S2. Dictionary learning In this step, the goal of this application is to find a set of basic images (the so-called "dictionary") so that the input grain image can be represented in a sparse manner through these basic images. This method can not only effectively capture the key features in the image, but also improve processing efficiency while reducing redundant information.

[0038] In this step, the basic patterns that appear repeatedly in the wafer are identified and stored through dictionary learning methods, reducing the space waste caused by repeatedly storing the same pattern. Each basic image in the dictionary represents a typical pattern or feature on the wafer.

[0039] In dictionary learning, the above goals are usually achieved by optimizing a specific objective function. This objective function aims to minimize the reconstruction error while maximizing sparsity. Specifically, an objective function used in this application is as follows: , Where: represents the input grain image; is the dictionary of base images we are trying to find; represents the coefficient vector, which describes how to approximate the input image by combining the basis images in the dictionary; It is a parameter used to balance reconstruction error and sparsity, and adjust the model's preference for sparsity; It is a parameter used to balance reconstruction error and periodicity, and adjust the model's preference for periodicity; Represents the L2 norm, which is used to measure the difference between matrices; Represents the L1 norm, which is used to promote the sparsity of the solution Indicates Fourier transform of X for calculating the spectrum; Indicates modulus, which is used to calculate the spectrum amplitude; Entroy() means calculating the entropy value of the spectrum. The lower the entropy, the stronger the periodicity. The calculation formula is as follows: , , in and Indicates the frequency position in the spectrum.

[0040] Since wafer defects (such as scratches and particles) may be mistaken for normal patterns, it is necessary to introduce robust optimization objectives in dictionary learning, for example: Norm or Norm substitution norm to suppress the influence of outliers.

[0041] The process of solving the above objective function is usually iterative, and a variety of algorithms can be used. The following is the solution method used in this application: (1) K-SVD (K-Singular Value Decomposition): This is a classic dictionary learning algorithm that updates the dictionary alternately. and the coefficient matrix to gradually approach the optimal solution.

[0042] (2) Gradient descent method: Based on the gradient information of the objective function with respect to the variable, the parameters are adjusted along the negative gradient direction to gradually reduce the objective function value until it converges to the local optimal solution.

[0043] When studying grain images, we need to pay attention to the following: (1) Noise suppression: Wafer images may contain noise due to shooting conditions (such as uneven lighting, lens distortion) or wafer surface defects. The input image must first be subjected to noise reduction processing (such as local histogram matching, median filtering, and distortion correction). Otherwise, the noise may be mistaken for a valid feature, resulting in deviations in dictionary learning results.

[0044] (2) Image alignment: The grain patterns on the wafer need to be strictly aligned (such as through feature point matching or template matching). Otherwise, the misalignment of patterns at different positions will result in the inability to correctly capture the features of repeated modules during dictionary learning.

[0045] (3) Dictionary size: A dictionary that is too large will increase computational complexity and may reduce generalization ability due to overfitting; a dictionary that is too small may not be able to cover the complex patterns of the grains. The appropriate dictionary size should be selected according to the complexity of the wafer design (e.g., determined through pre-experimental cross-validation of different dictionary sizes).

[0046] (4) Diversity of training data: Dictionary training needs to be based on sufficiently diverse normal grain images (covering normal variations in different batches and under different process conditions). Otherwise, it may lead to overfitting and fail to adapt to natural variations in actual detection.

[0047] In this step, the present application can provide a more compact and effective template representation method for grain defect detection through dictionary learning, thereby helping to improve detection accuracy and speed. The dictionary diagram obtained by the present application is shown in FIG. Figure 3 As shown, there are four basic images: A, B, C, and D.

[0048] S3, sparse coding In this step, the process of encoding the grain image using the learned dictionary to obtain a sparse coefficient vector is a key step in dictionary learning applications. This process calculates the corresponding sparse coefficient vector by identifying how the input image is linearly combined from the basis images (i.e. atoms) in the dictionary. The algorithms used in this application include Matching Pursuit (MP) and Orthogonal Matching Pursuit (OMP), both of which are greedy algorithms that aim to select a small number of optimal elements from an overcomplete dictionary to approximate a given image.

[0049] Matching pursuit (MP): Matching pursuit is an iterative algorithm for finding a set of bases in an overcomplete dictionary that best matches the input signal (in this case, the grain image). The basic idea is to select the dictionary atom that is most relevant to the current residual in each iteration and update the residual until a preset stopping condition (such as a sparsity limit or a reconstruction error threshold) is reached.

[0050] Orthogonal Matching Pursuit (OMP): Orthogonal matching pursuit is an improvement on MP. It not only selects the best matching atom at each iteration, but also ensures that the selected atom maintains an orthogonal relationship with all previously selected atoms. The advantage of this is that it can more effectively reduce the reconstruction error and usually obtain better sparse representation results than MP. OMP gradually approximates the original signal by orthogonal projection, and updates the coefficients of all selected atoms in each iteration, making the final sparse coefficient vector more accurate.

[0051] Specifically, the coding process of this application is as follows: (1) Preparation stage: First, you need a dictionary that has been obtained through dictionary learning methods. .

[0052] (2) Input image processing: The complete image of the grain to be encoded is used as input .

[0053] (3) Algorithm selection: Select an appropriate algorithm according to the specific situation (for example, MP is used when the image is relatively simple or the reconstruction accuracy requirement is not high, and OMP is used when the image is relatively complex or the reconstruction accuracy requirement is high) to perform sparse coding.

[0054] (4) Execute the code: Run the selected algorithm and use the dictionary For the input image Encode and get the sparse coefficient vector .

[0055] (5) Verification and optimization: Check the quality of the obtained sparse representation (the smaller the image reconstruction error, the better; the greater the sparsity, the better); if necessary, adjust parameters or try different algorithms to optimize the results.

[0056] In this way, the complex grain image can be effectively represented as a sparse linear combination of its corresponding basis image in the dictionary. The encoding result is as follows, where v represents the non-zero value in the coefficient, and x and y represent the coordinates of the non-zero value: Coefficient vector: [(v_A, x_A, y_A), (v_B, x_B, y_B), (v_C1, x_C1, y_C1), (v_C2, x_C2,y_C2), …, (v_C2, x_C9, y_C9), (v_D, x_D, y_D)] In this step, the present application uses matching pursuit (MP), orthogonal matching pursuit (OMP) and other algorithms to encode the grain image and obtain a sparse coefficient vector, which indicates how to use the combination of basis images in the dictionary to approximate the input image. Only the non-zero values ​​and their positions in the coefficient vector are recorded, thereby further reducing storage requirements.

[0057] S4. Template combination In this step, after completing dictionary learning and sparse coding, the next step is to combine the dictionary, the non-zero values ​​in the coefficient vector and their positions to form a comprehensive die template. This step is crucial for subsequent die positioning and defect detection on the wafer. The following is a detailed process: (1) Record dictionary: This is a collection of base images obtained through the dictionary learning process. Each base image represents a certain feature or pattern in the grain pattern. The dictionary is the core component of the template because it provides all the "building blocks" needed to build a complete grain image.

[0058] (2) Record the non-zero values ​​and their positions in the coefficient vector: It is necessary to record all non-zero values ​​in the coefficient vector and their specific positions. This information indicates which specific basis images (i.e. atoms in the dictionary) are used to reconstruct the original image, as well as the weight of each atom's contribution to the final image.

[0059] (3) Combine into a grain template: The non-zero values ​​and their positions in the dictionary and coefficient vector are combined to form a complete grain template. This template not only contains all the basic information required to reconstruct the grain image, but also significantly reduces the storage space required because it is not necessary to save the entire coefficient vector, but only those non-zero values ​​and their positions.

[0060] Finally, the template combination obtained by one embodiment of the present application is as follows Figure 4shown.

[0061] In this application, for wafers with similar patterns, the same dictionary can be shared, and only different coefficient vectors need to be calculated for each wafer or die to construct its template. This method not only saves storage space, but also improves processing efficiency and the versatility of the template.

[0062] The application embodiment provides a template extraction system based on sparse coding, which is used to execute the template extraction method based on sparse coding described in the above embodiment, such as Figure 5 As shown, the system includes: A data preparation module 501 is used to capture at least one complete image of a grain to be inspected; A dictionary learning module 502 is used to determine a set of basic images as a dictionary by optimizing and solving an objective function in the at least one complete image, and to train and learn the dictionary; A coefficient encoding module 503, for encoding the grain image using the learned dictionary to obtain a sparse coefficient vector; The template combination module 504 is used to combine the non-zero values ​​and their positions in the dictionary and the sparse coefficient vector to form a comprehensive grain template.

[0063] The sparse coding-based template extraction system provided in the above-mentioned embodiment of the present application and the sparse coding-based template extraction method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0064] The embodiment of the present application also provides an electronic device corresponding to the template extraction method based on sparse coding provided in the above embodiment, so as to execute the template extraction method based on sparse coding. The embodiment of the present application is not limited.

[0065] Please refer to Figure 6 , which shows a schematic diagram of an electronic device provided by some embodiments of the present application. Figure 6 As shown, the electronic device 20 includes: a processor 200, a memory 201, a bus 202 and a communication interface 203, and the processor 200, the communication interface 203 and the memory 201 are connected via the bus 202; the memory 201 stores a computer program that can be run on the processor 200, and when the processor 200 runs the computer program, it executes the sparse coding-based template extraction method provided in any of the aforementioned embodiments of the present application.

[0066] The memory 201 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 203 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0067] The bus 202 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs, and the processor 200 executes the programs after receiving the execution instruction. The template extraction method based on sparse coding disclosed in any implementation of the embodiment of the present application may be applied to the processor 200, or implemented by the processor 200.

[0068] The processor 200 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 200. The above processor 200 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and completes the steps of the above method in combination with its hardware.

[0069] The electronic device provided in the embodiment of the present application and the template extraction method based on sparse coding provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented therein.

[0070] The present application also provides a computer-readable storage medium corresponding to the template extraction method based on sparse coding provided in the above embodiment. Figure 7 The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is executed by a processor, the template extraction method based on sparse coding provided in any of the aforementioned embodiments is executed.

[0071] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0072] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the sparse coding-based template extraction method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0073] It should be noted that: The algorithms and displays provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems may also be used together with the teachings based thereon. It is apparent from the above description that the structure required for constructing such systems is not intended for any particular programming language. In addition, the present application is not intended for any particular programming language either. It should be understood that the content of the present application described herein may be implemented using various programming languages, and the description of the particular language above is intended to disclose the best mode of implementation of the present application.

[0074] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0075] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0076] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0077] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0078] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components in the creation system of the virtual machine according to the embodiment of the present application. The present application can also be implemented as a device or system program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0079] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application may be implemented by means of hardware including several different elements and by means of appropriately programmed computers. In a unit claim that lists several systems, several of these systems may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

[0080] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A template extraction method based on sparse coding, characterized in that: include: Taking at least one complete image of the grain to be inspected; In the at least one complete image, a set of basic images is determined as a dictionary by optimizing and solving an objective function, and the dictionary is trained and learned; Encode the grain image using the learned dictionary to obtain a sparse coefficient vector; The dictionary and the non-zero values ​​and their positions in the sparse coefficient vector are combined to form a comprehensive grain template.

2. The method according to claim 1, characterized in that The capturing of at least one complete image of the grain to be inspected further comprises: Balance the number of images and computational efficiency based on specific application requirements, acceptable processing time, and available computing resources; Ensure that all images have a preset resolution when taking pictures to capture the subtle features of the grain surface; Control the consistency of shooting conditions to reduce errors caused by changes in external factors and ensure the authenticity and comparability of the acquired images.

3. The method according to claim 1, characterized in that The objective function is used to maximize sparsity while minimizing the reconstruction error, and the objective function has the following form: ， Where: represents the input grain image; is the dictionary that attempts to determine the base image composition; represents the coefficient vector, which describes how to approximate the input image by combining the basis images in the dictionary; It is a parameter used to balance reconstruction error and sparsity, and adjust the model's preference for sparsity; It is a parameter used to balance reconstruction error and periodicity, and adjust the model's preference for periodicity; Represents the L2 norm, which is used to measure the difference between matrices; represents the L1 norm, which is used to promote the sparsity of the solution; Indicates Fourier transform of X for calculating the spectrum; Indicates modulus, which is used to calculate the spectrum amplitude; Entroy() means calculating the entropy value of the spectrum. The lower the entropy, the stronger the periodicity. The calculation formula is as follows: ， ， in and Indicates the frequency position in the spectrum.

4. The method according to claim 1, characterized in that: The algorithm for optimizing and solving the objective function is a K-SVD algorithm or a gradient descent method.

5. The method according to claim 1, characterized in that The training and learning of the dictionary includes: Perform noise reduction on the input image; Strictly align the die patterns on the wafer; Choose the appropriate dictionary size based on the complexity of the wafer design; The training and learning are performed based on normal grain images covering different batches and under different process conditions.

6. The method according to claim 1, characterized in that The specific process of the encoding is: (1) Preparation stage: First, obtain a dictionary that has been learned through dictionary learning ; (2) Input image processing: The complete image of the grain to be encoded is used as input ; (3) Select algorithm: Select a preset algorithm for sparse coding; (4) Execute the code: Run the preset algorithm and use the dictionary For the input image Encode and get the sparse coefficient vector ; (5) Verification and optimization: Check the obtained sparse coefficient vector quality, adjust parameters or use different algorithms to optimize the results.

7. The method according to any one of claims 1 to 6, characterized in that: The non-zero values ​​and positions in the sparse coefficient vector are used to indicate which base images are used to reconstruct the original image, and the weight of each base image's contribution to the final image.

8. A template extraction system based on sparse coding, characterized in that: include: A data preparation module, used for taking at least one complete image of the grain to be inspected; A dictionary learning module, used to determine a set of basic images as a dictionary by optimizing and solving an objective function in the at least one complete image, and to train and learn the dictionary; A coefficient encoding module, for encoding the grain image using the learned dictionary to obtain a sparse coefficient vector; The template combination module is used to combine the non-zero values ​​and their positions in the dictionary and the sparse coefficient vector to form a comprehensive grain template.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor runs the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images