Optical crystal defect detection method and system

Abstract

The invention relates to the technical field of crystal detection, in particular to an optical crystal defect detection method and system.The method comprises the steps that firstly, an optical crystal to be detected is preprocessed to obtain an SEM sample, and SEM image collection is conducted to obtain an SEM crystal image; preprocessing the same optical crystal to be detected to obtain a TEM sample, and performing TEM image acquisition to obtain a TEM crystal image; the SEM crystal image and the TEM crystal image are paired one by one to obtain an optical crystal image pair, and image preprocessing including graying, denoising and normalization processing is carried out; feature extraction and feature matching are carried out on the preprocessed optical crystal image pairs, and a spatial corresponding relation between the optical crystal image pairs is established; and setting a reasonable difference threshold value to identify a defect area and carrying out defect positioning and classification. By combining the SEM and the TEM, comprehensive and efficient defect detection from the surface to the interior of the optical crystal is realized, and the accuracy of optical crystal detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of crystal detection, and particularly relates to a method and system for detecting optical crystal defects. Background Art

[0002] As a core component of many optical devices, the defects inside and on the surface of an optical crystal will seriously affect the performance of the device. Traditional detection techniques cannot simultaneously take into account the defect detection of the crystal surface and the interior, and the detection accuracy and comprehensiveness are poor. A scanning electron microscope (SEM) can provide high-resolution images of the crystal surface, while a transmission electron microscope (TEM) can deeply obtain the internal structural information of the crystal. However, previous methods have not effectively integrated the SEM and TEM image information, making it difficult to comprehensively and accurately detect optical crystal defects. Therefore, developing a method for detecting optical crystal defects by integrating SEM and TEM images is of great practical significance for improving the detection efficiency and accuracy. Summary of the Invention

[0003] In view of the above technical deficiencies, the present invention provides a method and system for detecting optical crystal defects to achieve comprehensive and accurate detection of optical crystal defects.

[0004] The present invention is achieved through the following technical solutions:

[0005] A method for detecting optical crystal defects is provided, and the method includes the following steps:

[0006] Step S10: Pretreat the optical crystal to be tested and make a unique mark. After marking, fix the optical crystal sample to be tested on the SEM sample stage to collect the surface image of the optical crystal, obtain the SEM crystal image to be tested, and associate and store it with the corresponding mark information;

[0007] Step S20: After the collection of the SEM crystal image to be tested is completed, perform ultra-thin sectioning on the optical crystal sample to be tested, control the section thickness within 50 - 100 nm, fix the section sample on the TEM sample stage to collect the internal image of the optical crystal, obtain the TEM crystal image to be tested, and associate and store it with the corresponding mark information;

[0008] Step S30: Pair the SEM crystal image to be tested and the TEM crystal image to be tested one by one according to the mark information to obtain the optical crystal image pair to be tested, and perform image preprocessing on each image pair;

[0009] Step S40: Use the scale-invariant feature transform algorithm to extract feature points and feature descriptors from the SEM crystal image to be tested and the TEM crystal image to be tested in the image pair. Based on the extracted feature points and feature descriptors, use the nearest neighbor matching algorithm to calculate the mutually matching feature point pairs and establish the spatial correspondence relationship between the image pairs;

[0010] Step S50: Identify the defective areas by setting a reasonable difference threshold, and perform defect location and classification.

[0011] Preferably, in step S10, the optical crystal to be tested is pre-treated and marked with a unique identifier. The pre-treatment is to remove contaminants such as impurities, oil stains, and dust on the surface of the optical crystal by using a suitable cleaning method, including ultrasonic cleaning and chemical reagent cleaning. Ultrasonic cleaning is to put the sample into a solution containing an appropriate cleaning agent, and the contaminants are removed by the vibration of ultrasonic waves; chemical cleaning is to select a suitable chemical reagent for soaking or wiping according to the nature of the contaminants. When selecting a chemical reagent, it should be noted that the selected chemical reagent cannot react with the optical crystal to avoid damaging the crystal surface; after cleaning, the optical crystal is dried to remove the surface moisture. The natural drying method can be used, that is, the sample is placed in a clean and well-ventilated environment to allow the moisture to evaporate naturally; the drying method can also be used, such as blowing dry the sample with nitrogen or a dry air stream to avoid the influence of residual moisture on the test results; after the drying treatment, the optical crystal sample is sputtered with gold or carbon. Through a vacuum coating device, a thin layer of gold or carbon film is evenly coated on the surface of the sample, and the thickness is generally between a few nanometers and dozens of nanometers.

[0012] Preferably, the steps of image pre-processing for each image pair in step S30 include:

[0013] Grayscale processing of the image pair: Use the function in the OpenCV library in the image processing software to read the collected image pair of the optical crystal to be tested. According to the characteristics of the image pair of the optical crystal to be tested, the weighted average method is used to calculate the grayscale value according to the weights of the RGB three channels, and each pixel point in the image pair of the optical crystal to be tested is traversed to calculate and replace the original RGB value to obtain the grayscale image of the optical crystal to be tested;

[0014] Denoising processing of the image pair: Gaussian filtering is used to perform denoising processing on the grayscale image of the optical crystal to be tested, and the values of the filter kernel size and standard deviation are determined according to the noise distribution characteristics of the grayscale image of the optical crystal to be tested;

[0015] Normalization processing: Normalize the images in the image pair, map the grayscale values of the images to the unified range of [0,1], and eliminate the problems of inconsistent image brightness and contrast caused by differences in acquisition equipment and parameters.

[0016] Preferably, the steps of extracting feature points and feature descriptors from the SEM crystal image to be tested and the TEM crystal image to be tested in the image pair by using the scale-invariant feature transform algorithm in step S40 include:

[0017] Scale - space extreme value detection: Convolve the SEM crystal image to be measured and the TEM crystal image to be measured in the image pair with Gaussian kernels of different scales to obtain a series of images at different scales, which form a Gaussian pyramid. Subtract the Gaussian pyramid images of adjacent two layers to obtain a difference Gaussian pyramid. In the difference pyramid, each pixel point is compared with its 26 adjacent pixel points (8 adjacent pixel points in the same layer and 9 adjacent pixel points in each of the upper and lower layers) to find local extreme points as candidate feature points;

[0018] Accurate localization of feature points: For each candidate feature point, use the pixel points around it to fit a three - dimensional quadratic function to accurately determine the position and scale of the feature point. According to the result of the quadratic function fitting, calculate the contrast of the feature point and set a threshold, and remove the points with contrast less than the threshold to improve the stability and reliability of the feature points. Judge whether the point is an edge response point by calculating the Hessian matrix at the feature point, and remove the feature points judged as edge response points. Feature points on the edge are more sensitive to noise and small changes, which is not conducive to feature matching;

[0019] Generation of feature descriptors: With the accurately located feature point as the center, divide its neighborhood into a certain number of sub - regions. Calculate the gradient direction histogram in each sub - region, and statistically analyze the gradient magnitude distribution in different directions in this sub - region. Combine the gradient histograms of all sub - regions to obtain a feature descriptor. This feature descriptor describes the gradient distribution characteristics in the neighborhood of the feature point and has scale, rotation, and illumination invariance.

[0020] Preferably, the step of calculating the mutually matching feature point pairs based on the extracted feature points and feature descriptors and establishing the spatial correspondence relationship between the image pairs in step S40 includes:

[0021] Calculation of the distance between feature point descriptors: For each feature point in the SEM crystal image to be measured, use the Euclidean distance to calculate its distance from each feature point descriptor in the TEM crystal image to be measured. For example, as shown in Equation (1):

[0022]

[0023] where xi and yi are the corresponding elements of the two feature descriptors respectively, and n is the dimension of the feature descriptor;

[0024] Determination of the nearest neighbor and the second - nearest neighbor: For each feature point in the SEM crystal image to be measured, after calculating the distance from each feature point descriptor in the TEM crystal image to be measured, find the feature point with the shortest distance as its nearest neighbor, and at the same time find the feature point with the second - shortest distance as its second - nearest neighbor;

[0025] Determine matching point pairs: Determine matching point pairs by comparing the ratio of the nearest neighbor distance to the second nearest neighbor distance. When the ratio of the nearest neighbor distance to the second nearest neighbor distance is less than a certain threshold (such as 0.8), it is determined that the nearest neighbor match is reliable, and these two feature points are used as a matching point pair; when the ratio of the nearest neighbor distance to the second nearest neighbor distance is greater than or equal to a certain threshold (such as 0.8), it is determined that the match is unreliable, discard this matching point pair, and exclude some incorrect matches caused by low similarity of feature points.

[0026] Preferably, in step S50, the defective area is identified by setting a reasonable difference threshold, and defect location and classification are performed, including:

[0027] Training sample collection and annotation: Collect a large number of optical crystal SEM and TEM image pairs covering different types, sizes, and severity levels of defects, and accurately annotate the defective areas in the images by professionals. Divide the image pairs and annotation information into a training set and a test set;

[0028] Feature difference calculation: For each pair of optical crystal SEM and TEM images in the training set, use the method in step S40 to calculate feature points, feature descriptors, and matching feature point pairs. For each pair of matching feature points, calculate the differences in their positions, gray values, and surrounding neighborhood structures, etc.;

[0029] Threshold determination: Perform statistical analysis on the difference data of all matching feature points in the training set, draw a distribution histogram of the difference values, and determine the difference threshold that can effectively distinguish the defective area from the normal area according to the defect annotation information;

[0030] Optical crystal image pair detection: Calculate the difference value according to the matching feature point pairs obtained in step S40, compare the calculated difference value with the set threshold. When the difference value of a certain matching feature point pair is greater than the threshold, it is preliminarily determined that the neighborhood where the feature point is located is a potential defective area. A neighborhood of a certain size centered on this feature point (such as a circular neighborhood with a radius of 5 - 10 pixels) can be set as the potential defective area;

[0031] Region merging and screening: Perform merging processing on the preliminarily determined potential defective areas, remove isolated and overly small areas, which may be caused by noise or mis-matching. Use morphological operations, such as dilation and erosion, to optimize the potential defective areas. Finally, further analyze the merged and screened areas to determine the final possible defective areas.

[0032] In addition, to achieve the above object, the present invention also proposes an optical crystal defect detection system, and the optical crystal defect detection system includes:

[0033] SEM image acquisition module for the optical crystal to be tested: It is used to preprocess the optical crystal to be tested and make a unique mark. After the mark, the optical crystal sample to be tested is fixed on the SEM sample stage to collect the surface image of the optical crystal, obtain the SEM crystal image to be tested, and associate and store it with the corresponding mark information;

[0034] TEM image acquisition module for the optical crystal to be tested: After the acquisition of the SEM crystal image to be tested is completed, it is used to perform ultra-thin sectioning on the optical crystal sample to be tested, control the section thickness within 50 - 100 nm, fix the section sample on the TEM sample stage to collect the internal image of the optical crystal, obtain the TEM crystal image to be tested, and associate and store it with the corresponding mark information;

[0035] Preprocessing module for the image pair of the optical crystal to be tested: It is used to pair the SEM crystal image to be tested and the TEM crystal image to be tested one by one according to the mark information to obtain the image pair of the optical crystal to be tested, and perform image preprocessing on each image pair;

[0036] Feature extraction and matching module for the image pair of the optical crystal to be tested: It is used to extract feature points and feature descriptors from the SEM crystal image to be tested and the TEM crystal image in the image pair by using the scale-invariant feature transform algorithm, calculate the mutually matching feature point pairs by using the nearest neighbor matching algorithm based on the extracted feature points and feature descriptors, and establish the spatial correspondence relationship between the image pairs;

[0037] Optical crystal defect detection module: It is used to identify the defect area, locate and classify the defects by setting a reasonable difference threshold.

[0038] In addition, to achieve the above object, the present invention also proposes an optical crystal defect detection device, and the device includes: a memory, a processor, and programs such as an optical crystal defect detection algorithm stored on the memory and executable on the processor. The programs such as the optical crystal defect detection algorithm are for implementing the steps of an optical crystal defect detection method as described above.

[0039] In addition, to achieve the above object, the present invention also provides a computer program product, and the computer program product includes programs such as an optical crystal defect detection algorithm. When the programs such as the optical crystal defect detection algorithm are executed by a processor, they implement an optical crystal defect detection method as described above.

[0040] The advantages and effects of the present invention are:

[0041] An optical crystal defect detection method and system proposed by the present invention can simultaneously obtain the microscopic structure information of the surface and interior of the optical crystal by constructing and analyzing SEM and TEM image pairs, making up for the deficiency that traditional detection methods can only detect single information on the surface or interior, and providing rich data support for comprehensively evaluating the crystal quality; in addition, by means of the feature extraction and matching technology of the image pair, it can accurately identify and locate the tiny defects in the crystal, greatly improving the detection accuracy and reliability, and meeting the requirements of modern high-precision optical crystal production and application. Brief Description of the Drawings

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 It is a flowchart of an optical crystal defect detection method of the present invention.

[0044] Figure 2 It is a schematic structural diagram of an optical crystal defect detection system of the present invention.

[0045] Figure 3 It is a schematic block diagram of the structure of an electronic device for optical crystal defect detection of the present invention. Detailed Embodiments

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0047] As Figure 1 shown, in an embodiment of the present invention, an optical crystal defect detection method includes the following steps:

[0048] Step S10: Pretreat the optical crystal to be measured and make a unique mark. After marking, fix the optical crystal sample to be measured on the SEM sample stage, ensure that the sample surface is flat and facing the electron beam directly, and reasonably adjust parameters such as the acceleration voltage (usually 5 - 30 kV), working distance, and beam current of the SEM according to the sample characteristics. Collect the surface image of the optical crystal to obtain the SEM crystal image to be measured, and associate and store it with the corresponding mark information. For example, fix the marked optical crystal sample on the SEM sample stage, adjust the SEM acceleration voltage to 15 kV, the working distance to 8 mm, and the beam current to 8 μA, collect the surface image of the crystal, and record the mark information corresponding to the image.

[0049] Specifically, in step S10, the optical crystal to be measured is pretreated and a unique mark is made. The pretreatment is to remove contaminants such as impurities, oil stains, and dust on the surface of the optical crystal by using a suitable cleaning method, including ultrasonic cleaning and chemical reagent cleaning. Ultrasonic cleaning is to put the sample into a solution containing an appropriate cleaning agent, and make the contaminants fall off through the vibration of ultrasonic waves; chemical cleaning is to select a suitable chemical reagent for soaking or wiping according to the nature of the contaminants. When selecting a chemical reagent, it should be noted that the selected chemical reagent cannot react with the optical crystal to avoid damaging the crystal surface; after cleaning, the optical crystal is dried to remove the surface moisture. It can be dried naturally by placing the sample in a clean and well-ventilated environment to let the moisture evaporate naturally; it can also use the method of blowing dry, such as blowing dry the sample with nitrogen or a dry air stream to avoid the influence of moisture residue on the detection result; after the drying treatment, the optical crystal sample is sputter-coated with gold or carbon. Through a vacuum coating device, a thin layer of gold or carbon film is evenly coated on the sample surface, and the thickness is generally between a few nanometers and dozens of nanometers.

[0050] Step S20: After the collection of the SEM crystal image to be measured is completed, perform ultramicrotomy on the optical crystal sample to be measured, control the section thickness within 50 - 100 nm, fix the section sample on the TEM sample stage, and adjust parameters such as the acceleration voltage (generally 100 - 300 kV) of the TEM to collect the internal image of the optical crystal to obtain the TEM crystal image to be measured, and associate and store it with the corresponding mark information. For example, perform ultramicrotomy on the optical crystal sample to be measured, with the section thickness of 60 nm, place the section sample on the TEM sample holder, adjust the TEM acceleration voltage to 200 kV, collect the internal image of the crystal, and record the corresponding mark information.

[0051] Step S30: Pair the SEM crystal image to be measured and the TEM crystal image to be measured one by one according to the mark information to obtain the optical crystal image pair to be measured, and perform image preprocessing on each image pair.

[0052] Specifically, the steps of performing image preprocessing on each image pair in step S30 include:

[0053] Grayscale processing of the image pair: Use the functions in the OpenCV library in the image processing software to read the collected optical crystal image pair to be measured. According to the characteristics of the optical crystal image pair to be measured, the weighted average method is adopted to calculate the grayscale value based on the weights of the RGB three channels, and each pixel point in the optical crystal image pair to be measured is traversed to calculate and replace the original RGB value, obtaining the grayscale image of the optical crystal to be measured;

[0054] Denoising processing of the image pair: Gaussian filtering is used to perform denoising processing on the grayscale image of the optical crystal to be measured, and the values of the filter kernel size and standard deviation are determined according to the noise distribution characteristics of the grayscale image of the optical crystal to be measured;

[0055] Normalization processing: Normalize the images in the image pair, map the grayscale values of the images to the unified range of [0,1], and eliminate the problems of inconsistent image brightness and contrast caused by differences in acquisition equipment and parameters.

[0056] Step S40: Use the Scale-Invariant Feature Transform (SIFT) algorithm to extract feature points and feature descriptors from the SEM crystal image to be measured and the TEM crystal image to be measured in the image pair. Based on the extracted feature points and feature descriptors, the nearest neighbor matching algorithm is used to calculate the mutually matching feature point pairs, and the spatial correspondence relationship between the image pairs is established.

[0057] Specifically, the steps of using the Scale-Invariant Feature Transform (SIFT) algorithm to extract feature points and feature descriptors from the SEM crystal image to be measured and the TEM crystal image to be measured in step S40 include:

[0058] Scale-space extreme value detection: Convolve the SEM crystal image to be measured and the TEM crystal image to be measured in the image pair with Gaussian kernels of different scales to obtain a series of images at different scales, forming a Gaussian pyramid. By subtracting the adjacent two-layer Gaussian pyramid images, a Difference-of-Gaussian (DoG) pyramid is obtained. In the DoG pyramid, each pixel point is compared with its 26 adjacent pixel points (8 adjacent pixel points in the same layer and 9 adjacent pixel points in the upper and lower layers) to find the local extreme points as candidate feature points;

[0059] Precise localization of feature points: For each candidate feature point, use the pixel points around it to fit a three-dimensional quadratic function to accurately determine the position and scale of the feature point. According to the results of the quadratic function fitting, calculate the contrast of the feature point and set a threshold to remove the points with a contrast less than the threshold to improve the stability and reliability of the feature points. Determine whether the point is an edge response point by calculating the Hessian matrix at the feature point, and remove the feature points determined to be edge response points. The feature points on the edge are more sensitive to noise and small changes, which is not conducive to feature matching;

[0060] Feature descriptor generation: Centered on the accurately located feature points, divide their neighborhoods into a certain number of sub-regions. Calculate the histogram of gradient directions within each sub-region, and statistically analyze the distribution of gradient magnitudes in different directions within the sub-region. Combine the gradient histograms of all sub-regions to obtain a feature descriptor, which describes the gradient distribution characteristics within the neighborhood of the feature points and has scale, rotation, and illumination invariance.

[0061] Specifically, the steps of calculating the mutually matching feature point pairs based on the extracted feature points and feature descriptors and establishing the spatial correspondence relationship between image pairs in step S40 include:

[0062] Calculating the distance between feature point descriptors: For each feature point in the SEM crystal image to be measured, use the Euclidean distance to calculate its distance from each feature point descriptor in the TEM crystal image to be measured. For example, as shown in Equation (1):

[0063]

[0064] where xi and yi are the corresponding elements of the two feature descriptors respectively, and n is the dimension of the feature descriptor;

[0065] Determining the nearest neighbor and the second nearest neighbor: For each feature point in the SEM crystal image to be measured, after calculating the distance from each feature point descriptor in the TEM crystal image to be measured, find the feature point with the shortest distance as its nearest neighbor, and at the same time find the feature point with the second shortest distance as its second nearest neighbor;

[0066] Determining the matching point pairs: Determine the matching point pairs by comparing the ratio of the nearest neighbor distance to the second nearest neighbor distance. When the ratio of the nearest neighbor distance to the second nearest neighbor distance is less than a certain threshold (such as 0.8), it is judged that this nearest neighbor match is reliable, and these two feature points are used as a matching point pair; when the ratio of the nearest neighbor distance to the second nearest neighbor distance is greater than or equal to a certain threshold (such as 0.8), it is judged that this match is unreliable, discard this matching point pair, and exclude some incorrect matches caused by low similarity of feature points.

[0067] Step S50: Identify the defect regions by setting a reasonable difference threshold and perform defect location and classification.

[0068] Specifically, in step S50, identifying the defect regions by setting a reasonable difference threshold and performing defect location and classification includes:

[0069] Training sample collection and annotation: Collect a large number of optical crystal SEM and TEM image pairs covering different types, sizes, and severity levels of defects, and accurately annotate the defect regions in the images by professionals. Divide the image pairs and annotation information into a training set and a test set;

[0070] Feature difference calculation: For each pair of SEM and TEM images of optical crystals in the training set, the methods in step S40 are used to calculate feature points, feature descriptors, and pairs of matched feature points. For each pair of matched feature points, calculate the differences in aspects such as position, gray value, and surrounding neighborhood structure. For example, calculate the coordinate difference of the feature points in the image coordinate system, and the mean square error of the gray values of the neighborhood pixels centered on the feature points, etc.;

[0071] Threshold determination: Statistically analyze the difference data of all matched feature points in the training set, draw a distribution histogram of the difference values, and determine the difference threshold that can effectively distinguish the defective area and the normal area according to the defect annotation information. For example, use a statistics-based method to select the upper limit of the difference values of feature points in a certain proportion (such as 95%) of the normal area as the threshold; or use a binary classification algorithm in machine learning, such as logistic regression, to train the difference values of the feature points to obtain the optimal threshold for distinguishing defects and normal areas;

[0072] Detection of optical crystal image pairs: Calculate the difference value according to the pairs of matched feature points obtained in step S40, and compare the calculated difference value with the set threshold. When the difference value of a pair of matched feature points is greater than the threshold, preliminarily determine that the neighborhood where the feature point is located is a potential defective area, and a neighborhood of a certain size centered on the feature point (such as a circular neighborhood with a radius of 5-10 pixels) can be set as the potential defective area;

[0073] Region merging and screening: Merge the preliminarily determined potential defective areas, and remove isolated and overly small areas, which may be caused by noise or mis-matching. Use morphological operations, such as dilation and erosion, to optimize the potential defective areas, and finally further analyze the merged and screened areas to determine the final areas where defects may exist.

[0074] In addition, as Figure 2 shown, in an embodiment of the present invention, an optical crystal defect detection system is proposed. The optical crystal defect detection system includes:

[0075] SEM image acquisition module for the optical crystal to be measured: Used to preprocess the optical crystal to be measured and make a unique mark. After marking, fix the optical crystal sample on the SEM sample stage to collect the surface image of the optical crystal, obtain the SEM crystal image to be measured, and associate and store it with the corresponding mark information;

[0076] TEM image acquisition module for the optical crystal to be measured: After the SEM crystal image acquisition of the optical crystal to be measured is completed, the optical crystal sample to be measured is subjected to ultra-thin sectioning, and the section thickness is controlled within 50 - 100 nm. The section sample is fixed on the TEM sample stage to acquire the internal image of the optical crystal, obtaining the TEM crystal image to be measured, and associating and storing it with the corresponding marking information;

[0077] Preprocessing module for the image pairs of the optical crystal to be measured: Used to pair the SEM crystal image to be measured and the TEM crystal image to be measured one by one according to the marking information to obtain the image pairs of the optical crystal to be measured, and perform image preprocessing on each image pair;

[0078] Feature extraction and matching module for the image pairs of the optical crystal to be measured: Used to extract feature points and feature descriptors from the SEM crystal image to be measured and the TEM crystal image to be measured in the image pair by using the scale-invariant feature transform algorithm, calculate the mutually matching feature point pairs based on the extracted feature points and feature descriptors, and establish the spatial correspondence relationship between the image pairs;

[0079] Optical crystal defect detection module: Used to identify the defect area, locate and classify the defects by setting a reasonable difference threshold.

[0080] An optical crystal defect detection system provided by the present application adopts an optical crystal defect detection method in the above-mentioned embodiment, and can solve the technical problems of low accuracy and single detection of traditional optical crystal defect detection methods. Compared with the prior art, the beneficial effects of an optical crystal defect detection system provided by the present application are the same as those of an optical crystal defect detection method provided by the above-mentioned embodiment, and other technical features in the optical crystal defect detection system are the same as those disclosed in the method of the above-mentioned embodiment, and will not be elaborated here.

[0081] The present application provides an optical crystal defect detection device, and the optical crystal defect detection device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute an optical crystal defect detection method in the first embodiment above.

[0082] As Figure 3As shown, in an embodiment of the present invention, a schematic structural diagram of an optical crystal defect detection device suitable for implementing the embodiments of the present application is shown. An optical crystal defect detection device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description: tablet computers), PMPs (Portable Media Player), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The shown optical crystal defect detection device is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0083] Figure 3 The shown optical crystal defect detection device may include a processing system 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage system 1003 into the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of an optical crystal defect detection device are also stored. The processing system 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input system 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output system 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage system 1003 including, for example, magnetic tapes, hard disks, etc.; and a communication system 1009. The communication system 1009 may allow an optical crystal defect detection device to communicate with other devices wirelessly or wiredly to exchange data. Although an optical crystal defect detection device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems may be implemented or had alternatively.

[0084] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication system, or installed from a storage system 1003, or installed from a ROM 1002. When the computer program is executed by a processing system 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0085] An optical crystal defect detection device provided by the present application adopts an optical crystal defect detection method in the above embodiment, and can solve the technical problems of low accuracy and single detection in the traditional optical crystal defect detection method. Compared with the prior art, the beneficial effects of the optical crystal defect detection device provided by the present application are the same as those of the optical crystal defect detection method provided by the above embodiment, and other technical features in the optical crystal defect detection device are the same as the features disclosed in the method of the previous embodiment, and will not be elaborated here.

[0086] Each part disclosed in the present application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0087] The present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of an optical crystal defect detection method as described above are implemented.

[0088] The computer program product provided by the present application can solve the technical problems of low accuracy and single detection in the traditional optical crystal defect detection method. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the optical crystal defect detection method provided by the above embodiment, and will not be elaborated here.

[0089] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. An optical crystal defect detection method, characterized in that, The method includes the following steps: Step S10: Pretreat the optical crystal to be measured and make a unique mark. After the marking, fix the optical crystal sample to be measured on the SEM sample stage to collect the surface image of the optical crystal, obtain the SEM crystal image to be measured, and store it in association with the corresponding marking information; Step S20: After the collection of the SEM crystal image to be measured is completed, perform slicing on the optical crystal sample to be measured, control the slicing thickness within 50 - 100 nm, fix the sliced sample on the TEM sample stage to collect the internal image of the optical crystal, obtain the TEM crystal image to be measured, and store it in association with the corresponding marking information; Step S30: Pair the SEM crystal image to be measured and the TEM crystal image to be measured one by one according to the marking information to obtain the optical crystal image pair to be measured, and perform image preprocessing on each image pair; Step S40: Use the Scale - Invariant Feature Transform (SIFT) algorithm to extract feature points and feature descriptors from the SEM crystal image to be measured and the TEM crystal image to be measured in the image pair. Based on the extracted feature points and feature descriptors, use the nearest - neighbor matching algorithm to calculate the mutually - matching feature point pairs, and establish the spatial correspondence relationship between the image pairs; Step S50: Identify the defect area by setting a reasonable difference threshold, and perform defect location and classification.

2. The method for detecting optical crystal defects according to claim 1, wherein In the step S10, when pretreating the optical crystal to be measured and making a unique mark, the pretreatment is to use a suitable cleaning method to remove impurities, oil stains, and dust on the surface of the optical crystal, including ultrasonic cleaning and chemical reagent cleaning; after cleaning, perform drying treatment on the optical crystal to remove the surface moisture; after the drying treatment, perform gold spraying or carbon spraying on the optical crystal sample.

3. The method for detecting optical crystal defects according to claim 1, characterized in that, The steps of performing image preprocessing on each image pair in the step S30 include: Grayscale processing of the image pair: Use the function in the OpenCV library in the image processing software to read the collected optical crystal image pair to be measured. According to the characteristics of the optical crystal image pair to be measured, use the weighted average method to calculate the grayscale value based on the weights of the RGB three channels, traverse each pixel point in the optical crystal image pair to be measured, calculate and replace the original RGB value, and obtain the grayscale image of the optical crystal to be measured; Denoising processing of the image pair: Use Gaussian filtering to perform denoising processing on the grayscale image of the optical crystal to be measured, and determine the values of the filter kernel size and standard deviation according to the noise distribution characteristics of the grayscale image of the optical crystal to be measured; Normalization processing: Normalize the images in the image pair, and map the grayscale values of the images to the unified range of [0, 1].

4. The method for detecting optical crystal defects according to claim 1, wherein The steps of using the Scale - Invariant Feature Transform (SIFT) algorithm to extract feature points and feature descriptors from the SEM crystal image to be measured and the TEM crystal image to be measured in the step S40 include: Scale - space extreme value detection: Convolve the SEM crystal image to be measured and the TEM crystal image to be measured in the image pair with Gaussian kernels of different scales to obtain a series of images at different scales, which form a Gaussian pyramid. Subtract the adjacent two - layer Gaussian pyramid images to obtain a difference Gaussian pyramid. In the difference pyramid, compare each pixel point with its adjacent 26 pixel points to find the local extreme points as candidate feature points; Accurate feature point localization: For each candidate feature point, use the pixel points around it to fit a three-dimensional quadratic function. According to the result of the quadratic function fitting, calculate the contrast of the feature point and set a threshold to remove the points with a contrast less than the threshold. Determine whether the point is an edge response point by calculating the Hessian matrix at the feature point, and remove the feature points determined to be edge response points; Feature descriptor generation: Taking the accurately located feature points as the center, divide its neighborhood into a certain number of sub-regions. Calculate the gradient direction histogram within each sub-region, and statistically analyze the distribution of gradient magnitudes in different directions within the sub-region. Combine the gradient histograms of all sub-regions to obtain a feature descriptor.

5. The method for detecting optical crystal defects according to claim 1, wherein The step in S40 of calculating the mutually matching feature point pairs based on the extracted feature points and feature descriptors and establishing the spatial correspondence relationship between image pairs includes: Calculating the distance of feature point descriptors: For each feature point in the SEM crystal image to be measured, use the Euclidean distance to calculate its distance from each feature point descriptor in the TEM crystal image to be measured; Determining the nearest neighbor and the second nearest neighbor: For each feature point in the SEM crystal image to be measured, after calculating the distance from each feature point descriptor in the TEM crystal image to be measured, find the feature point with the shortest distance as its nearest neighbor, and at the same time find the feature point with the second shortest distance as its second nearest neighbor; Determining the matching point pairs: Determine the matching point pairs by comparing the ratio of the nearest neighbor distance to the second nearest neighbor distance.

6. The method for detecting optical crystal defects according to claim 1, wherein, The step in S50 of identifying the defect region by setting a reasonable difference threshold and performing defect localization and classification includes: Training sample collection and annotation: Collect optical crystal SEM and TEM image pairs covering different types, sizes, and defects, accurately annotate the defect regions in the images, and divide the image pairs and annotation information into a training set and a test set; Feature difference calculation: For each pair of optical crystal SEM and TEM images in the training set, use the method in step S40 to calculate the feature points, feature descriptors, and matching feature point pairs. For each pair of matching feature points, calculate the differences in their positions, gray values, and surrounding neighborhood structures; Threshold determination: Conduct statistical analysis on the difference data of all matching feature points in the training set, draw the distribution histogram of the difference values, and determine the difference threshold for distinguishing the defect region and the normal region according to the defect annotation information; Detection of optical crystal image pairs: Calculate the difference value based on the matching feature point pairs obtained in step S40, and compare the calculated difference value with the set threshold. When the difference value of a certain matching feature point pair is greater than the threshold, preliminarily determine that the neighborhood where the feature point is located is a potential defect region; Region merging and screening: Perform merging processing on the preliminarily determined potential defect regions, use morphological operations to optimize the potential defect regions, and finally conduct further analysis on the merged and screened regions to determine the final defect regions.

7. An optical crystal defect detection system, characterized in that, The optical crystal defect detection system described above includes: SEM image acquisition module for the optical crystal to be measured: It is used to preprocess the optical crystal to be measured and make a unique mark. After the mark, the optical crystal sample to be measured is fixed on the SEM sample stage to collect the surface image of the optical crystal, obtain the SEM crystal image to be measured, and associate and store it with the corresponding mark information; TEM image acquisition module for the optical crystal to be measured: It is used to slice the optical crystal sample to be measured after the acquisition of the SEM crystal image to be measured. The slice thickness is controlled within 50 - 100 nm. The sliced sample is fixed on the TEM sample stage to collect the internal image of the optical crystal, obtain the TEM crystal image to be measured, and associate and store it with the corresponding mark information; Preprocessing module for the image pair of the optical crystal to be measured: It is used to pair the SEM crystal image to be measured and the TEM crystal image to be measured one by one according to the mark information to obtain the image pair of the optical crystal to be measured, and perform image preprocessing on each image pair; Feature extraction and matching module for the image pair of the optical crystal to be measured: It is used to extract feature points and feature descriptors from the SEM crystal image to be measured and the TEM crystal image in the image pair by using the scale-invariant feature transform algorithm, and calculate the mutually matching feature point pairs by using the nearest neighbor matching algorithm based on the extracted feature points and feature descriptors, and establish the spatial correspondence relationship between the image pairs; Defect detection module for the optical crystal: It is used to identify the defect area, locate and classify the defects by setting a reasonable difference threshold.

8. An optical crystal defect detection device, characterized in that, The optical crystal defect detection device described above includes: A memory, a processor, and an optical crystal defect detection program stored on the memory and executable on the processor. When the optical crystal defect detection program is executed by the processor, it implements the optical crystal defect detection method according to any one of claims 1 to 6.

9. A computer program product, characterized in that, The computer program product includes an optical crystal defect detection program. When the optical crystal defect detection program is executed by the processor, it implements the optical crystal defect detection method according to any one of claims 1 to 6.

Images