Crystal surface defect detection method and system based on image processing

Abstract

The present invention relates to the field of crystal image detection technology, and more specifically to a method and system for detecting crystal surface defects based on image processing. The present invention first integrates the defect comprehensive morphological measurements of all crystal block images in a crystal training image to obtain the overall defect spot level of the crystal training image; based on the overall defect spot level of each crystal training image, the crystal training image is enhanced to obtain an enhanced crystal training image; and a trained neural network model is used to perform defect detection on the crystal image to be tested. By fully considering the degree of crystal defects, the present invention reasonably enhances the crystal training image and effectively trains the neural network model, thereby improving the accuracy of the trained neural network model in identifying crystal surface defects.

Description

Technical Field

[0001] The present invention relates to the technical field of crystal image detection, and in particular to a crystal surface defect detection method and system based on image processing. Background Art

[0002] Crystal inspection is a core component of semiconductor manufacturing, ensuring chip performance, yield, and cost control. During the crystal production process, surface defects can appear due to environmental factors and other factors. These defects can affect crystal performance and even cause failure of the resulting semiconductor device. Therefore, accurately detecting surface defects is crucial. For semiconductor crystal surface defect detection, the accuracy of convolutional neural networks relies heavily on crystal training images.

[0003] Due to the limited number of crystal training images, these images need to be enhanced. Existing techniques directly enhance these images using geometric transformation enhancement to expand the training sample. However, during this enhancement process, traditional geometric transformation enhancement methods ignore the unique morphological characteristics and spatial continuity of speckle defects on the crystal surface. This leads to irrational enhancement, making it difficult to effectively train the neural network, and consequently, making it difficult for the trained neural network to accurately identify crystal surface defects. Summary of the Invention

[0004] In order to solve the technical problem of unreasonable image enhancement of crystal training images, the purpose of the present invention is to provide a crystal surface defect detection method and system based on image processing. The technical solutions adopted are as follows:

[0005] A method for detecting crystal surface defects based on image processing, the method comprising:

[0006] Acquire the crystal image to be tested and each crystal training image;

[0007] Acquire all crystal block images corresponding to the crystal training image; identify suspected crystal defect regions in the crystal block images from the crystal block images; obtain a comprehensive defect morphology metric of the crystal block images based on the aggregation and morphology of all the suspected crystal defect regions in the crystal block images; and obtain an overall defect spot degree of the crystal training image by integrating the comprehensive defect morphology metrics of all the crystal block images in the crystal training image;

[0008] According to the overall defect spot degree of each crystal training image, each crystal training image is enhanced to obtain an enhanced crystal training image corresponding to each crystal training image; based on all the enhanced crystal training images, a neural network model is trained to obtain a trained neural network model; and using the trained neural network model, defect detection is performed on the crystal image to be tested.

[0009] Furthermore, the method for obtaining the suspected crystal defect region includes:

[0010] In the crystal block image, the area enclosed by each closed edge is used as each area to be analyzed;

[0011] Among all the regions to be analyzed in the crystal block image, each region to be analyzed whose area is greater than a preset first area threshold and not greater than a preset second area threshold is marked as each suspected crystal defect region in the crystal block image.

[0012] Furthermore, the method for obtaining the comprehensive defect morphology measurement includes:

[0013] In the crystal block image, obtaining a defect arrangement index in the crystal block image according to the aggregation of all the suspected crystal defect regions;

[0014] In the crystal block image, obtaining a defect morphology index in the crystal block image according to the grayscale characteristics and morphological characteristics of all the suspected crystal defect regions;

[0015] The defect arrangement index and the defect morphology index in the crystal block image are integrated to obtain the defect comprehensive morphology metric in the crystal block image.

[0016] Furthermore, the method for obtaining the defect arrangement index includes:

[0017] In the crystal block image, the Euclidean distance between the corresponding geometric center points of each two suspected crystal defect regions is used as a distance measurement value of each two suspected crystal defect regions;

[0018] Based on an iterative self-organizing clustering method, clustering all the suspected crystal defect regions in the crystal block image according to the distance metric value between each two suspected crystal defect regions, to obtain each suspected defect cluster in the crystal block image;

[0019] Taking the total number of all the suspected crystal defect regions of the suspected defect cluster as the first defect parameter of the suspected defect cluster;

[0020] In the suspected defect clusters, calculating the Euclidean distance between the corresponding geometric center points of every two suspected defect clusters, and taking the average of all the Euclidean distances as the second defect parameter of the suspected defect cluster;

[0021] Calculating the product of the first defect parameter of the suspected defect cluster and the second defect parameter to obtain a third defect parameter of the suspected defect cluster;

[0022] In the crystal block image, the mean value of the third defect parameter of all the suspected defect clusters is calculated to obtain the defect arrangement index in the crystal block image.

[0023] Furthermore, the method for obtaining the defect morphology index includes:

[0024] In the crystal block image, the roundness value corresponding to the suspected crystal defect region is used as the first defect morphology parameter of the suspected crystal defect region;

[0025] Calculating an absolute value of a difference between an average of grayscale values ​​of all pixels in the suspected crystal defect region and a preset reference grayscale value as a second defect morphology parameter of the suspected crystal defect region;

[0026] The average value of the product of the first defect morphology parameter and the second defect morphology parameter of all the suspected crystal defect areas is calculated to obtain a defect morphology index in the crystal block image.

[0027] Furthermore, the method of integrating the defect arrangement index and the defect morphology index in the crystal block image to obtain the comprehensive morphology measurement of the defects in the crystal block image includes:

[0028] The defect arrangement index and the defect morphology index in the crystal block image are forwardly fused to obtain a comprehensive defect morphology metric in the crystal block image.

[0029] Furthermore, the method for obtaining the overall defect spot degree includes:

[0030] According to a preset direction, sequentially counting the defect comprehensive morphology measurements corresponding to all the crystal block images in the crystal training image to obtain a defect comprehensive morphology measurement sequence of the crystal training image;

[0031] The total number of maximum values ​​in the defect comprehensive morphology measurement sequence is used as the first overall defect parameter of the crystal training image;

[0032] In the crystal training image, taking the average of the defect comprehensive morphology measurements of all the crystal block images as the second overall defect parameter of the crystal training image;

[0033] The product of the first overall defect parameter and the second overall defect parameter of the crystal training image is calculated and normalized to obtain the overall defect spot degree of the crystal training image.

[0034] Furthermore, the method for obtaining the trained neural network model includes:

[0035] All the enhanced crystal training images are used to train a neural network model to obtain a trained neural network model.

[0036] Furthermore, the method for performing defect detection includes:

[0037] The crystal image to be tested is input into the trained neural network model to perform defect detection and output the defect detection result.

[0038] The present invention proposes a crystal surface defect detection system based on image processing, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, the steps of the crystal surface defect detection method based on image processing are implemented.

[0039] The present invention has the following beneficial effects:

[0040] To perform targeted enhancement on the training images, the crystal training images are first divided to obtain all the crystal block images corresponding to the crystal training images. From the crystal block images, suspected crystal defect regions are identified. These suspected crystal defect regions initially reflect the suspected crystal defect regions. The defect comprehensive morphology metrics of the crystal block images are used to measure the likelihood of crystal defects in the crystal block images. By combining the defect comprehensive morphology metrics of all crystal block images, the overall defect spot level of the crystal training images is obtained. The overall defect spot level more comprehensively reflects the degree of defects in the crystal block images. This allows for targeted image enhancement of the crystal training images, resulting in enhanced crystal training images with better defect representation. The neural network model is then effectively trained to improve the accuracy of the trained neural network model in identifying crystal surface defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A flow chart of a crystal surface defect detection method based on image processing provided by one embodiment of the present invention;

[0043] Figure 2 A flow chart of a method for obtaining comprehensive defect morphology measurement provided by one embodiment of the present invention;

[0044] Figure 3 This is a structural diagram of a crystal surface defect detection system based on image processing provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0045] To further illustrate the technical means and effectiveness of the present invention in achieving its intended objectives, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of a crystal surface defect detection method and system based on image processing proposed by the present invention. In the following description, different references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0046] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0047] The following describes in detail a method and system for detecting crystal surface defects based on image processing provided by the present invention with reference to the accompanying drawings.

[0048] The embodiment of the present invention provides a method and system for detecting crystal surface defects based on image processing. Figure 1 , which shows a flow chart of a crystal surface defect detection method based on image processing provided by one embodiment of the present invention, the method comprising the following steps:

[0049] Step S1: Acquire a crystal image to be tested and various crystal training images.

[0050] From the training system, the corresponding crystal surface images of each crystal are obtained. In the present invention, the corresponding crystal surface images of the crystals with defects to be detected are used as the crystal images to be tested; the crystal surface images corresponding to the crystals used to train the neural network are used as crystal training images. The crystal training images are the key data source for subsequent training of the neural network model. Through sufficient training, the model has the ability to accurately identify crystal surface defects.

[0051] It should be noted that the process of obtaining a corresponding crystal surface image of any crystal is well known to those skilled in the art and will be briefly described here. High-resolution cameras are deployed at multiple locations on the production line to capture images of the crystal and obtain partial images of the crystal. All partial images are then stitched together using image stitching technology to obtain an original crystal surface image. Because the original crystal surface images contain noise, which can affect subsequent crystal defect detection results, noise reduction is performed on the original crystal surface images to obtain a de-noised image. This eliminates the effects of noise and some external interference, enhancing the accuracy of subsequent analysis. The de-noised image is then grayscaled to obtain a crystal surface image. In this embodiment of the present invention, Gaussian filtering is used for image noise reduction and the RANdom Sampling Consensus (RANSAC) algorithm is used for image stitching. The implementation can customize these settings based on actual circumstances. It should be noted that the RANSAC algorithm, Gaussian filtering, and grayscale processing are well known to those skilled in the art and will not be described in detail here. By stitching all partial images of the crystal, the present invention obtains a complete crystal surface image. In the present invention, the resolution and size of all crystal training images are the same to ensure the reliability of defect training.

[0052] It should be noted that, for ease of calculation, all indicator data involved in the calculation in the embodiments of the present invention are pre-processed to eliminate dimension effects. Specific means of eliminating dimension effects are well known to those skilled in the art and are not limited here.

[0053] The present invention is constrained by various factors, including cost, and samples approximately 5,000 crystal training images, all of which contain speckle defects of varying sizes and shapes. To further expand the training sample and meet the large data requirements of convolutional neural networks, traditional geometric transformation enhancement methods are often considered. Traditional geometric transformation enhancement methods, such as translation, rotation, scaling, and flipping, can increase data diversity and expand the size of the training set in general data enhancement scenarios. These transformations can generate a large number of new samples from limited crystal training images, alleviating the data shortage problem to a certain extent. However, during the enhancement process of crystal training images, traditional geometric transformation enhancement methods often overlook the unique morphological characteristics and spatial continuity of speckle defects on the crystal surface. For example, speckle defects on the crystal surface are related to uneven temperature distribution and exhibit regional clustering, with large speckles concentrated in high-temperature areas and small speckles concentrated in low-temperature areas, with a continuous transition trend in space. Traditional geometric transformation enhancement methods can disrupt this spatial continuity, making it difficult to properly enhance crystal training images, resulting in overfitting of the trained neural network model and an inability to accurately identify surface defects. The present invention fully considers the degree of crystal defects, reasonably enhances the crystal training image, effectively trains the neural network model, and improves the accuracy of the trained neural network model in identifying crystal surface defects.

[0054] Step S2: Obtain all crystal block images corresponding to the crystal training image; identify suspected crystal defect areas in the crystal block images from the crystal block images; obtain the comprehensive defect morphology measurement of the crystal block images based on the aggregation and morphology of all suspected crystal defect areas in the crystal block images; in the crystal training image, obtain the overall defect spot degree of the crystal training image by integrating the comprehensive defect morphology measurement of all crystal block images.

[0055] In order to perform targeted enhancement on the training images, the crystal training images are first divided to obtain all the crystal block images corresponding to the crystal training images; from the crystal block images, the suspected crystal defect areas in the crystal block images are identified; the suspected crystal defect areas preliminarily reflect the areas of suspected crystal defects, and the defect comprehensive morphology measurement of the crystal block images is used. The defect comprehensive morphology measurement reflects the possibility of crystal defects in the crystal block images, and then the defect comprehensive morphology measurement of all crystal block images is combined to obtain the overall defect spot degree of the crystal training images. The overall defect spot degree more comprehensively reflects the degree of defects in the crystal block images.

[0056] In order to divide the crystal training image, preferably, in one embodiment of the present invention, the method for obtaining all crystal block images corresponding to the crystal training image includes:

[0057] According to the preset specifications, the crystal training image is divided equally to obtain all the crystal block images corresponding to the crystal training image. In one embodiment of the present invention, the preset specifications are 10 10. Implementers can set it according to the implementation scenario.

[0058] Taking into account that in the process of crystal surface production, different processes have different and strict requirements for the external environment. Among them, the distribution of the temperature field plays a key role in the quality of the crystal. When the temperature field is unevenly distributed, it will cause changes in the stress inside the crystal. This stress change will appear on the surface of the crystal as bubbles or small cracks. From a visual point of view, it usually appears as a number of concentrated or discrete spots. Further in-depth analysis shows that local spots caused by temperature unevenness often show obvious regional characteristics. In some areas, because the temperature is too high, more large spots will appear; on the contrary, in areas with lower temperatures, small spots with regional concentration will appear. Moreover, in the visual space, the size changes of these spots are not sudden, but show a continuous transition trend, that is, gradually transitioning from large spots in high-temperature areas to small spots in low-temperature areas. In order to screen out suspected defect areas, preferably, in one embodiment of the present invention, the method for obtaining suspected crystal defect areas includes:

[0059] In the crystal block image, the area enclosed by each closed edge is used as each area to be analyzed;

[0060] In all the areas to be analyzed in the crystal block image, each area to be analyzed whose area is greater than the preset first area threshold and not greater than the preset second area threshold is marked as each suspected crystal defect area in the crystal block image. In one embodiment of the present invention, the preset first area threshold is 3, and the preset second area threshold is 13. The implementer can set the preset first area threshold and the preset second area threshold by himself according to the implementation scenario, such as the type of crystal, the manufacturing process, and the parameters of the image acquisition device. It should be noted that the method for obtaining closed edges is an existing technology well known to those skilled in the art and is only briefly described here: using the Canny edge detection algorithm, each edge in the crystal block image is extracted, and then the edge tracking algorithm is used to determine whether the edge is closed, thereby obtaining all closed edges, wherein the Canny edge detection algorithm and the edge tracking algorithm are both existing technologies well known to those skilled in the art and are not described in detail here.

[0061] Regarding the above steps, considering that crystal surface spot defects appear as closed contours in imaging, the areas enclosed by each closed edge in the crystal block image are used as the regions to be analyzed, which preliminarily reflect the defect area. Considering the physical characteristics of defects, their sizes range: too small may be noise, too large may be other types of defects or non-defective structures. Among all the regions to be analyzed in the crystal block image, the regions to be analyzed with an area greater than a preset first area threshold and no greater than a preset second area threshold are marked as suspected crystal defect regions in the crystal block image. Suspected crystal defect regions more accurately reflect the defect area.

[0062] See also Figure 2 , which shows a flow chart of a method for obtaining a comprehensive defect morphology metric in one embodiment of the present invention. Preferably, in one embodiment of the present invention, the method for obtaining a comprehensive defect morphology metric includes:

[0063] Step S101: in a crystal block image, obtaining a defect arrangement index in the crystal block image according to the aggregation of all suspected crystal defect regions.

[0064] In order to quantify the degree of aggregation of spot defects, the defect arrangement index in the crystal block image is used to reflect the regional concentration caused by the temperature gradient.

[0065] Preferably, in one embodiment of the present invention, the method for obtaining the defect arrangement index includes:

[0066] In the crystal block image, the Euclidean distance between the geometric center points corresponding to each two suspected crystal defect regions is used as the distance measurement value of each two suspected crystal defect regions; based on the iterative self-organizing clustering method, all suspected crystal defect regions in the crystal block image are clustered according to the distance measurement value of each two suspected crystal defect regions to obtain each suspected defect cluster in the crystal block image; the total number of all suspected crystal defect regions in the suspected defect cluster is used as the first defect parameter of the suspected defect cluster; in the suspected defect cluster, the Euclidean distance between the geometric center points corresponding to each two suspected defect clusters is calculated, and the mean of all Euclidean distances is used as the second defect parameter of the suspected defect cluster; the product of the first defect parameter and the second defect parameter of the suspected defect cluster is calculated to obtain the third defect parameter of the suspected defect cluster; in the crystal block image, the mean of the third defect parameters of all suspected defect clusters is calculated to obtain the defect arrangement index in the crystal block image. It should be noted that the Euclidean distance, the geometric center point and the iterative self-organizing clustering method are existing technologies well known to those skilled in the art and will not be described in detail here.

[0067] In the above steps, for each pair of suspected crystal defect regions in the crystal block image, the Euclidean distance between their corresponding geometric center points is calculated and used as the distance metric between the two suspected crystal defect regions. The Euclidean distance accurately measures the spatial proximity of two regions, providing basic data for subsequent cluster analysis. All suspected crystal defect regions in the crystal block image are clustered based on the iterative self-organizing clustering method. The iterative self-organizing clustering method is an adaptive clustering algorithm that automatically determines the number and centers of clusters based on the distribution characteristics of the data, classifying closely spaced suspected crystal defect regions into the same cluster, thereby obtaining individual suspected defect clusters in the crystal block image. These suspected defect clusters reflect the spatial clustering of spot defects. The total number of all suspected crystal defect regions in each suspected defect cluster is used as the first defect parameter of the suspected defect cluster. The first defect parameter reflects the number of defect regions in the cluster; a larger number indicates a higher probability of defect clustering in that region. The second defect parameter reflects the compactness of the defect regions within the cluster; a smaller mean indicates a more concentrated defect region. The product of the first and second defect parameters for each suspected defect cluster is calculated to obtain the third defect parameter for that suspected defect cluster. This third defect parameter comprehensively considers the number and compactness of defect regions, providing a more comprehensive reflection of the clustering characteristics of defects within the cluster. The mean of the third defect parameters for all suspected defect clusters is calculated to obtain the defect arrangement index in the crystal block image. This index quantifies the overall degree of clustering of spot defects in the crystal block image. Larger values ​​indicate greater defect clustering, and are closely related to the regional concentration caused by temperature gradients.

[0068] Step S102: in the crystal block image, obtaining defect morphology indicators in the crystal block image according to the grayscale characteristics and morphological characteristics of all suspected crystal defect regions.

[0069] In addition to considering the arrangement of defects, the morphology of defects is also an important aspect of crystal quality. On the crystal surface, defects of varying degrees exhibit distinct grayscale and morphological characteristics. To quantify these characteristics, the following method is used to obtain defect morphology metrics from crystal block images.

[0070] Preferably, in one embodiment of the present invention, the method for obtaining the defect morphology index includes:

[0071] In the crystal block image, the roundness value corresponding to the suspected crystal defect area is used as the first defect morphology parameter of the suspected crystal defect area;

[0072] Calculating the absolute value of the difference between the mean grayscale value of all pixels in the suspected crystal defect area and a preset reference grayscale value as a second defect morphology parameter of the suspected crystal defect area;

[0073] The average value of the product of the first defect morphology parameter and the second defect morphology parameter of all suspected crystal defect areas is calculated to obtain the defect morphology index in the crystal block image.

[0074] Following the above steps, a roundness value is calculated for each suspected crystal defect region in the crystal block image and used as the first defect morphology parameter for that suspected crystal defect region. The roundness value is an indicator of how closely an object's shape approximates an ideal circle. In crystal defect analysis, it reflects the morphological regularity of the defect. For example, the roundness value of an ideal circular object is close to 1. The more irregular the shape of the defect region on the crystal surface, the smaller the roundness value. By calculating the roundness value of each suspected crystal defect region, the defect morphology can be quantitatively described from the perspective of shape regularity. For each suspected crystal defect region, the mean grayscale value of all pixels in the region is first calculated. This mean value reflects the overall grayscale level of the defect region. The absolute value of the difference between this grayscale mean and a preset reference grayscale value is then calculated and used as the second defect morphology parameter for that suspected crystal defect region. The preset reference grayscale value is typically a baseline value set based on the grayscale characteristics of normal crystal regions. By calculating the absolute value of this difference, the degree of grayscale difference between the defect region and the normal region can be measured. The larger the absolute value of the difference, the more obvious the grayscale contrast between the defect area and the normal area, and the easier the defect may be visually perceived. After obtaining the first defect morphology parameter and the second defect morphology parameter of each suspected crystal defect area, the mean of the product of the first defect morphology parameter and the second defect morphology parameter of all suspected crystal defect areas is calculated. The mean is the defect morphology index in the crystal block image. The roundness value and the absolute value of the grayscale difference are multiplied because these two parameters reflect the morphological characteristics of the defect from the aspects of shape and grayscale respectively. The multiplication can comprehensively consider the influence of these two factors on the defect morphology. Calculating the mean of the product of all suspected crystal defect areas is to evaluate the defect morphology in the crystal block image as a whole. The larger the defect morphology index value obtained, the more obvious the morphological characteristics of the crystal surface defect, and the greater the impact on the crystal quality may be. It should be noted that the roundness value is a prior art well known to those skilled in the art and will not be elaborated here.

[0075] As an example, the defect morphology indicator formula includes:

[0076] in, For the Defect morphology indicators of crystal block images; For the The total number of all suspected crystal defect areas in the crystal block image; For the In the crystal block image, the The roundness value of the suspected crystal defect area; In the In the crystal block image, the The mean grayscale value of all pixels in the suspected crystal defect area; is the preset reference grayscale value; For the A second defect morphology parameter of a suspected crystal defect region; In one embodiment of the present invention, the preset reference grayscale value is 160, which can be set by the implementer according to the real-time scenario.

[0077] Step S103: integrating the defect arrangement index and the defect morphology index in the crystal block image to obtain a comprehensive morphology measurement of the defects in the crystal block image.

[0078] In order to more comprehensively and accurately evaluate the defects in crystal block images, it is necessary to combine the defect arrangement index and the defect morphology index to obtain the comprehensive defect morphology measurement.

[0079] Preferably, in one embodiment of the present invention, the method for obtaining the comprehensive defect morphology measurement includes:

[0080] The defect arrangement index and the defect morphology index in the crystal block image are forwardly fused to obtain the comprehensive morphology measurement of the defects in the crystal block image. It should be noted that forward fusion is an existing technology well known to those skilled in the art, and forward fusion can adopt simple product, arithmetic mean or other suitable fusion methods. In one embodiment of the present invention, the mean of the defect arrangement index and the defect morphology index in the crystal block image is calculated, and normalization is performed to obtain the comprehensive morphology measurement of the defects in the crystal block image. It should be noted that the normalization method adopted is: normalization is performed using the norm normalization function, and the numerical range is limited to between 0 and 1. Normalization is a technical means well known to those skilled in the art, and the choice of normalization function can be linear normalization or standard normalization, etc. The specific normalization method is not limited here.

[0081] In addition to the above steps, the comprehensive defect morphology metric comprehensively considers the arrangement and morphological characteristics of defects, providing a more comprehensive and accurate reflection of the defect status of the crystal surface, and providing an important basis for crystal quality assessment and defect classification. A larger value indicates more severe defects on the crystal surface and lower quality; conversely, a smaller value indicates less severe defects on the crystal surface and higher quality.

[0082] In order to more comprehensively reflect the degree of defects in the crystal block image, preferably, in one embodiment of the present invention, the method for obtaining the overall defect spot degree includes:

[0083] According to a preset direction, the defect comprehensive morphology measurements corresponding to all crystal block images in the crystal training image are sequentially counted to obtain a defect comprehensive morphology measurement sequence of the crystal training image;

[0084] The total number of maximum values ​​in the defect comprehensive morphology measurement sequence is used as the first overall defect parameter of the crystal training image;

[0085] In the crystal training image, the average value of the defect comprehensive morphology measurement of all crystal block images is used as the second overall defect parameter of the crystal training image;

[0086] The product of the first overall defect parameter and the second overall defect parameter of the crystal training image is calculated and normalized to obtain the overall defect spot degree of the crystal training image. In one embodiment of the present invention, the preset direction is from left to right and from top to bottom.

[0087] Following the above steps, the defect comprehensive morphological metrics corresponding to all crystal block images in the crystal training image are sequentially counted along a preset direction to obtain a sequence of comprehensive defect morphological metrics. This ordered statistical method can reflect the spatial distribution of defects in the crystal image, providing basic data for subsequent analysis of the spatial characteristics of defects. The number of maxima in the defect comprehensive morphological metric sequence is used as the first overall defect parameter. A greater number of maxima indicates a more uneven spatial distribution of defects, with the presence of multiple localized high-defect regions. This allows for quantification of the overall defect severity from a spatial distribution perspective. The mean of the defect comprehensive morphological metrics for all crystal block images is used as the second overall defect parameter. The mean comprehensively considers the defect status of all block images in the crystal training image and reflects the average severity of defects on the crystal surface. A larger mean indicates more severe defects on the crystal surface overall; conversely, a smaller mean indicates relatively mild defects on the crystal surface. The product of the first and second overall defect parameters of the crystal training image is calculated and normalized to obtain the overall defect spot level of the crystal training image. The overall defect spot level more comprehensively reflects the degree of defects in the crystal block image.

[0088] In other embodiments of the present invention, a method for obtaining the overall defect spot level includes:

[0089] In the crystal training image, the average of the defect comprehensive morphology metrics of all crystal block images is used as the overall defect spot degree of the crystal training image.

[0090] Step S3: Based on the overall defect spot level of each crystal training image, each crystal training image is enhanced to obtain an enhanced crystal training image corresponding to each crystal training image; based on all enhanced crystal training images, a neural network model is trained to obtain a trained neural network model; and the trained neural network model is used to perform defect detection on the crystal image to be tested.

[0091] Considering that the overall defect spot degree more comprehensively reflects the degree of defects in the crystal block image, targeted image enhancement is performed on the crystal training image based on the overall defect spot degree of each crystal training image to obtain an enhanced crystal training image with better defect representation effect; the neural network model is effectively trained to improve the accuracy of the trained neural network model in identifying crystal surface defects.

[0092] Preferably, in one embodiment of the present invention, the method for acquiring the enhanced crystal training image includes:

[0093] For any crystal training image, based on the geometric transformation enhancement method, the crystal training image is enhanced to obtain an enhanced image corresponding to the crystal training image;

[0094] Among all enhanced images corresponding to the crystal training image, those with an overall defect speckle severity greater than a preset threshold are marked as the enhanced crystal training images of the crystal training image. In one embodiment of the present invention, the preset threshold is 0.4, which can be set by the implementer based on implementation requirements. It should be noted that the geometric transformation enhancement method is well known to those skilled in the art and will not be described in detail here.

[0095] For the above steps, the scale of training data was expanded through the geometric transformation enhancement method, and the screening of enhanced images with a higher degree of overall defect spots ensured that the images used for training had obvious defect characteristics, avoiding the introduction of some images with poor quality or unclear defects into the training set, thereby improving the quality of training data and laying the foundation for the subsequent training of high-performance neural network models.

[0096] Preferably, in one embodiment of the present invention, the method for obtaining the trained neural network model includes:

[0097] Utilize all enhanced crystal training images, train neural network model, obtain trained neural network model.It should be noted that training neural network is a technical means well known to those skilled in the art, and will not be described in detail here.Neural network in the present invention can adopt CNN neural network.

[0098] For the above steps, after training with a large number of enhanced crystal training images, the neural network model can learn the characteristic patterns of various defects on the crystal surface, with high detection accuracy and generalization ability. It can automatically identify and locate defects on different crystal images, greatly improving the efficiency and reliability of defect detection.

[0099] Preferably, in one embodiment of the present invention, the method for performing defect detection includes:

[0100] The crystal image to be tested is input into the trained neural network model to perform defect detection and output the defect detection results.

[0101] For the above steps, the trained neural network model is used for defect detection to improve the efficiency and accuracy of crystal defect detection.

[0102] In summary, the embodiments of the present invention provide a method and system for detecting crystal surface defects based on image processing. First, in a crystal training image, the defect comprehensive morphological measurements of all crystal block images are integrated to obtain the overall defect spot degree of the crystal training image; based on the overall defect spot degree of each crystal training image, the crystal training image is enhanced to obtain an enhanced crystal training image; and the trained neural network model is used to perform defect detection on the crystal image to be tested. By fully considering the degree of crystal defects, the present invention reasonably enhances the crystal training image and effectively trains the neural network model, thereby improving the accuracy of the trained neural network model in identifying crystal surface defects.

[0103] The present invention also proposes a crystal surface defect detection system based on image processing, see Figure 3 , which shows a structural diagram of a crystal surface defect detection system based on image processing provided by an embodiment of the present invention. The system includes: an image acquisition module 101, a defect analysis module 102 and a defect detection module 103.

[0104] The image acquisition module 101 is used to acquire the crystal image to be tested and various crystal training images.

[0105] The defect analysis module 102 is used to obtain all crystal block images corresponding to the crystal training image; identify suspected crystal defect areas in the crystal block images from the crystal block images; obtain the comprehensive defect morphology measurement of the crystal block images based on the aggregation and morphology of all suspected crystal defect areas in the crystal block images; and obtain the overall defect spot degree of the crystal training image by integrating the comprehensive defect morphology measurements of all crystal block images in the crystal training image.

[0106] The defect detection module 103 is used to perform image enhancement on each crystal training image according to the overall defect spot level of each crystal training image, and obtain an enhanced crystal training image corresponding to each crystal training image; train a neural network model based on all enhanced crystal training images, and obtain a trained neural network model; and use the trained neural network model to perform defect detection on the crystal image to be tested.

[0107] It should be noted that the system provided in the above embodiment is merely an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the crystal surface defect detection system based on image processing and the crystal surface defect detection method based on image processing provided in the above embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0108] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0109] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A crystal surface defect detection method based on image processing, characterized in that: include: Acquire the crystal image to be tested and each crystal training image; Obtain all crystal block images corresponding to the crystal training image; identifying suspected crystal defect regions in the crystal block images; In the crystal block image, according to the aggregation and morphology of all suspected crystal defect areas, a comprehensive defect morphology measurement of the crystal block image is obtained, including: in the crystal block image, the Euclidean distance between the corresponding geometric center points of each two suspected crystal defect areas is used as the distance measurement value of each two suspected crystal defect areas; based on the iterative self-organizing clustering method, all suspected crystal defect areas in the crystal block image are clustered according to the distance measurement value of each two suspected crystal defect areas to obtain various suspected defect clusters in the crystal block image; the total number of all suspected crystal defect areas in the suspected defect cluster is used as the first defect parameter of the suspected defect cluster; in the suspected defect cluster, the number of suspected crystal defect areas in the suspected defect cluster is calculated. The Euclidean distances between the corresponding geometric center points of every two suspected defect clusters are used, and the mean of all Euclidean distances is used as the second defect parameter of the suspected defect cluster; the product of the first defect parameter and the second defect parameter of the suspected defect cluster is calculated to obtain the third defect parameter of the suspected defect cluster; in the crystal block image, the mean of the third defect parameters of all suspected defect clusters is calculated to obtain the defect arrangement index in the crystal block image; in the crystal block image, the defect morphology index in the crystal block image is obtained based on the grayscale characteristics and morphological characteristics of all suspected crystal defect areas; the defect arrangement index and the defect morphology index in the crystal block image are integrated to obtain the comprehensive defect morphology measurement in the crystal block image; In the crystal training image, the defect comprehensive morphology measurement of all crystal block images is integrated to obtain the overall defect spot degree of the crystal training image; according to the overall defect spot degree of each crystal training image, each crystal training image is enhanced to obtain the enhanced crystal training image corresponding to each crystal training image; based on all the enhanced crystal training images, a neural network model is trained to obtain the trained neural network model; and the trained neural network model is used to perform defect detection on the crystal image to be tested.

2. The method for detecting crystal surface defects based on image processing according to claim 1, characterized in that: The method for obtaining the suspected crystal defect region includes: In the crystal block image, the area enclosed by each closed edge is used as each area to be analyzed; Among all the regions to be analyzed in the crystal block image, each region to be analyzed whose area is greater than a preset first area threshold and not greater than a preset second area threshold is marked as each suspected crystal defect region in the crystal block image.

3. The method for detecting crystal surface defects based on image processing according to claim 1, characterized in that: The method for obtaining the defect morphology index includes: In the crystal block image, the roundness value corresponding to the suspected crystal defect region is used as the first defect morphology parameter of the suspected crystal defect region; Calculating an absolute value of a difference between an average of grayscale values ​​of all pixels in the suspected crystal defect region and a preset reference grayscale value as a second defect morphology parameter of the suspected crystal defect region; The average value of the product of the first defect morphology parameter and the second defect morphology parameter of all the suspected crystal defect areas is calculated to obtain a defect morphology index in the crystal block image.

4. The method for detecting crystal surface defects based on image processing according to claim 1, characterized in that: The method of obtaining the comprehensive defect morphology metric in the crystal block image by integrating the defect arrangement index and the defect morphology index in the crystal block image includes: The defect arrangement index and the defect morphology index in the crystal block image are forwardly fused to obtain a comprehensive defect morphology metric in the crystal block image.

5. The method for detecting crystal surface defects based on image processing according to claim 1, characterized in that: The method for obtaining the overall defect spot degree includes: According to a preset direction, sequentially counting the defect comprehensive morphology measurements corresponding to all the crystal block images in the crystal training image to obtain a defect comprehensive morphology measurement sequence of the crystal training image; The total number of maximum values ​​in the defect comprehensive morphology measurement sequence is used as the first overall defect parameter of the crystal training image; In the crystal training image, taking the average of the defect comprehensive morphology measurements of all the crystal block images as the second overall defect parameter of the crystal training image; The product of the first overall defect parameter and the second overall defect parameter of the crystal training image is calculated and normalized to obtain the overall defect spot degree of the crystal training image.

6. The method for detecting crystal surface defects based on image processing according to claim 1, characterized in that: The method for obtaining the trained neural network model includes: All the enhanced crystal training images are used to train a neural network model to obtain a trained neural network model.

7. The method for detecting crystal surface defects based on image processing according to claim 1, characterized in that: The method for performing defect detection comprises: The crystal image to be tested is input into the trained neural network model to perform defect detection and output the defect detection result.

8. A crystal surface defect detection system based on image processing, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the crystal surface defect detection method based on image processing as described in any one of claims 1 to 7 are implemented.

Images