A defect detection method and system based on heat sink image recognition

Abstract

The present invention discloses a defect detection method and system based on heat sink image recognition, which relates to the fields of image processing and machine vision. The method comprises the following steps: fusing a collected 2D image and a 3D depth image to obtain a fused composite image; performing local region segmentation and feature extraction on the fused composite image to obtain features of the segmented regions; generating a binary image based on the features of the segmented regions through regional adaptive threshold adjustment; performing defect detection and classification based on the binary image; and labeling defects based on the results of defect detection and classification. The present invention achieves the comprehensive utilization of two and three dimensions, thereby improving the ability to identify complex surface defects; enhancing the accuracy of detection and sensitivity to subtle defects; and finally, classifying and labeling defects using a support vector machine classifier, ensuring the accuracy, comprehensiveness, and automated processing of defect recognition, thereby greatly improving the efficiency and reliability of heat sink defect detection.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and machine vision, and in particular to a defect detection method and system based on heat sink image recognition. Background Art

[0002] With the rapid development of industrial automation and intelligent manufacturing, the precision manufacturing industry is placing increasing demands on product quality. This is particularly true in industries such as electronics and automotive. Heat sinks, as critical heat dissipation components, have a surface quality that directly impacts the equipment's heat dissipation performance and service life. Consequently, the need to detect surface defects on heat sinks is becoming increasingly urgent. Traditional heat sink inspection relies primarily on manual visual inspection or simple 2D image processing techniques, which struggle to effectively identify complex surface defects and are inefficient. In recent years, advances in computer vision, deep learning, and 3D image acquisition have led to widespread adoption of automated defect detection methods based on image recognition. Through the combined analysis of 2D and 3D depth images, defects such as dents, protrusions, and scratches on the heat sink surface can be more comprehensively detected. Furthermore, using machine learning methods to classify and label defects can significantly improve detection accuracy and efficiency.

[0003] However, the existing single detection technology based on 2D images or 3D depth images still has many shortcomings. First, when dealing with complex surface textures, single 2D image processing technology often finds it difficult to accurately distinguish defects in three-dimensional structures such as depressions and protrusions. When faced with changing lighting conditions or complex background noise, the detection accuracy will also be significantly affected. Secondly, although 3D depth image technology can capture the surface morphology of the heat sink, it lacks sufficient texture information, resulting in poor performance in identifying small surface texture defects (such as scratches). Therefore, how to effectively fuse 2D image and 3D depth image information and comprehensively utilize the characteristics of two-dimensional texture and three-dimensional morphology has become an important challenge in current heat sink defect detection technology. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a defect detection method based on heat sink image recognition to solve the problem in the prior art that it is difficult to accurately detect defects by combining the texture information of a 2D image with the morphological information of a 3D depth image.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a defect detection method based on heat sink image recognition, which includes collecting 2D images and 3D depth images of the heat sink; fusing the collected 2D images and 3D depth images to obtain a fused composite image; performing local area segmentation and feature extraction on the fused composite image to obtain features of the segmented areas; generating a binary image based on the features of the segmented areas through regional adaptive threshold adjustment; performing defect detection and classification based on the binary image; and labeling defects based on the results of the defect detection and classification.

[0008] As a preferred solution of the defect detection method based on heat sink image recognition of the present invention, wherein: the 2D image and 3D depth image of the heat sink are collected, and the specific steps are as follows;

[0009] The 2D image and 3D depth image of the heat sink are simultaneously collected using an industrial camera and a structured light 3D sensor.

[0010] As a preferred solution of the defect detection method based on heat sink image recognition of the present invention, the steps of fusing the collected 2D image and 3D depth image to obtain a fused composite image are as follows;

[0011] Use scale-invariant feature transformation to extract key points from the 2D image, extract the depth values ​​of the 3D depth image pixels, use Euclidean distance to match the feature points of the 2D image and 3D depth image, and calculate through affine transformation to obtain the alignment image mapped from the key points of the 3D depth image to the 2D image coordinate system;

[0012] The bilinear interpolation method is used to interpolate the depth values ​​of the aligned images and perform normalization processing to map the 3D depth information to the pixels of the 2D image to obtain the normalized depth values ​​corresponding to the pixels of the 2D image;

[0013] Based on the fusion method of joint gradient field and depth mapping, the gradient field of the 2D image and the gradient field of the 3D depth image are calculated, and the modulus of the joint gradient field of the 2D image and the 3D depth image is calculated. The weights of the 2D texture information and the 3D depth information are assigned by the joint gradient field to obtain the fused composite image F(i, j), which is expressed as:

[0014]

[0015] Among them, F(i,j) is the composite image after fusion, |G 2D (i, j)| is the modulus of the 2D image gradient field, |G 3D (i, j)| is the modulus of the 3D depth image gradient field, |G *(i, j)| is the modulus of the joint gradient field, I(i, j) is the grayscale value of the 2D image at (i, j), D(i, j) is the depth value of the 3D depth image at (i, j), i represents the vertical position of the pixel in the image, and j represents the horizontal position of the pixel in the image.

[0016] As a preferred solution of the defect detection method based on heat sink image recognition of the present invention, the steps of performing local area segmentation and feature extraction on the fused composite image to obtain the features of the segmented area are as follows;

[0017] Calculate the gradient field of the composite image F(i,j), perform region segmentation based on the change of the gradient field, obtain the image of the segmented region, and extract the grayscale features and gradient features from the segmented region.

[0018] As a preferred solution of the defect detection method based on heat sink image recognition of the present invention, the adaptive threshold of the segmented area is calculated based on the grayscale feature and the gradient feature, and the expression is:

[0019]

[0020] Among them, T k is the adaptive threshold of the kth segmentation area, μ k is the average gray value of the kth segmented area, G k is the average gradient modulus of the kth segmented area, is the grayscale variance of the kth segmentation area, φ k is the shape complexity of the kth segmentation area, σ k is the grayscale standard deviation of the kth segmented area;

[0021] According to the adaptive threshold T of the segmented area k , for the segmented region R k Perform binarization processing, the expression is:

[0022]

[0023] Among them, B k (i, j) is the binarization result of pixel (i, j) in the kth segmentation area, R k To segment the area;

[0024] Combine the binarization results of the segmented regions to generate a binary image.

[0025] As a preferred solution of the defect detection method based on heat sink image recognition of the present invention, wherein: the defect detection and classification based on the binary image is performed, the specific steps are as follows;

[0026] Use the connected component labeling method to connect the foreground pixels in the binary image into independent regions, and filter out defective areas through geometric features;

[0027] Extracting geometric features and texture features in the defect area, and combining the geometric features and texture features of the defect area into a feature vector;

[0028] The support vector machine classifier is used to classify the feature vectors into local deformation, linear defect and surface damage categories based on the area, perimeter, geometric complexity and center of gravity of geometric features and the average grayscale value, grayscale variance and gradient modulus of texture features.

[0029] As a preferred solution of the defect detection method based on heat sink image recognition of the present invention, wherein: the defect marking is performed based on the results of defect detection and classification, and the specific steps are as follows;

[0030] Through edge detection, the boundary of the defect area is extracted and the edge pixels are marked;

[0031] Obtain the center of gravity of the defect area and mark the defect type, area and perimeter at the center of gravity;

[0032] The annotated composite image is output as an image file, and the detailed information of the defect area is output as a data file.

[0033] In a second aspect, the present invention provides a defect detection system based on heat sink image recognition, comprising:

[0034] The data acquisition module is used to collect 2D images and 3D depth images of the heat sink; the image fusion module is used to fuse the collected 2D images and 3D depth images to obtain a fused composite image; the image segmentation module is used to perform local area segmentation and feature extraction on the fused composite image to obtain the features of the segmented areas; the binarization module is used to generate a binary image based on the features of the segmented areas through regional adaptive threshold adjustment; the defect detection module is used to perform defect detection and classification based on the binary image; the defect annotation module is used to perform defect annotation based on the results of defect detection and classification.

[0035] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the defect detection method based on heat sink image recognition as described in the first aspect of the present invention is implemented.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the defect detection method based on heat sink image recognition as described in the first aspect of the present invention is implemented.

[0037] The beneficial effects of the present invention are as follows: by collecting 2D images and 3D depth images of the heat sink and fusing the two, the present invention realizes the comprehensive utilization of two-dimensional texture information and three-dimensional morphological information, thereby improving the recognition ability of complex surface defects; on this basis, an adaptive threshold adjustment method is adopted to dynamically generate binary images based on the characteristics of each segmented area, which significantly improves the detection accuracy and sensitivity to subtle defects; finally, the defects are classified and labeled by the support vector machine classifier, ensuring the accuracy, comprehensiveness and automated processing effect of defect identification, and greatly improving the efficiency and reliability of heat sink defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 This is a flow chart of the defect detection method based on heat sink image recognition in Example 1.

[0040] Figure 2 Schematic diagram of the defect detection system based on heat sink image recognition in Example 1. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0042] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0043] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0044] Example 1, with reference to Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a defect detection method based on heat sink image recognition, comprising the following steps:

[0045] Collect 2D images and 3D depth images of the heat sink;

[0046] Furthermore, the 2D image and 3D depth image of the heat sink are simultaneously captured by an industrial camera and a structured light 3D sensor.

[0047] It should be noted that 2D images are acquired by industrial cameras to record the texture and color information of the heat sink surface, and 3D depth data are acquired by laser scanning or structured light sensors to provide accurate distribution of the heat sink surface height.

[0048] Fusing the collected 2D image and 3D depth image to obtain a fused composite image;

[0049] Furthermore, we use scale-invariant feature transformation to extract key points from the 2D image, extract the depth values ​​of the 3D depth image pixels, use Euclidean distance to match the feature points of the 2D image and 3D depth image, and calculate the alignment image by mapping the key points of the 3D depth image to the 2D image coordinate system through affine transformation.

[0050] It should be noted that the depth value of a 3D depth image is the physical depth information of each pixel obtained directly by the sensor;

[0051] The bilinear interpolation method is used to interpolate the depth values ​​of the aligned images and perform normalization processing to map the 3D depth information to the pixels of the 2D image to obtain the normalized depth values ​​corresponding to the pixels of the 2D image;

[0052] It should be noted that the depth value interpolation is performed to ensure that each 2D pixel has a corresponding depth value in the 3D depth image after affine transformation;

[0053] Based on the fusion method of joint gradient field and depth map, the gradient field of 2D image and the gradient field of 3D depth image are calculated;

[0054] Calculate the gradient field of a 2D image, the expression is:

[0055]

[0056]

[0057] in, Indicates the rate of change of grayscale in the x direction, Represents the rate of change of grayscale in the y direction, I(i,j) is the grayscale value of the 2D image at pixel (i,j), (i+1,j) represents the pixel value in the next row after row i in the image and the same column, and (i,j+1) represents the pixel value in the next column after column j in the image and the same row;

[0058] Calculate the gradient field of the 3D depth image, the expression is:

[0059]

[0060]

[0061] in, represents the rate of change of depth in the x direction, Denotes the rate of change of depth in the y direction, D(i,j) is the depth value of the 3D depth image at pixel (i,j), (i+1,j) represents the pixel value in the next row after row i in the image and the same column, and (i,j+1) represents the pixel value in the next column after column j in the image and the same row;

[0062] And calculate the modulus of the joint gradient field of the 2D image and the 3D depth image, the expression is:

[0063]

[0064] Among them, |G * (i,j)| is the modulus of the joint gradient field;

[0065] By assigning weights to 2D texture information and 3D depth information through the joint gradient field, the fused composite image F(i, j) is obtained, which is expressed as:

[0066]

[0067] Among them, F(i,j) is the composite image after fusion, |G 2D (i, j)| is the modulus of the 2D image gradient field, |G 3D (i, j)| is the modulus of the 3D depth image gradient field, |G * (i, j)| is the modulus of the joint gradient field, I(i, j) is the grayscale value of the 2D image at (i, j), D(i, j) is the depth value of the 3D depth image at (i, j), i represents the vertical position of the pixel in the image, and j represents the horizontal position of the pixel in the image.

[0068] Perform local region segmentation and feature extraction on the fused composite image to obtain the features of the segmented region;

[0069] Furthermore, the gradient field of the composite image F(i,j) is calculated as follows:

[0070]

[0071] Among them, |G j (i,j)| is the gradient modulus of the composite image, is the rate of change of the composite image F(i,j) in the x direction, is the rate of change of the composite image F(i,j) in the y direction;

[0072] Perform region segmentation according to the change of gradient field, obtain the image of the segmented region, and extract grayscale features and gradient features from the segmented region;

[0073] It should be noted that, firstly, grayscale feature extraction includes calculating the average grayscale value, grayscale variance and grayscale histogram of pixels in the region, reflecting the brightness distribution of the region; secondly, gradient feature extraction calculates the horizontal and vertical gradients to obtain the gradient modulus and gradient direction, describing the edge strength and texture changes in the region;

[0074] Combining these two types of features, the brightness and structural information of the segmented area can be fully characterized, providing a basis for subsequent defect detection;

[0075] For example, the watershed algorithm is used to segment an image. The watershed algorithm is an image segmentation algorithm based on gradient changes, which can handle edge areas in images well.

[0076] Based on the characteristics of the segmented area, a binary image is generated through regional adaptive threshold adjustment;

[0077] Furthermore, based on the grayscale features and gradient features, the adaptive threshold of the segmented area is calculated, and the expression is:

[0078]

[0079] Among them, T k is the adaptive threshold of the kth segmentation area, μ k is the average gray value of the kth segmented area, G k is the average gradient modulus of the kth segmented area, is the grayscale variance of the kth segmentation area, φ k is the shape complexity of the kth segmentation area, σ k is the grayscale standard deviation of the kth segmented area;

[0080] The shape complexity φ of the kth segmented region k The expression is:

[0081]

[0082] Among them, P k is the perimeter of the kth segmentation area, A k is the area of ​​the kth segmentation region;

[0083] According to the adaptive threshold T of the segmented area k , for the segmented region R k Perform binarization processing, the expression is:

[0084]

[0085] Among them, B k (i, j) is the binarization result of pixel (i, j) in the kth segmentation area, R k For the segmented area, if the pixel value of the composite image F(i,j) is greater than or equal to the adaptive threshold T of the segmented area k , then the pixel (i, j) is marked as foreground, otherwise it is marked as background;

[0086] Combine the binarization results of the segmented areas to generate a binary image;

[0087] It should be noted that the combined binarization results first initialize the binarized image matrix with the same size as the original image, with all pixels initially set to 0, traverse each segmented area, and fill the binarization results corresponding to the area into the global binarized image matrix. This operation is performed for all segmented areas until the binarization results of all segmented areas are combined into the global binarized image.

[0088] Defect detection and classification based on binary images;

[0089] Furthermore, the connected component labeling method is used to connect the foreground pixels in the binary image into independent regions, and the defective areas are screened out by geometric features;

[0090] It should be noted that first, an empty labeling matrix with the same size as the binary image is created, with an initial value of 0. Each pixel is traversed from the upper left to the lower right. For each foreground pixel, its 8 adjacent pixels are checked to see if they have been labeled. If any adjacent pixels have been labeled, the pixel is assigned to the same connected area. If no adjacent pixels have been labeled, a new connected area number is created and assigned to the pixel. Finally, a labeling matrix is ​​obtained, in which the value of each pixel represents the connected area number to which the pixel belongs.

[0091] Extracting geometric features and texture features in the defect area, and combining the geometric features and texture features of the defect area into a feature vector;

[0092] Using a support vector machine classifier, the feature vectors are classified into local deformation, linear defect, and surface damage categories based on the area, perimeter, geometric complexity, and center of gravity of the geometric features, as well as the average grayscale value, grayscale variance, and gradient modulus of the texture features.

[0093] It should be noted that local deformation includes depressions and protrusions, linear defects include scratches and cracks, and surface damage includes holes, impurities, and wear.

[0094] The geometric features of local deformation are large area, short perimeter, small shape factor and regular shape, while the texture features are average gray value significantly different from background, large gradient modulus, small gray variance and significant edge change.

[0095] The geometric characteristics of linear defects are small area, long perimeter, large shape factor, irregular shape and slender shape. The texture characteristics are slightly low or low average grayscale value, large gradient modulus, large grayscale variance and significant edge changes.

[0096] The geometric characteristics of surface damage are different area sizes, perimeter and shape factors depending on the defect type, relatively regular holes, irregular impurities and wear. The texture characteristics are significant differences in average grayscale value from the background, large gradient modulus, large grayscale variance, relatively uniform grayscale of holes and significant edge changes.

[0097] Defect marking is performed based on the results of defect detection and classification;

[0098] Furthermore, through edge detection, the boundaries of the defect area are extracted and the edge pixels are marked;

[0099] It should be noted that the edge detection algorithm is used to detect pixels with significant changes in grayscale or depth values ​​within the defect area. The pixels obtained by edge detection are connected to form a closed boundary of the defect area. The detected edge pixels are marked, and geometric features such as perimeter and edge complexity are extracted from the boundary to further characterize the morphology of the defect.

[0100] Obtain the center of gravity of the defect area and mark the defect type, area and perimeter at the center of gravity;

[0101] It should be noted that the purpose of defect annotation is to visualize the detected defect areas on the original image for easy viewing and analysis;

[0102] The annotated composite image is output as an image file, and the detailed information of each defect area is output as a data file.

[0103] This embodiment also provides a defect detection system based on heat sink image recognition, including: a data acquisition module, used to acquire 2D images and 3D depth images of the heat sink; an image fusion module, used to fuse the acquired 2D images and 3D depth images to obtain a fused composite image; an image segmentation module, used to perform local area segmentation and feature extraction on the fused composite image to obtain the features of the segmented areas; a binarization module, used to generate a binarized image based on the features of the segmented areas through regional adaptive threshold adjustment; a defect detection module, used to perform defect detection and classification based on the binarized image; and a defect annotation module, used to perform defect annotation based on the results of defect detection and classification.

[0104] This embodiment also provides a computer device, which is suitable for the defect detection method based on heat sink image recognition, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the defect detection method based on heat sink image recognition proposed in the above embodiment.

[0105] The computer device may be a terminal, comprising a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse.

[0106] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the defect detection method based on heat sink image recognition proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0107] In summary, the present invention achieves the comprehensive utilization of two-dimensional texture information and three-dimensional morphological information by collecting 2D images and 3D depth images of the heat sink and fusing the two, thereby improving the recognition ability of complex surface defects; on this basis, an adaptive threshold adjustment method is adopted to dynamically generate a binary image based on the features of each segmented area, which significantly improves the detection accuracy and sensitivity to subtle defects; finally, the defects are classified and labeled by the support vector machine classifier, ensuring the accuracy, comprehensiveness and automated processing effect of defect identification, and greatly improving the efficiency and reliability of heat sink defect detection.

[0108] Example 2, referring to Table 1, is the second example of the present invention. To further verify the technical solution of the present invention, experimental simulation data of a defect detection method based on heat sink image recognition is provided.

[0109] The preparatory phase of this experiment includes equipment preparation and sample selection. The experimental equipment mainly includes a high-resolution industrial camera (resolution of 1920×1080 pixels) for acquiring 2D surface images of the heat sink, and a structured light 3D sensor (depth accuracy of 0.01mm) for acquiring 3D depth images of the heat sink. Five heat sinks with different geometric defects (including dents, protrusions, and scratches) were selected as experimental samples. The surface of each heat sink was manually processed to form different defect types in order to evaluate the comprehensiveness and accuracy of the detection method. In addition, the computer system was pre-installed with experimental software for image fusion, segmentation, feature extraction, threshold adjustment, defect classification and annotation. After all equipment was debugged, the synchronization and accuracy of the data during image acquisition and processing were ensured.

[0110] First, the experiment used an industrial camera and a structured light 3D sensor to simultaneously capture 2D and 3D depth images of the heat sink, ensuring the image resolution and depth accuracy were sufficient to capture the texture and morphological changes on the heat sink surface. The captured 2D and 3D depth images provided rich texture and height information, respectively, laying the foundation for subsequent image fusion.

[0111] Next, the Scale-Invariant Feature Transform (SIFT) is used to extract key points from the 2D image. Euclidean distance is used to match feature points in the 3D depth image, and the two images are aligned using an affine transformation. By combining gradient fields with depth mapping, the 2D texture information and 3D depth information are organically integrated to generate a composite image containing rich texture and depth features.

[0112] Next, the fused composite image is segmented into local regions, extracting features such as grayscale, morphology, and gradient for each segmented region. Based on these features, an adaptive threshold is calculated. Using this threshold, each segmented region is binarized to generate a complete binary image. This binary image is used for subsequent defect detection and classification.

[0113] Finally, based on the binary image, the connected component labeling method is used to connect foreground pixels into independent regions, extract geometric and texture features, and classify defects using a support vector machine (SVM) classifier. Edge detection is used to mark the boundaries of the defects, and the defect category, area, and perimeter are annotated at the center of gravity. Finally, the annotated image file and a detailed defect data report are output.

[0114] The existing traditional method is a detection method based on 2D image processing technology.

[0115] The details are shown in Table 1 below:

[0116] Table 1 Comparative experimental data of heat sink defect detection methods

[0117]

[0118] By analyzing the data in the above table, it can be clearly seen that the heat sink defect detection method based on the fusion of 2D and 3D depth images of the present invention significantly outperforms traditional 2D detection methods in multiple key performance indicators. Compared with traditional 2D image detection methods, the present invention achieves more accurate and comprehensive detection of complex defects on the surface of heat sinks by combining the texture information of 2D images and the morphological information of 3D depth images. For example, in terms of detection accuracy, the accuracy of traditional 2D image detection methods is only 78.3% (depression) and 72.6% (convexity), while the detection accuracy of the method of the present invention reaches 92.5% (depression) and 90.1% (convexity), significantly improving the accuracy of detection. This is because the 2D image detection method relies only on planar images and has difficulty capturing the three-dimensional deformation of the heat sink surface. By fusing 3D depth information, the present invention can clearly identify defects such as depressions and convexities. In addition, the false detection rate of traditional 2D image detection methods is 14.7% and 16.9%, while the false detection rate of the method of the present invention is significantly reduced to 5.3% and 6.1%. This is because in 2D images, factors such as lighting variations and texture complexity can lead to false detections. By introducing 3D information, the method of the present invention can effectively reduce the interference of these external factors, thereby reducing false positives. In terms of missed detection rates, the traditional method has a missed detection rate of 9.8% and 11.5%, while the method of the present invention has a missed detection rate of only 3.6% and 4.4%, significantly reducing missed detections. This is especially true when detecting defects with small depth changes but still presenting defects. 3D depth images provide more detection information, avoiding defects that cannot be identified by 2D images alone.

[0119] The present invention's defect detection method based on heat sink image recognition not only effectively improves the detection accuracy of heat sink surface defects, but also significantly reduces the false detection rate and missed detection rate, ensuring the reliability of the detection results. In particular, by fusing the texture information of the 2D image with the morphological information of the 3D depth image, complex three-dimensional deformation defects such as depressions and protrusions can be more accurately identified. These defects, which are difficult to accurately identify using traditional 2D image detection methods, are significantly improved. This reflects the innovation and technical advantages of the present invention in image fusion, detection accuracy improvement, and defect classification, and can provide a more comprehensive and efficient solution for high-precision industrial inspection.

[0120] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A defect detection method based on heat sink image recognition, characterized by: include, Collect 2D images and 3D depth images of the heat sink; The collected 2D image and 3D depth image are fused to obtain a fused composite image. The specific steps are as follows: Use scale-invariant feature transformation to extract key points from the 2D image, extract the depth values ​​of the 3D depth image pixels, use Euclidean distance to match the feature points of the 2D image and 3D depth image, and calculate through affine transformation to obtain the alignment image mapped from the key points of the 3D depth image to the 2D image coordinate system; The bilinear interpolation method is used to interpolate the depth values ​​of the aligned images and perform normalization processing to map the depth information of the 3D depth image to the pixels of the 2D image to obtain the normalized depth values ​​corresponding to the pixels of the 2D image; Based on the fusion method of joint gradient field and depth mapping, the gradient field of the 2D image and the gradient field of the 3D depth image are calculated, and the modulus of the joint gradient field of the 2D image and the 3D depth image is calculated. The weights of the 2D texture information and the 3D depth information are assigned by the joint gradient field to obtain the fused composite image F(i, j), which is expressed as: Among them, F(i,j) is the composite image after fusion, |G 2D (i, j)| is the modulus of the 2D image gradient field, |G 3D (i, j)| is the modulus of the 3D depth image gradient field, |G * (i, j)| is the modulus of the joint gradient field, I(i, j) is the grayscale value of the 2D image at (i, j), D(i, j) is the depth value of the 3D depth image at (i, j), i represents the vertical position of the pixel in the image, and j represents the horizontal position of the pixel in the image; Perform local region segmentation and feature extraction on the fused composite image to obtain the features of the segmented region; Based on the characteristics of the segmented area, a binary image is generated through regional adaptive threshold adjustment. The specific steps are as follows: Based on the grayscale features and gradient features, the adaptive threshold of the segmented area is calculated. The expression is: Among them, T k is the adaptive threshold of the kth segmentation area, μ k is the average gray value of the kth segmented area, G k is the average gradient modulus of the kth segmented area, is the grayscale variance of the kth segmentation area, φ k is the shape complexity of the kth segmentation area, σ k is the grayscale standard deviation of the kth segmentation area; Defect detection and classification based on binary images; Defects are marked based on the results of defect detection and classification.

2. The heat sink defect detection method based on image recognition according to claim 1, characterized in that: The specific steps of collecting the 2D image and 3D depth image of the heat sink are as follows: The 2D image and 3D depth image of the heat sink are simultaneously collected using an industrial camera and a structured light 3D sensor.

3. The heat sink defect detection method based on image recognition according to claim 2, wherein: The specific steps of performing local region segmentation and feature extraction on the fused composite image to obtain the features of the segmented region are as follows: Calculate the gradient field of the composite image F(i,j), perform region segmentation based on the change of the gradient field, obtain the image of the segmented region, and extract the grayscale features and gradient features from the segmented region.

4. The heat sink defect detection method based on image recognition according to claim 3, wherein: The method of generating a binary image based on the features of the segmented region by adjusting the region adaptive threshold may further include the following steps: According to the adaptive threshold T of the segmented area k , for the segmented region R k Perform binarization processing, the expression is: Among them, B k (i, j) is the binarization result of pixel (i, j) in the kth segmentation area, R k To segment the area; Combine the binarization results of the segmented regions to generate a binary image.

5. The defect detection method based on heat sink image recognition according to claim 4, characterized in that: The defect detection and classification is performed based on the binary image, and the specific steps are as follows: Use the connected component labeling method to connect the foreground pixels in the binary image into independent regions, and filter out defective areas through geometric features; Extracting geometric features and texture features in the defect area, and combining the geometric features and texture features of the defect area into a feature vector; The support vector machine classifier is used to classify the feature vectors into local deformation, linear defect and surface damage categories.

6. The heat sink defect detection method based on image recognition according to claim 5, characterized in that: The defect detection and classification results are used to mark defects. The specific steps are as follows: Through edge detection, the boundary of the defect area is extracted and the edge pixels are marked; Obtain the center of gravity of the defect area and mark the defect type, area and perimeter at the center of gravity; The annotated composite image is output as an image file, and the detailed information of the defect area is output as a data file.

7. A heat sink image recognition-based defect detection system, based on the heat sink image recognition-based defect detection method according to any one of claims 1 to 6, characterized in that: It includes data acquisition module, image fusion module, image segmentation module, binarization module, defect detection module and defect marking module; A data acquisition module, used to acquire 2D images and 3D depth images of the heat sink; An image fusion module is used to fuse the collected 2D image and 3D depth image to obtain a fused composite image; Image segmentation module, used to perform local area segmentation and feature extraction on the fused composite image to obtain the features of the segmented area; The binarization module is used to generate a binary image based on the characteristics of the segmented area through regional adaptive threshold adjustment; Defect detection module, used for defect detection and classification based on binary images; The defect annotation module is used to perform defect annotation based on the results of defect detection and classification.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the defect detection method based on heat sink image recognition according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the defect detection method based on heat sink image recognition according to any one of claims 1 to 6 are implemented.

Images