Defect detection method and system based on cooling fin image recognition

Abstract

The invention discloses a defect detection method and system based on cooling fin image recognition, and relates to the field of image processing and machine vision, and the method comprises the steps: carrying out the fusion of a collected 2D image and a collected 3D depth image, and obtaining a fused composite image; performing local region segmentation and feature extraction on the fused composite image to obtain features of segmented regions; based on the features of the segmented regions, generating a binarized image through region adaptive threshold adjustment; performing defect detection and classification based on the binary image; and performing defect labeling according to a defect detection and classification result. According to the method, two-dimensional and three-dimensional comprehensive utilization is realized, so that the recognition capability of complex surface defects is improved; the detection precision and the sensitivity to fine defects are improved; and finally, the defects are classified and marked through a support vector machine classifier, so that the accuracy, comprehensiveness and automatic processing effect of defect identification are ensured, and the efficiency and reliability of cooling fin defect detection are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing and machine vision, and in particular to a defect detection method and system based on heat sink image recognition. Background Art

[0002] With the rapid development of industrial automation and intelligent manufacturing, the precision manufacturing industry has increasingly higher requirements for product quality, especially in the electronics, automobile and other industries. As a key heat dissipation component, the surface quality of the heat sink directly affects the heat dissipation performance and service life of the equipment. Therefore, the demand for the detection of surface defects of heat sinks is becoming increasingly urgent. Traditional heat sink detection mainly relies on manual visual inspection or simple 2D image processing technology, which is difficult to effectively identify complex surface defects and is inefficient. In recent years, with the advancement of computer vision technology, deep learning and three-dimensional image acquisition technology, automated defect detection methods based on image recognition have been widely used. Through the comprehensive analysis of 2D images and 3D depth images, defects such as depressions, protrusions, scratches, etc. on the surface of the heat sink can be more comprehensively detected. In addition, the use of machine learning methods to classify and label defects can greatly improve the accuracy and efficiency of detection.

[0003] However, the existing single detection technology based on 2D images or 3D depth images still has many shortcomings. First, when dealing with complex surface textures, single 2D image processing technology often has difficulty in accurately distinguishing three-dimensional structural defects such as depressions and protrusions. When faced with changing lighting conditions or complex background noise, the detection accuracy will also be significantly affected. Secondly, although 3D depth image technology can capture the surface morphology of the heat sink, it lacks sufficient texture information, resulting in poor performance in identifying small surface texture defects (such as scratches). Therefore, how to effectively fuse 2D image and 3D depth image information and comprehensively utilize the characteristics of two-dimensional texture and three-dimensional morphology has become an important challenge in the current heat sink defect detection technology. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a defect detection method based on heat sink image recognition to solve the problem in the prior art that it is difficult to combine the texture information of a 2D image with the morphological information of a 3D depth image for accurate defect detection.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a defect detection method based on heat sink image recognition, which includes collecting 2D images and 3D depth images of the heat sink; fusing the collected 2D image and 3D depth image to obtain a fused composite image; performing local area segmentation and feature extraction on the fused composite image to obtain features of the segmented areas; generating a binary image based on the features of the segmented areas by adjusting the regional adaptive threshold; performing defect detection and classification based on the binary image; and marking defects based on the results of the defect detection and classification.

[0008] As a preferred solution of the defect detection method based on heat sink image recognition of the present invention, the specific steps of collecting the 2D image and 3D depth image of the heat sink are as follows;

[0009] The 2D image and 3D depth image of the heat sink are simultaneously collected through industrial cameras and structured light 3D sensors.

[0010] As a preferred solution of the defect detection method based on heat sink image recognition of the present invention, the specific steps of fusing the collected 2D image and 3D depth image to obtain a fused composite image are as follows;

[0011] Use scale-invariant feature transformation to extract key points in the 2D image, extract the depth value of the 3D depth image pixels, use Euclidean distance to match the feature points of the 2D image and the 3D depth image, and calculate through affine transformation to obtain the alignment image mapped from the key points of the 3D depth image to the 2D image coordinate system;

[0012] The bilinear interpolation method is used to interpolate the depth value of the aligned image, and then normalized, the 3D depth information is mapped to the pixel points of the 2D image, and the normalized depth value corresponding to the pixel point of the 2D image is obtained;

[0013] Based on the fusion method of joint gradient field and depth mapping, the gradient field of the 2D image and the gradient field of the 3D depth image are calculated, and the modulus of the joint gradient field of the 2D image and the 3D depth image is calculated. The weights of the 2D texture information and the 3D depth information are assigned through the joint gradient field to obtain the fused composite image F(i,j), which is expressed as:

[0014]

[0015] Among them, F(i,j) is the composite image after fusion, |G 2D (i,j)| is the modulus of the 2D image gradient field, |G 3D (i,j)| is the modulus of the 3D depth image gradient field, |G *(i, j)| is the modulus of the joint gradient field, I(i, j) is the grayscale value of the 2D image at (i, j), D(i, j) is the depth value of the 3D depth image at (i, j), i represents the vertical position of the pixel in the image, and j represents the horizontal position of the pixel in the image.

[0016] As a preferred solution of the defect detection method based on heat sink image recognition of the present invention, the specific steps of performing local area segmentation and feature extraction on the fused composite image to obtain the features of the segmented area are as follows;

[0017] The gradient field of the composite image F(i,j) is calculated, and the region is segmented according to the change of the gradient field to obtain the image of the segmented region, and the grayscale features and gradient features are extracted from the segmented region.

[0018] As a preferred solution of the defect detection method based on heat sink image recognition of the present invention, the adaptive threshold of the segmented area is calculated based on the grayscale feature and the gradient feature, and the expression is:

[0019]

[0020] Among them, T k is the adaptive threshold of the kth segmentation region, μ k is the average gray value of the kth segmented area, G k is the average gradient modulus of the kth segmented region, is the grayscale variance of the kth segmented area, φ k is the shape complexity of the kth segmentation area, σ k is the grayscale standard deviation of the kth segmented area;

[0021] According to the adaptive threshold T of the segmented area k , for the segmented region R k After binarization, the expression is:

[0022]

[0023] Among them, B k (i, j) is the binarization result of pixel (i, j) in the kth segmentation area, R k To segment the area;

[0024] Combine the binarization results of the segmented regions to generate a binary image.

[0025] As a preferred solution of the defect detection method based on heat sink image recognition of the present invention, wherein: the defect detection and classification are performed based on the binary image, and the specific steps are as follows;

[0026] Use the connected region labeling method to connect the foreground pixels in the binary image into independent regions, and filter out defective regions through geometric features;

[0027] Extracting geometric features and texture features in the defect area, and combining the geometric features and texture features of the defect area into a feature vector;

[0028] The support vector machine classifier is used to classify the feature vectors into local deformation, linear defect and surface damage categories based on the area, perimeter, geometric complexity and center of gravity of the geometric features, and the average grayscale value, grayscale variance and gradient modulus of the texture features.

[0029] As a preferred solution of the defect detection method based on heat sink image recognition of the present invention, wherein: the defect marking is performed based on the results of defect detection and classification, and the specific steps are as follows;

[0030] Through edge detection, the boundary of the defect area is extracted and marked at the edge pixels;

[0031] Obtain the center of gravity of the defect area and mark the defect category, area and perimeter at the center of gravity;

[0032] The annotated composite image is output as an image file, and the detailed information of the defect area is output as a data file.

[0033] In a second aspect, the present invention provides a defect detection system based on heat sink image recognition, comprising:

[0034] The data acquisition module is used to collect 2D images and 3D depth images of the heat sink; the image fusion module is used to fuse the collected 2D image and 3D depth image to obtain a fused composite image; the image segmentation module is used to perform local area segmentation and feature extraction on the fused composite image to obtain the features of the segmented area; the binarization module is used to generate a binary image based on the features of the segmented area through regional adaptive threshold adjustment; the defect detection module is used to perform defect detection and classification based on the binary image; the defect annotation module is used to perform defect annotation based on the results of defect detection and classification.

[0035] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the defect detection method based on heat sink image recognition as described in the first aspect of the present invention is implemented.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the defect detection method based on heat sink image recognition as described in the first aspect of the present invention is implemented.

[0037] The beneficial effects of the present invention are as follows: by collecting 2D images and 3D depth images of the heat sink and fusing the two, the present invention realizes the comprehensive utilization of two-dimensional texture information and three-dimensional morphological information, thereby improving the recognition ability of complex surface defects; on this basis, an adaptive threshold adjustment method is adopted to dynamically generate a binary image based on the characteristics of each segmented area, which significantly improves the detection accuracy and sensitivity to subtle defects; finally, the defects are classified and labeled by a support vector machine classifier, ensuring the accuracy, comprehensiveness and automated processing effect of defect identification, greatly improving the efficiency and reliability of heat sink defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0039] Figure 1 This is a flow chart of the defect detection method based on heat sink image recognition in Example 1.

[0040] Figure 2 Schematic diagram of a defect detection system based on heat sink image recognition in Example 1. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0042] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0043] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0044] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a defect detection method based on heat sink image recognition, comprising the following steps:

[0045] Collect 2D images and 3D depth images of the heat sink;

[0046] Furthermore, the 2D image and 3D depth image of the heat sink are simultaneously collected by an industrial camera and a structured light 3D sensor.

[0047] It should be noted that the 2D image is acquired by an industrial camera to record the texture and color information of the heat sink surface, and the 3D depth data is acquired by laser scanning or structured light sensor to provide the precise distribution of the heat sink surface height.

[0048] The collected 2D image and 3D depth image are fused to obtain a fused composite image;

[0049] Furthermore, the scale-invariant feature transform is used to extract the key points in the 2D image, the depth values ​​of the 3D depth image pixels are extracted, the feature points of the 2D image and the 3D depth image are matched using the Euclidean distance, and the key points of the 3D depth image are mapped to the aligned image in the 2D image coordinate system through affine transformation calculation;

[0050] It should be noted that the depth value of a 3D depth image is the physical depth information of each pixel directly obtained by the sensor;

[0051] The bilinear interpolation method is used to interpolate the depth value of the aligned image, and then normalized, the 3D depth information is mapped to the pixel points of the 2D image, and the normalized depth value corresponding to the pixel point of the 2D image is obtained;

[0052] It should be noted that the depth value interpolation is performed to ensure that each 2D pixel has a corresponding depth value in the 3D depth image after affine transformation;

[0053] Based on the fusion method of joint gradient field and depth map, the gradient field of 2D image and the gradient field of 3D depth image are calculated;

[0054] Calculate the gradient field of a 2D image, the expression is:

[0055]

[0056]

[0057] in, Represents the rate of change of grayscale in the x direction, represents the rate of change of grayscale in the y direction, I(i,j) is the grayscale value of the 2D image at pixel (i,j), (i+1,j) represents the pixel value of the next row after row i in the image and the same column, (i,j+1) represents the pixel value of the next column after column j in the image and the same row;

[0058] Calculate the gradient field of the 3D depth image, the expression is:

[0059]

[0060]

[0061] in, represents the rate of change of depth in the x direction, represents the rate of change of depth in the y direction, D(i,j) is the depth value of the 3D depth image at pixel (i,j), (i+1,j) represents the pixel value of the next row after row i in the image and the same column, and (i,j+1) represents the pixel value of the next column after column j in the image and the same row;

[0062] And calculate the modulus of the joint gradient field of the 2D image and the 3D depth image, the expression is:

[0063]

[0064] Among them, |G * (i,j)| is the modulus of the joint gradient field;

[0065] By assigning weights to 2D texture information and 3D depth information through the joint gradient field, the fused composite image F(i,j) is obtained, which is expressed as:

[0066]

[0067] Among them, F(i,j) is the composite image after fusion, |G 2D (i,j)| is the modulus of the 2D image gradient field, |G 3D (i,j)| is the modulus of the 3D depth image gradient field, |G * (i, j)| is the modulus of the joint gradient field, I(i, j) is the grayscale value of the 2D image at (i, j), D(i, j) is the depth value of the 3D depth image at (i, j), i represents the vertical position of the pixel in the image, and j represents the horizontal position of the pixel in the image.

[0068] Perform local area segmentation and feature extraction on the fused composite image to obtain the features of the segmented area;

[0069] Furthermore, the gradient field of the composite image F(i,j) is calculated as follows:

[0070]

[0071] Among them, |G j (i,j)| is the gradient modulus of the composite image, is the rate of change of the composite image F(i,j) in the x direction, is the rate of change of the composite image F(i,j) in the y direction;

[0072] Perform region segmentation according to the change of gradient field, obtain the image of the segmented region, and extract grayscale features and gradient features from the segmented region;

[0073] It should be noted that, firstly, grayscale feature extraction includes calculating the average grayscale value, grayscale variance and grayscale histogram of pixels in the region, reflecting the brightness distribution of the region; secondly, gradient feature extraction calculates the gradients in the horizontal and vertical directions, obtains the gradient modulus and gradient direction, and describes the edge strength and texture changes in the region;

[0074] Combining these two types of features, the brightness and structural information of the segmented area can be fully characterized, providing a basis for subsequent defect detection;

[0075] For example, the watershed algorithm is used to segment an image. The watershed algorithm is an image segmentation algorithm based on gradient changes, which can handle edge areas in images well.

[0076] Based on the features of the segmented area, a binary image is generated through regional adaptive threshold adjustment;

[0077] Furthermore, based on the grayscale features and gradient features, the adaptive threshold of the segmented area is calculated, and the expression is:

[0078]

[0079] Among them, T k is the adaptive threshold of the kth segmentation region, μ k is the average gray value of the kth segmented area, G k is the average gradient modulus of the kth segmented region, is the grayscale variance of the kth segmented area, φ k is the shape complexity of the kth segmentation area, σ k is the grayscale standard deviation of the kth segmented area;

[0080] The shape complexity φ of the kth segmented region k The expression is:

[0081]

[0082] Among them, P k is the perimeter of the kth segmentation area, A k is the area of ​​the kth segmentation region;

[0083] According to the adaptive threshold T of the segmented area k , for the segmented region R k After binarization, the expression is:

[0084]

[0085] Among them, B k (i, j) is the binarization result of pixel (i, j) in the kth segmentation area, R k For the segmented area, if the pixel value of the composite image F(i,j) is greater than or equal to the adaptive threshold T of the segmented area k , then the pixel (i, j) is marked as foreground, otherwise it is marked as background;

[0086] Combine the binarization results of the segmented areas to generate a binary image;

[0087] It should be noted that the combined binarization results first initialize the binarized image matrix with the same size as the original image, with all pixels initially set to 0, traverse each segmented area, and fill the binarization result corresponding to the area into the global binarized image matrix. This operation is performed for all segmented areas until the binarization results of all segmented areas are combined into the global binarized image.

[0088] Defect detection and classification based on binary images;

[0089] Furthermore, the connected region labeling method is used to connect the foreground pixels in the binary image into independent regions, and the defective regions are screened out by geometric features;

[0090] It should be noted that first, an empty labeling matrix with the same size as the binary image is created, with an initial value of 0. Each pixel is traversed from the upper left to the lower right. For each foreground pixel, its 8 neighboring pixels are checked to see if they have been labeled. If any neighboring pixels have been labeled, the pixel is assigned to the same connected region. If no neighboring pixels have been labeled, a new connected region number is created and assigned to the pixel. Finally, a labeling matrix is ​​obtained, in which the value of each pixel represents the number of the connected region to which the pixel belongs.

[0091] Extracting geometric features and texture features in the defect area, and combining the geometric features and texture features of the defect area into a feature vector;

[0092] Using the support vector machine classifier, the feature vectors are classified into local deformation, linear defect and surface damage based on the area, perimeter, geometric complexity and center of gravity of the geometric features and the average grayscale value, grayscale variance and gradient modulus of the texture features.

[0093] It should be noted that local deformation includes depressions and protrusions, linear defects include scratches and cracks, and surface damage includes holes, impurities, and wear;

[0094] The geometric features of local deformation are large area, short perimeter, small shape factor and regular shape, and the texture features are that the average gray value is significantly different from the background, the gradient modulus is large, the gray variance is small and the edge changes significantly.

[0095] The geometric features of linear defects are small area, long perimeter, large shape factor, irregular shape and slender shape, and the texture features are slightly low or low average gray value, large gradient modulus, large gray variance and significant edge change;

[0096] The geometric features of surface damage are different sizes of areas, circumference and shape factors depending on the defect type, relatively regular holes, irregular impurities and wear, and texture features are significant differences in the average grayscale value from the background, large gradient modulus, large grayscale variance, relatively uniform grayscale of holes, and significant edge changes.

[0097] Defects are marked based on the results of defect detection and classification;

[0098] Furthermore, through edge detection, the boundary of the defect area is extracted and marked at the edge pixels;

[0099] It should be noted that the edge detection algorithm is used to detect pixels where the grayscale value or depth value changes significantly in the defect area. The pixels obtained by edge detection are connected to form a closed boundary of the defect area. The detected edge pixels are marked, and geometric features are extracted from the boundary, such as perimeter, edge complexity, etc., to further characterize the morphology of the defect.

[0100] Obtain the center of gravity of the defect area and mark the defect category, area and perimeter at the center of gravity;

[0101] It should be noted that the purpose of defect annotation is to visualize the detected defect area on the original image for easy viewing and analysis;

[0102] The annotated composite image is output as an image file, and the detailed information of each defect area is output as a data file.

[0103] The present embodiment also provides a defect detection system based on heat sink image recognition, including: a data acquisition module, used to acquire 2D images and 3D depth images of the heat sink; an image fusion module, used to fuse the acquired 2D image and 3D depth image to obtain a fused composite image; an image segmentation module, used to perform local area segmentation and feature extraction on the fused composite image to obtain the features of the segmented area; a binarization module, used to generate a binary image based on the features of the segmented area through regional adaptive threshold adjustment; a defect detection module, used to perform defect detection and classification based on the binary image; a defect annotation module, used to perform defect annotation based on the results of defect detection and classification.

[0104] This embodiment also provides a computer device, which is suitable for the defect detection method based on heat sink image recognition, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the defect detection method based on heat sink image recognition proposed in the above embodiment.

[0105] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0106] The present embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the defect detection method based on heat sink image recognition proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0107] In summary, the present invention achieves the comprehensive utilization of two-dimensional texture information and three-dimensional morphological information by: collecting 2D images and 3D depth images of the heat sink and fusing the two, thereby improving the recognition ability of complex surface defects; on this basis, an adaptive threshold adjustment method is adopted to dynamically generate a binary image based on the features of each segmented area, which significantly improves the detection accuracy and sensitivity to subtle defects; finally, the defects are classified and labeled by a support vector machine classifier, which ensures the accuracy, comprehensiveness and automated processing effect of defect identification, and greatly improves the efficiency and reliability of heat sink defect detection.

[0108] Example 2, referring to Table 1, is the second example of the present invention. To further verify the technical solution of the present invention, experimental simulation data of a defect detection method based on heat sink image recognition is provided.

[0109] The preparation stage of this experiment includes equipment preparation and sample selection. The experimental equipment mainly includes a high-resolution industrial camera (resolution of 1920×1080 pixels) for collecting 2D surface images of the heat sink, and a structured light 3D sensor (depth accuracy of 0.01mm) for obtaining 3D depth images of the heat sink. Five heat sinks with different geometric defects (including dents, protrusions and scratches) were selected as experimental samples. The surface of each heat sink was artificially processed to form different defect types in order to evaluate the comprehensiveness and accuracy of the detection method. In addition, experimental software for image fusion, segmentation, feature extraction, threshold adjustment, defect classification and annotation was pre-installed in the computer system. After all equipment debugging was completed, the data synchronization and accuracy during image acquisition and processing were ensured.

[0110] First, the experiment used an industrial camera and a structured light 3D sensor to synchronously collect 2D images and 3D depth images of the heat sink, ensuring that the resolution and depth accuracy of the image can capture the texture and morphological changes on the surface of the heat sink. The collected 2D images and 3D depth images provide rich texture information and height information respectively, which lays the foundation for subsequent image fusion.

[0111] Secondly, the scale-invariant feature transform (SIFT) is used to extract key points from the 2D image, the feature points in the 3D depth image are matched using the Euclidean distance, and the two images are aligned through affine transformation. By combining the gradient field with the depth mapping method, the 2D texture information and the 3D depth information are organically integrated to generate a composite image containing rich texture and depth features.

[0112] Next, the fused composite image is segmented into local areas, and the grayscale, morphology, gradient and other features of each segmented area are extracted. Based on these features, an adaptive threshold is calculated. Through this threshold, each segmented area is binarized to generate a complete binary image. This binary image will be used for subsequent defect detection and classification.

[0113] Finally, based on the binary image, the connected region labeling method is used to connect the foreground pixels into independent regions, extract geometric and texture features, and use the support vector machine (SVM) classifier to classify the defects. The boundaries of the defects are marked by edge detection, and the defect category, area and perimeter are marked at the center of gravity. Finally, the labeled image file and detailed defect data report are output.

[0114] The existing traditional method is a detection method based on 2D image processing technology.

[0115] The details are shown in Table 1 below:

[0116] Table 1 Comparative experimental data of heat sink defect detection methods

[0117]

[0118] Through the data analysis of the above table, it can be clearly seen that the heat sink defect detection method based on the fusion of 2D and 3D depth images of the present invention is significantly superior to the traditional 2D detection method in multiple key performance indicators. Compared with the traditional 2D image detection method, the present invention combines the texture information of the 2D image and the morphological information of the 3D depth image to achieve a more accurate and comprehensive detection of complex defects on the surface of the heat sink. For example, in terms of detection accuracy, the accuracy of the traditional 2D image detection method is only 78.3% (depression) and 72.6% (convexity), while the detection accuracy of the method of the present invention reaches 92.5% (depression) and 90.1% (convexity), which significantly improves the accuracy of detection. This is because the 2D image detection method only relies on plane images and it is difficult to capture the three-dimensional deformation of the heat sink surface. The present invention can clearly identify defects such as depressions and convexities by fusing 3D depth information. In addition, the false detection rate of the traditional 2D image detection method is 14.7% and 16.9%, while the false detection rate of the method of the present invention is significantly reduced to 5.3% and 6.1%. This is because in 2D images, factors such as lighting changes and texture complexity may lead to false detections. By introducing 3D information, the method of the present invention can effectively reduce the interference of these external factors, thereby reducing false positives. In terms of missed detection rate, the missed detection rate of the traditional method is 9.8% and 11.5%, while the method of the present invention is only 3.6% and 4.4%, which greatly reduces the missed detection phenomenon, especially in the case of small changes in the detection depth but still a defect. The 3D depth image provides more detection information, avoiding defects that cannot be identified by 2D images alone.

[0119] The defect detection method based on heat sink image recognition of the present invention can not only effectively improve the detection accuracy of heat sink surface defects, but also greatly reduce the false detection rate and missed detection rate, ensuring the reliability of the detection results. In particular, by fusing the texture information of the 2D image with the morphological information of the 3D depth image, complex three-dimensional deformation defects such as depressions and protrusions can be more accurately identified. These defects that are difficult to accurately identify in traditional 2D image detection methods have been significantly improved. This reflects the innovation and technical advantages of the present invention in image fusion, detection accuracy improvement, and defect classification, and can provide a more comprehensive and efficient solution for high-precision industrial detection.

[0120] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A defect detection method based on heat sink image recognition, characterized in that: include, Collect 2D images and 3D depth images of the heat sink; The collected 2D image and 3D depth image are fused to obtain a fused composite image; Perform local area segmentation and feature extraction on the fused composite image to obtain the features of the segmented area; Based on the features of the segmented area, a binary image is generated through regional adaptive threshold adjustment; Defect detection and classification based on binary images; Defects are marked based on the results of defect detection and classification.

2. The defect detection method based on heat sink image recognition according to claim 1, characterized in that: The specific steps of collecting the 2D image and 3D depth image of the heat sink are as follows: The 2D image and 3D depth image of the heat sink are simultaneously collected through an industrial camera and a structured light 3D sensor.

3. The defect detection method based on heat sink image recognition according to claim 2, characterized in that: The collected 2D image and 3D depth image are fused to obtain a fused composite image, and the specific steps are as follows; Use scale-invariant feature transformation to extract key points in the 2D image, extract the depth value of the 3D depth image pixels, use Euclidean distance to match the feature points of the 2D image and the 3D depth image, and calculate through affine transformation to obtain the alignment image mapped from the key points of the 3D depth image to the 2D image coordinate system; The bilinear interpolation method is used to interpolate the depth value of the aligned image, and normalization is performed to map the depth information of the 3D depth image to the pixel points of the 2D image to obtain the normalized depth value corresponding to the pixel points of the 2D image; Based on the fusion method of joint gradient field and depth mapping, the gradient field of the 2D image and the gradient field of the 3D depth image are calculated, and the modulus of the joint gradient field of the 2D image and the 3D depth image is calculated. The weights of the 2D texture information and the 3D depth information are assigned through the joint gradient field to obtain the fused composite image F(i,j), which is expressed as: Among them, F(i,j) is the composite image after fusion, |G 2D (i,j)| is the modulus of the 2D image gradient field, |G 3D (i,j)| is the modulus of the 3D depth image gradient field, |G * (i, j)| is the modulus of the joint gradient field, I(i, j) is the grayscale value of the 2D image at (i, j), D(i, j) is the depth value of the 3D depth image at (i, j), i represents the vertical position of the pixel in the image, and j represents the horizontal position of the pixel in the image.

4. The defect detection method based on heat sink image recognition according to claim 3, characterized in that: The specific steps of performing local area segmentation and feature extraction on the fused composite image to obtain the features of the segmented area are as follows; The gradient field of the composite image F(i,j) is calculated, and the region is segmented according to the change of the gradient field to obtain the image of the segmented region, and the grayscale features and gradient features are extracted from the segmented region.

5. The defect detection method based on heat sink image recognition according to claim 4, characterized in that: The binary image is generated by adjusting the regional adaptive threshold based on the features of the segmented region, and the specific steps are as follows: Based on the grayscale features and gradient features, the adaptive threshold of the segmented area is calculated, and the expression is: Among them, T k is the adaptive threshold of the kth segmentation region, μ k is the average gray value of the kth segmented area, G k is the average gradient modulus of the kth segmented region, is the grayscale variance of the kth segmented area, φ k is the shape complexity of the kth segmented region, σ k is the grayscale standard deviation of the kth segmented area; According to the adaptive threshold T of the segmented area k , for the segmented region R k After binarization, the expression is: Among them, B k (i, j) is the binarization result of pixel (i, j) in the kth segmentation area, R k To segment the area; Combine the binarization results of the segmented regions to generate a binary image.

6. The defect detection method based on heat sink image recognition according to claim 5, characterized in that: The defect detection and classification is performed based on the binary image, and the specific steps are as follows: Use the connected region labeling method to connect the foreground pixels in the binary image into independent regions, and filter out defective regions through geometric features; Extracting geometric features and texture features in the defect area, and combining the geometric features and texture features of the defect area into a feature vector; Using the support vector machine classifier, the feature vectors are divided into local deformation, linear defect and surface damage categories.

7. The defect detection method based on heat sink image recognition according to claim 6, characterized in that: Defect labeling is performed based on the results of defect detection and classification, and the specific steps are as follows: Through edge detection, the boundary of the defect area is extracted and marked at the edge pixels; Obtain the center of gravity of the defect area and mark the defect category, area and perimeter at the center of gravity; The annotated composite image is output as an image file, and the detailed information of the defect area is output as a data file.

8. A defect detection system based on heat sink image recognition, based on the defect detection method based on heat sink image recognition according to any one of claims 1 to 7, characterized in that: It includes data acquisition module, image fusion module, image segmentation module, binarization module, defect detection module and defect marking module; A data acquisition module, used to acquire 2D images and 3D depth images of the heat sink; An image fusion module is used to fuse the collected 2D image and 3D depth image to obtain a fused composite image; An image segmentation module is used to perform local area segmentation and feature extraction on the fused composite image to obtain the features of the segmented area; Binarization module, used to generate a binary image based on the characteristics of the segmented area through regional adaptive threshold adjustment; Defect detection module, used for defect detection and classification based on binary images; The defect annotation module is used to perform defect annotation based on the results of defect detection and classification.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the defect detection method based on heat sink image recognition according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the defect detection method based on heat sink image recognition according to any one of claims 1 to 7 are implemented.

Images