A Super-Resolution Reconstruction and Feature Extraction Method for Defect Detection

Abstract

The present invention belongs to the technical field of defect detection, and specifically relates to a super-resolution reconstruction and feature extraction method for defect detection. The basic steps of the method of the present invention include: determining a set of low-resolution image block training samples and a set of high-resolution image block training samples; generating a reconstruction model to obtain a predicted output of high-resolution image blocks; performing smoothing processing on the edge pixels of the high-resolution image blocks; and extracting the defect edge region. The super-resolution image for defect detection reconstructed by the method of the present invention has a relatively high contrast, and the edge detail information of the defect region is complete and the contour is clear. The overall quality of the reconstructed defect detection image is relatively high, and there are no obvious phenomena of distortion, blurring and distortion, which is of great significance for the extraction and analysis of key defect information in the process of surface defect detection of additive manufacturing workpieces.

Description

Technical Field

[0001] The present invention belongs to the technical field of defect detection, and particularly relates to a super-resolution reconstruction and feature extraction method for defect detection. Background Art

[0002] Additive manufacturing technology is a highly competitive, low-cost, and high-freedom manufacturing technology that has been widely applied in fields such as aerospace, military, medical equipment, energy, and automotive manufacturing. However, there are still significant limitations in the production process of additive manufacturing, mainly due to two reasons: quality and repeatability, which may be severely affected by certain defects (such as cracks and balling) in the additive manufacturing process. To solve this problem, scholars at home and abroad have conducted extensive research on additive manufacturing defect detection technology to improve the processing quality of parts, and a series of research on multi-sensor detection systems has emerged. In multi-sensor vision detection systems, in addition to developing image fusion algorithms to enhance the richness of detection information, it is also hoped to obtain high-resolution images to improve the accuracy and efficiency of defect detection, because the defect areas are often relatively subtle, and the resolution of defect images is easily limited by hardware devices.

[0003] To improve the resolution of images, super-resolution technology is often introduced. Super-resolution has been widely applied in many fields such as pattern recognition, medical diagnosis, video surveillance, remote sensing, and biometric recognition. Currently proposed image super-resolution algorithms mainly include reconstruction-based super-resolution algorithms, difference-based super-resolution algorithms, and learning-based super-resolution algorithms. The learning-based super-resolution method estimates the missing information of low-resolution images by introducing an external dataset to offline learn the mapping relationship between low-resolution images and high-resolution image samples and using the mapping relationship, and can obtain high-quality reconstructed images. However, it is sensitive to the number and selection of neighboring image patches, which is prone to problems such as insufficient or overfitting of the image fitting. Secondly, the smoothing process of the image is achieved by taking the average value of the pixels in the overlapping area of the image patches. When the variance of the image area is very high, this processing method is difficult to highlight clear edge detail information, seriously affecting the quality of defect detection and defect feature extraction. Summary of the Invention

[0004] To overcome the above-mentioned shortcomings and deficiencies of the prior art, the purpose of the present invention is to provide a super-resolution reconstruction and feature extraction method for defect detection. The reconstructed super-resolution defect detection image has a high contrast, the edge detail information of the defect area is complete, the contour is clear, and the overall quality of the reconstructed defect detection image is high, which is of great significance for the extraction and analysis of key defect information in the surface defect detection process of additive manufacturing workpieces.

[0005] The present invention provides a super-resolution reconstruction and feature extraction method for defect detection, which is based on a direct mapping method, directly establishing a linear mapping relationship between the low-resolution image block and the high-resolution image block training sample set, obtaining a regression function through learning, and then directly transforming the input low-resolution image block into a high-resolution image block through the regression function. Mapping the predicted output high-resolution image block to obtain the high-resolution reconstructed image comprises the following steps:

[0006] (1) Input low-resolution image I L The image is divided into non-overlapping blocks of 3×3 pixels, and the low-resolution image test set X is input externally. l Each image block in Calculate its low-resolution image block training sample set Y l The corresponding K nearest neighbor image blocks in form a set

[0007] (2) Set the K nearest neighbor low-resolution image blocks P l Corresponding to the high-resolution image block training sample set Y h , get the K nearest neighbor high-resolution image blocks and form a set

[0008] (3) Generate reconstruction model: get Predicted high-resolution image patches: The regularized empirical error is:

[0009]

[0010] Expressed as:

[0011] Among them, λ is the regularization parameter, which is used to balance the relationship between the reconstruction error and the regularization term; I is the identity matrix, is a high-resolution image patch, is a low-resolution image patch, is a set of high-resolution image patch training samples, is a set of low-resolution image patch training samples, φ i is the regularized empirical error; is a regularization term used to constrain the super-resolution reconstruction results.

[0012] (4) The high-resolution image block obtained by the prediction output is:

[0013] (5) smoothing of edge pixels of the predicted high-resolution image blocks;

[0014] (6) Use the Canny edge detection method to extract the edge region of the defect from the image obtained in step (5).

[0015] Furthermore, the high-resolution image block edge pixel smoothing process uses the second derivative in the orthogonal diagonal direction of the pixels to obtain the image I H , which is expressed as:

[0016]

[0017] where I(i, j) is the edge pixel selected for smoothing in the image block. In the image coordinate system, I(i + 3, j + 1) represents the pixel determined by the intersection of the fourth column in the positive X-axis direction and the second row in the positive Y-axis direction with the edge pixel I(i, j) as the origin, and the same applies to others. F L is the second derivative in the direction from the upper left corner to the lower right corner of the image, and F H is the second derivative in the direction from the upper right corner to the lower left corner of the image;

[0018] If F L < F H , then:

[0019]

[0020] If F L > F H , then:

[0021]

[0022] Furthermore, use information entropy (E), standard deviation (SD), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) to evaluate the performance of the super-resolution reconstruction and feature extraction of the defect detection image.

[0023] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0024] (1) The defect detection super-resolution image reconstructed by the method of the present invention has a relatively high contrast, the edge detail information of the defect area is complete, the contour is clear, and the overall quality of the reconstructed defect detection image is relatively high, which is of great significance for the extraction and analysis of key defect information in the surface defect detection of additive manufacturing workpieces.

[0025] (2) The method of the present invention is based on the direct mapping method, and directly establishes a linear mapping relationship between the low-resolution image block and the high-resolution image block training sample set, which can effectively overcome the problems of insufficient or overfitting of image fitting.

[0026] (3) The method of the present invention adopts a strategy of non-overlapping processing of adjacent image blocks, and specifically adjusts the edge pixels of the image blocks, which can effectively solve the problem that it is difficult to highlight clear edge detail information when the variance of the image region is very high, seriously affecting the quality of defect detection and defect feature extraction.

[0027] (4) The method of the present invention has good robustness, and all evaluation indexes are significantly better than the bicubic interpolation algorithm and the neighborhood embedding algorithm in the prior art, which is of great significance for the extraction and analysis of key defect information in the surface defect detection of additive manufacturing workpieces. Description of the Drawings

[0028] Figure 1 It is a schematic diagram of a super-resolution reconstruction and feature extraction method for defect detection according to the present invention.

[0029] Figure 2 It is a schematic diagram of the result comparison of different image super-resolution reconstruction methods in the embodiment of the present invention. Among them, left: bicubic interpolation; middle: neighborhood embedding; right: the method of the present invention.

[0030] Figure 3 It is a schematic diagram of the edge detection and defect feature extraction results in the embodiment of the present invention. Among them, left: the super-resolution reconstruction image of the defect area processed by the method of the present invention; middle: the binary image of the defect area; right: the edge detection of the defect area.

[0031] Figure 4 It is a schematic diagram of the result comparison of the information entropy (E) index under different super-resolution reconstruction methods in the embodiment of the present invention. Among them, (a) Image 1; (b) Image 2; (c) Image 3; (d) Image 4.

[0032] Figure 5 It is a schematic diagram of the result comparison of the standard deviation (SD) index under different super-resolution reconstruction methods in the embodiment of the present invention. Among them, (a) Image 1; (b) Image 2; (c) Image 3; (d) Image 4.

[0033] Figure 6 It is a schematic diagram of the result comparison of the peak signal-to-noise ratio (PSNR) index under different super-resolution reconstruction methods in the embodiment of the present invention. Among them, (a) Image 1; (b) Image 2; (c) Image 3; (d) Image 4.

[0034] Figure 7 It is a schematic diagram of the result comparison of the structural similarity (SSIM) index under different super-resolution reconstruction methods in the embodiment of the present invention. Among them, (a) Image 1; (b) Image 2; (c) Image 3; (d) Image 4. Detailed implementation manners

[0035] The present invention will be further described in detail below in conjunction with embodiments and the accompanying drawings, but the implementation manners of the present invention are not limited thereto.

[0036] As Figure 1 shown, a super-resolution reconstruction and feature extraction method for defect detection, which is based on a direct mapping method, directly establishes a linear mapping relationship between the low-resolution image patch and the high-resolution image patch training sample set, obtains a regression function through learning, and then directly maps the input low-resolution image patch to the predicted output high-resolution image patch to obtain a high-resolution reconstructed image; specifically including nine basic steps:

[0037] (1) Divide the input low-resolution image I L into non-overlapping image patches of 3×3 pixel size, and calculate the corresponding K nearest neighbor image patches in the low-resolution image patch training sample set y l for each image patch in the external input low-resolution image test set X l to form a set

[0038] (2) Corresponding the set P l of K nearest neighbor low-resolution image patches to the high-resolution image patch training sample set Y h to obtain K nearest neighbor high-resolution image patches, forming a set

[0039] (3) Generate a reconstruction model: Obtain the predicted high-resolution image patch: The regularized empirical error is:

[0040]

[0041] Expressed as:

[0042] where λ is the regularization parameter, used to balance the relationship between the reconstruction error and the regularization term; I is the identity matrix, is the high-resolution image patch, is the low-resolution image patch, is the high-resolution image patch training sample set, is the low-resolution image patch training sample set, and φ i is the regularized empirical error; is the regularization term, used to constrain the super-resolution reconstruction result.

[0043] (4) The obtained high-resolution image block for the predicted output is:

[0044] (5) Smooth processing of the edge pixels of the predicted output high-resolution image block;

[0045] Furthermore, the smooth processing of the edge pixels of the high-resolution image block is to obtain the image I by using the second derivative in the orthogonal diagonal direction of the pixels, H which is expressed as:

[0046]

[0047] where I(i, j) is the edge pixel selected for smooth processing in the image block. In the image coordinate system, I(i + 3, j + 1) represents the pixel determined by the intersection of the fourth column in the positive X-axis direction and the second row in the positive Y-axis direction with the edge pixel I(i, j) as the origin, and the same applies to others. F L is the second derivative in the direction from the upper left corner to the lower right corner of the image, and F H is the second derivative in the direction from the upper right corner to the lower left corner of the image;

[0048] If F L < F H , then:

[0049]

[0050] If F L > F H , then:

[0051]

[0052] Embodiment 1

[0053] As Figure 2 shown, the area selected by the red box is a typical defect of the laser additive manufacturing workpiece. Specifically, the region of interest (ROI) is selected for super-resolution reconstruction processing to achieve the extraction and characterization of the key defect information. The results after the reconstruction processing are respectively placed on the lower left side or the lower right side of the image. Subjectively, the super-resolution image reconstructed by the method of the present invention has a higher contrast, and the edge detail information of the defect area is clear. The overall quality of the image is high, and there are no obvious distortion, blurring, and distortion phenomena.

[0054] As Figure 3 shown are the edge detection and defect feature extraction results of the super-resolution reconstruction image processed by the method of the present invention. Obviously, the edge of the defect area is complete, the contour is clear, and the key defect information is not lost, which can provide good technical support for the subsequent defect characterization.

[0055] The present invention further uses information entropy (E), standard deviation (SD), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) to evaluate the performance of the super-resolution reconstruction and feature extraction of the defective detection image, and the analysis results are respectively as Figures 4 to 7 shown. In the comparative experiment, the performance of the super-resolution reconstructed images under different scaling factors s was obtained. It can be seen from the figure that the values of each evaluation index are larger when the scaling factor is 3 compared to other scaling factors. When the scaling factor is 3, the information entropy, standard deviation, peak signal-to-noise ratio, and structural similarity of Image 1 are 7.498, 49.357, 28.743, and 0.822 respectively, the information entropy, standard deviation, peak signal-to-noise ratio, and structural similarity of Image 2 are 7.404, 48.767, 29.539, and 0.830 respectively, the information entropy, standard deviation, peak signal-to-noise ratio, and structural similarity of Image 3 are 7.020, 36.727, 31.123, and 0.917 respectively, and the information entropy, standard deviation, peak signal-to-noise ratio, and structural similarity of Image 4 are 7.113, 46.354, 21.486, and 0.719 respectively.

[0056] In the case of the image scaling factors of 4, 5, and 6 for the method of the present invention, compared with the case where the scaling factor is 3, the information entropy, standard deviation, peak signal-to-noise ratio, and structural similarity all show a certain degree of decline. The average decline rates of the information entropy index are 0.90%, 2.26%, and 4.27% respectively, the average decline rates of the standard deviation index are 8.28%, 14.37%, and 21.98% respectively, the average decline rates of the peak signal-to-noise ratio index are 11.64%, 18.81%, and 23.54% respectively, and the average decline rates of the structural similarity index are 14.56%, 26.37%, and 36.23% respectively.

[0057] When the scaling factor is between 3 and 6, the average improvement rates of the information entropy E index of the method of the present invention compared with the bicubic interpolation algorithm [1] are 0.45%, 0.27%, 0.23% and 0.16% respectively, and the average improvement rates of the information entropy E index compared with the neighborhood embedding algorithm are 0.97%, 0.65%, 0.71% and 0.40% respectively; the average improvement rates of the standard deviation index of the algorithm proposed in this chapter compared with the bicubic interpolation algorithm are 7.15%, 3.36%, 2.30% and 1.57% respectively, and the average improvement rates of the standard deviation index compared with the neighborhood embedding algorithm are 13.40%, 9.83%, 9.12% and 9.15% respectively. Further, the average improvement rates of the peak signal-to-noise ratio index of the method of the present invention compared with the bicubic interpolation algorithm are 5.85%, 2.02%, 0.83% and 0.38% respectively, and the average improvement rates of the peak signal-to-noise ratio index compared with the neighborhood embedding algorithm are 10.55%, 6.94%, 5.50% and 5.89% respectively. Further, the average improvement rates of the structural similarity index of the method of the present invention compared with the bicubic interpolation algorithm are 6.35%, 3.97%, 2.04% and 1.28% respectively, and the average improvement rates of the structural similarity SSIM index compared with the neighborhood embedding algorithm [2] are 15.35%, 14.25%, 13.81% and 15.51% respectively. From the above analysis, it can be seen that the method of the present invention is significantly superior to the comparative method in the quality of super-resolution reconstructed images.

[0058] To prove the robustness of the method of the present invention, super-resolution reconstruction was further performed on images with salt-and-pepper noise, Gaussian noise, and speckle noise added respectively. The mean of the added noise was 0, and the standard deviation was 0.008. In the case of adding Gaussian noise, compared with the bicubic interpolation algorithm, the method of the present invention improved by 0.17%, 6.98%, 2.78%, and 9.32% respectively in terms of four indicators: information entropy, standard deviation, peak signal-to-noise ratio, and structural similarity. Compared with the neighborhood embedding algorithm, it improved by 0.60%, 15.38%, 13.32%, and 18.43% respectively. Further, in the case of adding salt-and-pepper noise, compared with the bicubic interpolation algorithm, the method of the present invention improved by 0.13%, 7.19%, 6.12%, and 6.87% respectively in terms of the four indicators of information entropy, standard deviation, peak signal-to-noise ratio, and structural similarity. Compared with the neighborhood embedding algorithm, it improved by 0.66%, 15.49%, 17.92%, and 16.76% respectively. Further, in the case of adding speckle noise, compared with the bicubic interpolation algorithm, the method of the present invention improved by 0.51%, 7.20%, 6.64%, and 5.80% respectively in terms of the four indicators of information entropy, standard deviation, peak signal-to-noise ratio, and structural similarity. Compared with the neighborhood embedding algorithm, it improved by 1.06%, 15.07%, 17.12%, and 14.54% respectively. Obviously, the method of the present invention has good robustness, and each evaluation index is significantly better than the comparative algorithm, which is of great significance for the extraction and analysis of key defect information in the surface defect detection process of additive manufacturing workpieces.

[0059] The above embodiments are only one implementation mode of the present invention, but the implementation mode of the present invention is not limited by the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

[0060] References

[0061] [1] Zhao Xiaole. Research on Single-Image Super-Resolution Technology. Southwest University of Science and Technology, 2013.

[0062] [2] Chang H, Yeung D Y, Xiong Y M. Super-resolution through neighbor embedding. IEEE Computer Society Conference on Computer Vision & Pattern Recognition 2004. 2004: 1063 - 6919.

Claims

1. A super-resolution reconstruction and feature extraction method for defect detection, characterized in that, Based on the direct mapping method, it directly establishes a linear mapping relationship between the low-resolution image patches and the high-resolution image patch training sample set, obtains the regression function through learning, and then directly maps the input low-resolution image patches to the predicted output high-resolution image patches through the regression function to obtain the high-resolution reconstructed image; Specifically, it includes the following steps: (1) Divide the input low-resolution image I L into non-overlapping image patches of 3×3 pixel size, and for each image patch in the externally input low-resolution image test set X l calculate its K nearest neighbor image patches in the low-resolution image patch training sample set Y and form a set l which consists of the corresponding K nearest neighbor image patches in the set (2) Corresponding the set P of low-resolution image patches of the K nearest neighbors l to the set Y of high-resolution image patch training samples h to obtain K high-resolution image patches of the nearest neighbors to form a set (3) Generate a reconstruction model: Obtain predicted high-resolution image patches: The regularized empirical error is: Expressed as: where λ is the regularization parameter used to balance the relationship between the reconstruction error and the regularization term; I is the identity matrix, is the high-resolution image patch, is the low-resolution image patch, is the high-resolution image patch training sample set, is the low-resolution image patch training sample set, φ i is the regularized empirical error; is the regularization term used to constrain the super-resolution reconstruction result; (4) The obtained high-resolution image patch of the predicted output is: (5) Smooth the edge pixels of the predicted output high-resolution image patches; (6) Extract the edge region of the defect from the image obtained in step (5) by using the Canny edge detection method.

2. The super-resolution reconstruction and feature extraction method for defect detection according to claim 1, wherein, In step (5), the smoothing process of the edge pixels of the high-resolution image patches uses the second derivative in the orthogonal diagonal direction of the pixels to represent the image, expressed as: Among them, I(i, j) is the edge pixel selected for smoothing in the image block. In the image coordinate system, I(i + 3, j + 1) represents the pixel determined by the intersection of the fourth column in the positive X-axis direction and the second row in the positive Y-axis direction with the edge pixel I(i, j) as the origin, and the same applies to others; F L is the second derivative in the direction from the upper left corner to the lower right corner of the image, and F H is the second derivative in the direction from the upper right corner to the lower left corner of the image; If F L <F H , then: If F L > F H , then:

Images