Terahertz image registration method fusing morphological features

Abstract

The invention discloses a terahertz image registration method fusing morphological characteristics. The terahertz image registration method comprises the following steps: filtering and denoising a terahertz image; performing adaptive threshold segmentation processing on the reference image and the to-be-registered image, and respectively generating three types of images, namely a grayscale image, a binary image and an edge image; extracting texture features, geometric features and contour features from the generated grayscale image, the binary image and the edge image, and carrying out integration and feature description on the extracted feature points; mismatching points are removed through bidirectional feature point matching, registration parameters are calculated based on correct matching points, and image registration and verification are completed. According to the method, the number of extracted feature points in terahertz can be increased, multiple features are fused for registration, the registration precision is improved, image registration can be achieved in similar images, the practicability is high, and the application range is wide.

Description

Technical Field

[0001] The invention belongs to the image processing technology in the field of terahertz time-domain spectroscopy nondestructive testing, and specifically relates to a terahertz image registration method integrating morphological features. Background Art

[0002] Terahertz waves refer to electromagnetic waves with a wavelength range of 30μm-3mm. Its long-wave band coincides with millimeter waves, and its short-wave band coincides with infrared light. It is the transition area from electronics to photonics. With its advantages of strong penetration, non-contact and high sensitivity to organic matter, terahertz waves have become an emerging technology in the field of non-destructive testing, especially in the use of terahertz images to detect internal damage and defects in materials, showing broad application prospects.

[0003] In recent years, terahertz nondestructive testing technology has developed towards terahertz nondestructive evaluation. For the full life cycle testing of materials and products, the evolution analysis of samples before and after the loading environment is crucial. However, due to the different testing environments and conditions, such as different testing angles, testing resolutions or testing equipment, it is difficult to directly realize the evolution analysis of terahertz images before and after the sample is loaded. The typical method is to align and fuse images under different conditions, and to align the feature points of these images by fusing multiple types of images such as infrared images, visible light images, etc. (CN 112381747A). However, the existing registration technology still faces problems such as noise interference and image deformation when processing terahertz images, which affects the registration accuracy. In addition, the terahertz image registration method based on deep learning can realize multi-time image registration, and can handle the deformation and displacement problems of images under different time and environmental conditions. However, deep learning methods usually require a large amount of training data and a long calculation time. Summary of the invention

[0004] The purpose of the present invention is to provide a terahertz image registration method that integrates morphological features. By integrating morphological features, terahertz image registration under different detection conditions can be achieved, which can effectively deal with problems such as noise interference and image deformation, so as to improve the accuracy of terahertz image registration and image change analysis, and provide a more reliable technical means for the change analysis of samples before and after loading.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A terahertz image registration method integrating morphological features comprises the following steps:

[0007] S1, filtering and denoising the terahertz image;

[0008] S2. Perform adaptive threshold segmentation on the reference image and the image to be registered, respectively generating three types of images: a grayscale image, a binary image, and an edge image;

[0009] S3. Extract texture features, geometric features, and contour features from the generated grayscale image, binary image, and edge image respectively, and integrate and describe the extracted feature points;

[0010] S4. Eliminate the mismatched points through two-way feature point matching, and calculate the registration parameters based on the correctly matched points to complete image registration and verification.

[0011] Furthermore, the step S1 includes: using a median filter to denoise the terahertz image based on the median of the sliding window.

[0012] Furthermore, the step S2 includes:

[0013] S21. For the reference image and the image to be registered, adopt an adaptive threshold segmentation algorithm to dynamically determine the threshold of each pixel point based on the mean value of the local area of the image;

[0014] S22. Normalize the result image of the threshold segmentation to generate a grayscale image, retaining the grayscale information of the target area of the image;

[0015] S23. The segmented binary mask is the binary image;

[0016] S24. Use an edge detection algorithm to extract the edges of the image after adaptive threshold segmentation processing.

[0017] Furthermore, the step S24 includes: calculating the gradient magnitude and direction of the image, reducing the pseudo-edge points through non-maximum suppression, and determining the edge pixel points in combination with the double-threshold detection method to obtain the edge image and display the edge contour information in the image.

[0018] Furthermore, the step S3 includes:

[0019] S31. For the grayscale image, adopt the gray-level co-occurrence matrix method to extract the texture feature F texture , calculate the gray-level combination probability of pixel pairs in different directions and distances in the image to obtain the texture feature parameters;

[0020] S32. For the binary image, adopt the KAZE algorithm to extract the geometric feature points F geometry ;

[0021] S33. For the edge image, adopt the contour tracking algorithm to obtain the contour point sequence of the target, thereby extracting the contour feature F contour ;

[0022] S34. Integrate and process the texture features, geometric features, and contour feature points separately extracted from the grayscale image, binary image, and edge image.

[0023] Further, in step S31, by statistically analyzing the joint distribution frequency of pixel gray values in the image in terms of feature methods and distances, construct a two-dimensional probability density distribution matrix, and calculate texture feature parameters such as contrast, correlation, and energy:

[0024] The contrast calculation formula is:

[0025]

[0026] The correlation calculation formula is:

[0027]

[0028] The energy calculation formula is:

[0029]

[0030] where L is the gray level, P(i,j) is the probability that gray values i and j appear simultaneously, μ i 、μ j 、σ i 、σ j are the mean and standard deviation of gray values i and j, respectively.

[0031] Further, in step S34, for each feature point, construct a multi-dimensional feature vector. Each dimension of the vector sequentially represents the attributes of the feature point in different feature spaces, including texture features, geometric features F geometry and contour features F contour , the texture feature parameters reflect the distribution characteristics of the texture, the geometric feature parameters include coordinate position and scale information, and the contour feature parameters include contour point position and curvature geometric characteristics. By uniformly mapping the multiple attributes of the above feature points into a unified feature vector, a comprehensive representation of the feature points in the three feature spaces of texture, geometry, and contour is achieved. To ensure the dimensional consistency and scale consistency of different feature parameters, normalization processing is performed on each parameter. This comprehensive feature vector improves the accuracy of feature description, enhances the robustness of feature matching, and provides higher reliability and adaptability for subsequent image registration.

[0032] Further, step S4 includes:

[0033] S41. Use a two-way feature point matching algorithm to match the final feature vector, and use the RANSAC algorithm to eliminate mis-matched points;

[0034] S42. After removing the mismatched points, the least squares method is used to estimate the registration parameters of translation, rotation, and scaling based on the coordinate information of the correct matching points;

[0035] S43. Use the calculated registration parameters to perform transformation operations on the image to be registered to complete image registration.

[0036] Further, the step S42 includes:

[0037] Based on the coordinate information of the remaining correct matching points, the least squares method is used to estimate the registration parameters to obtain the transformation matrix H of the image;

[0038] Based on the transformation matrix H, let the coordinate of a point in the image to be registered be (x, y, 1) T , and the registered coordinate be (x', y', 1) T , and the registration is achieved through the following formula:

[0039]

[0040] where H is a 3×3 homogeneous coordinate transformation matrix, which contains rotation, translation, scaling, and other transformation information in the registration process;

[0041] Introduce the reprojection error as an evaluation criterion. For each pair of matching points, the reprojection error is defined as the distance between the reprojection of a point in the image to be registered after passing through the transformation matrix H and the corresponding point in the actual reference image. The error formula is:

[0042]

[0043] Calculate the average reprojection error of all matching points to evaluate the overall registration effect.

[0044] The present invention has the following beneficial effects:

[0045] The present invention adaptively threshold-segments the filtered and denoised terahertz image to generate three types of images: grayscale image, binary image, and edge image. Different types of feature points are extracted according to the characteristics of these three images, and the terahertz image features are generally described from texture, geometry, and contour features. The two-way matching method is used to match the feature points. After removing the mismatched points by combining with the RANSAC algorithm, the least squares method is used to calculate the transformation matrix, and the registration result is obtained through the matrix.

[0046] The present invention can not only increase the number of feature points extracted in terahertz, but also fuse multiple features for registration, improve the registration accuracy, and can achieve image registration in similar images, with strong practicability and wide application range. Description of the Drawings

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for the embodiments will be briefly introduced below.

[0048] Figure 1 Flowchart of a terahertz image registration method integrating morphological features according to an embodiment of the present invention;

[0049] Figure 2 Images after denoising the original image and the image to be registered in an embodiment of the present invention;

[0050] Figure 3 Three types of images generated after adaptive thresholding of the original image and the image to be registered in an embodiment of the present invention. Detailed implementation manners

[0051] To make the objectives, technical solutions and advantages of the invention more clear and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0052] As Figure 1 shown, an embodiment of the present invention provides a terahertz image registration method integrating morphological features, including the following steps:

[0053] S1. Filter and denoise the terahertz image, as Figure 2 shown;

[0054] Use a median filter to denoise the terahertz image based on the median of the sliding window, so as to effectively suppress high-frequency noise while retaining the edge detail features of the image.

[0055] S2. Perform adaptive threshold segmentation processing on the reference image and the image to be registered, and generate three types of images: grayscale image, binary image and edge image respectively, as Figure 3 shown;

[0056] S21. For the reference image and the image to be registered, adopt an adaptive threshold segmentation algorithm to dynamically determine the threshold of each pixel point based on the mean value of the local area of the image. Specifically, a square window area with a side length of 1 / 2 is selected centered on the pixel point, and the points with pixel values greater than the threshold are marked as target pixels, and the points with pixel values less than the threshold are marked as background pixels. The calculation formula is as follows:

[0057]

[0058] where, T x,y is the adaptive threshold of the pixel point (i,j), I(i,j) is the mean value of the pixel values within the window centered on the pixel point (i,j), and Δ is the offset of the threshold, which is adjusted according to features such as the standard deviation of the local image.

[0059] S22. Normalize the result image of the threshold segmentation to generate a grayscale image, retaining the grayscale information of the target area of the image, thereby enhancing the contrast of the image and enhancing the image details.

[0060] S23. The binary mask after segmentation, i.e., the binary image, simplifies the structure of the image.

[0061] S24. Use an edge detection algorithm to extract the edges of the image after adaptive threshold segmentation. The specific steps include: calculating the gradient magnitude and direction of the image, reducing the pseudo-edge points through non-maximum suppression, and combining the double-threshold detection method to determine the edge pixel points, obtaining the edge image, and displaying the edge contour information in the image.

[0062] S3. Extract the texture feature F, geometric feature F, and contour feature F from the generated grayscale image, binary image, and edge image respectively. texture and geometry and integrate and describe the extracted feature points as the basis for subsequent terahertz image registration. contour

[0063] S31. For the grayscale image, use the gray-level co-occurrence matrix method to extract the texture feature F. texture By statistically analyzing the joint distribution frequency of pixel gray values in the feature method and distance in the image, construct a two-dimensional probability density distribution matrix, and calculate texture feature parameters such as contrast, correlation, and energy:

[0064] The contrast is used to measure the local variation degree of the gray distribution of the image. The higher the contrast, the more obvious the texture change of the image. The contrast calculation formula is:

[0065]

[0066] The correlation reflects the linear correlation between gray values. The higher the correlation, the stronger the dependence relationship between gray values. The correlation calculation formula is:

[0067]

[0068] The energy describes the uniformity of the gray distribution. The energy calculation formula is:

[0069]

[0070] where L is the gray level, P(i,j) is the probability that gray values i and j appear simultaneously, μ i , μ j , σ i , σ j are the mean and standard deviation of gray values i and j respectively.

[0071] These texture feature parameters comprehensively describe the texture structure characteristics of the image from different perspectives.

[0072] S32. For a binary image, the KAZE algorithm is used to extract feature points. The KAZE algorithm is based on a non-linear scale space. By calculating the Laplacian response at multiple scales, it detects feature points and generates a rotation-invariant feature descriptor to extract high-precision, multi-scale, and invariant geometric feature points F geometry .

[0073] S33. For an edge image, a contour tracking algorithm is used to obtain the sequence of contour points of the target. Starting from the upper left corner of the image, edge pixels are searched in a clockwise or counterclockwise direction. When the first edge pixel is found, the coordinates of the contour points are sequentially recorded along the edge direction. Based on the obtained sequence of contour points, the length G of the contour is calculated, and the curvature K of the contour is obtained by calculating the sum of the distances between adjacent contour points. The contour length and curvature can reflect the geometric characteristics of the contour at different positions F contour = [G, K].

[0074] S34. The texture features, geometric features, and contour feature points extracted from the grayscale image, binary image, and edge image respectively are integrated. Specifically, for each feature point, a multi-dimensional feature vector is constructed. Each dimension of the multi-dimensional feature vector sequentially represents the attributes of the feature point in different feature spaces, including texture feature F texture , geometric feature F geometry and contour feature F contour = [G, K]. The texture feature parameters reflect the distribution characteristics of the texture. The geometric feature parameters include coordinate position and scale information. The contour feature parameters include contour point position and curvature geometric characteristics. By uniformly mapping the multiple attributes of the above feature points into a unified feature vector, a comprehensive representation of the feature points in the three feature spaces of texture, geometry, and contour is achieved. To ensure the dimensional consistency and scale consistency of different feature parameters, normalization processing is performed on each parameter. This comprehensive feature vector improves the accuracy of feature description, enhances the robustness of feature matching, and provides higher reliability and adaptability for subsequent image registration.

[0075] S4. Eliminate the mis-matched points through two-way feature point matching, and calculate the registration parameters based on the correct matching points, thereby completing image registration and verification;

[0076] S41. For each feature point P r in the reference image, search for the feature point P t in the image to be registered that best matches it. This matching process is based on the distance metric between feature vectors and is calculated using the Euclidean distance. The formula is:

[0077]

[0078] where, F r,i and F t,i are respectively the i-th eigenvector component of the feature points in the reference image and the image to be registered, and n is the dimension of the eigenvector. Starting from the point P t in the image to be registered, search backward in the reference image for the matching point. If the backward matching point is the same as the original P r point, it is considered a correct matching point; otherwise, it is determined as a mismatching point. On this basis, the Random Sample Consensus (RANSAC) algorithm is used to screen the matching points and eliminate the mismatching points.

[0079] S42. Based on the coordinate information of the remaining correct matching points, use the least squares method to estimate the registration parameters and obtain the transformation matrix H of the image.

[0080] Based on the transformation matrix H, assume the coordinate of a point in the image to be registered is (x, y, 1) T , and the registered coordinate is (x', y', 1) T . The registration can be achieved through the following formula:

[0081]

[0082] where, H is a 3×3 homogeneous coordinate transformation matrix, which contains the rotation, translation, scaling and other transformation information in the registration process.

[0083] To further verify the registration accuracy, the reprojection error is introduced as the evaluation criterion. For each pair of matching points, the reprojection error is defined as the distance between the reprojection of a point in the image to be registered after passing through the transformation matrix H and the corresponding point in the actual reference image. The error formula is:

[0084]

[0085] Calculate the average reprojection error of all matching points to evaluate the overall registration effect.

[0086] The above content is only the preferred embodiment of the present invention. For those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. The content of this specification should not be construed as a limitation to the present invention.

Claims

1. A terahertz image registration method integrating morphological features, characterized in that: The following steps are involved: S1, filtering and denoising the terahertz image; S2, performing adaptive threshold segmentation processing on the reference image and the image to be registered, and generating three types of images: grayscale image, binary image and edge image; S3, extracting texture features, geometric features and contour features from the generated grayscale image, binary image and edge image respectively, and integrating and describing the extracted feature points; S4. Eliminate mismatched points through bidirectional feature point matching, and calculate registration parameters based on correct matching points to complete image registration and verification.

2. A terahertz image registration method integrating morphological features as claimed in claim 1, characterized in that: The step S1 includes: using a median filter to denoise the terahertz image based on the median of the sliding window.

3. The terahertz image registration method integrating morphological features according to claim 1, characterized in that: The step S2 comprises: S21, for the reference image and the image to be registered, an adaptive threshold segmentation algorithm is used to dynamically determine the threshold of each pixel based on the mean value of the local area of ​​the image; S22, normalizing the result image of the threshold segmentation to generate a grayscale image, retaining the grayscale information of the target area of ​​the image; S23, the segmented binary mask is a binary image; S24, using an edge detection algorithm to extract edges from the image after the adaptive threshold segmentation process.

4. The terahertz image registration method integrating morphological features as claimed in claim 3, characterized in that: The step S24 includes: calculating the gradient amplitude and direction of the image, reducing pseudo edge points by non-maximum suppression, and determining edge pixel points in combination with a dual threshold detection method to obtain an edge image and display edge contour information in the image.

5. The terahertz image registration method integrating morphological features according to claim 1, characterized in that: The step S3 comprises: S31. For grayscale images, the grayscale co-occurrence matrix method is used to extract texture features F texture , calculate the grayscale combination probability of pixel pairs in different directions and distances in the image, and obtain the texture feature parameters; S32, for the binary image, use the KAZE algorithm to extract geometric feature points F geometry ; S33, for the edge image, use the contour tracking algorithm to obtain the target contour point sequence, so as to extract the contour feature F contour ; S34, integrating the texture features, geometric features and contour feature points extracted from the grayscale image, the binary image and the edge image respectively.

6. The terahertz image registration method integrating morphological features as claimed in claim 5, characterized in that: In step S31, by counting the joint distribution frequency of pixel grayscale value pairs in the image in terms of feature method and distance, a two-dimensional probability density distribution matrix is ​​constructed to calculate texture feature parameters such as contrast, correlation, and energy: The contrast ratio is calculated as: The correlation calculation formula is: The energy calculation formula is: Where L is the gray level, P(i,j) is the probability that gray values ​​i and j appear at the same time, μ i , μ j , σ i , σ j are the mean and standard deviation of grayscale values ​​i and j respectively.

7. The terahertz image registration method integrating morphological features as claimed in claim 5, characterized in that: In step S34, for each feature point, a multidimensional feature vector is constructed, and each dimension of the multidimensional feature vector represents the attribute of the feature point in different feature spaces in turn, including the texture feature F texture , geometric features F geometry and contour feature F contour ,The texture feature parameters reflect the distribution characteristics of the texture, the ,geometric feature parameters include the coordinate position and scale information, and the contour ,feature parameters include the contour point position and the curvature geometric ,characteristics.

8. The terahertz image registration method integrating morphological features according to claim 1, characterized in that: The step S4 comprises: S41, using a bidirectional feature point matching algorithm to match the final feature vector, and using a RANSAC algorithm to eliminate mismatched points; S42, after removing the incorrect matching points, using the least square method to estimate the registration parameters of translation, rotation and scaling according to the coordinate information of the correct matching points; S43, using the calculated registration parameters to perform a transformation operation on the image to be registered, to complete the image registration.

9. The terahertz image registration method integrating morphological features as claimed in claim 8, characterized in that: The step S42 comprises: Based on the coordinate information of the remaining correct matching points, the least squares method is used to estimate the registration parameters to obtain the image transformation matrix H; Based on the transformation matrix H, let the coordinates of a point in the image to be registered be (x, y, 1) T , the coordinates after registration are (x',y',1) T , the registration is achieved through the following formula: Among them, H is a 3×3 homogeneous coordinate transformation matrix, which contains the rotation, translation, scaling and other transformation information in the registration process; The reprojection error is introduced as the evaluation criterion. For each pair of matching points, the reprojection error is defined as the distance between the reprojection of a point in the image to be registered after the transformation matrix H and the corresponding point in the actual reference image. The error formula is: The average reprojection error of all matching points is calculated to evaluate the overall registration effect.

Images