Three-dimensional fusion imaging method for laser ultrasonic ray infrared data

Abstract

The invention relates to a three-dimensional fusion imaging method for laser ultrasonic ray infrared data, which belongs to the technical field of nondestructive testing, and comprises the following steps of: performing infrared thermal imaging detection, laser ultrasonic detection and ray detection, then arranging a precisely positioned mark point network on the surface of a sample piece, and performing feature enhancement, defect identification and precise segmentation on various detection data, so as to obtain the three-dimensional fusion imaging method for the laser ultrasonic ray infrared data. The processed defect information is converted into point cloud data with material attributes, and a high-resolution three-dimensional grid model is established according to the detection precision requirement. Through space mapping and coordinate transformation, accurate presentation of multi-source defect point cloud in a three-dimensional model is realized, and cylindrical coordinate system transformation is supported to adapt to imaging requirements of a curved surface structure. According to the invention, the visual display of defects is provided, the quantitative analysis and intelligent evaluation functions of defects are integrated, and a reliable basis is provided for subsequent maintenance decisions.

Description

Technical Field

[0001] This invention belongs to the field of nondestructive testing technology, specifically relating to a three-dimensional fusion imaging method for laser, ultrasonic, X-ray, and infrared data. Background Technology

[0002] Traditional nondestructive testing methods (such as ultrasound, X-ray, and infrared) are limited by their single-modal detection capabilities, making it difficult to comprehensively identify multi-scale and multi-type defects in multi-layered composite structures. Existing technologies for 3D data fusion often employ 2D image overlay or simple geometric alignment, resulting in inaccurate defect localization, poor multi-source data coordination, and insufficient visualization. Furthermore, full 3D inspection of large components faces challenges such as large data volumes and difficulties in real-time processing. Therefore, a high-precision and high-efficiency multi-source data 3D fusion imaging method is urgently needed. Summary of the Invention

[0003] The purpose of this invention is to provide a three-dimensional fusion imaging method for laser, ultrasound, X-ray, and infrared data. Through multimodal data collaborative acquisition, intelligent registration, feature fusion, and three-dimensional reconstruction, it achieves comprehensive detection and visualization of defects in multi-layered composite structures.

[0004] The above-mentioned objectives of the present invention are mainly achieved through the following technical solutions: A three-dimensional fusion imaging method for laser, ultrasound, and infrared data includes the following steps: (1) Perform infrared thermography, laser ultrasonic testing and X-ray testing on the object to be tested to obtain defect information respectively; (2) Establish a three-dimensional model of the object to be detected, set marker points on the three-dimensional model, and divide the object to be detected into multiple regions according to the marker points; (3) Extract, locate and transform the defect information obtained from infrared thermal imaging, laser ultrasonic detection and X-ray detection in step (1) respectively to obtain defect point cloud maps; (4) The defect point cloud maps obtained in step (3) by infrared thermal imaging, laser ultrasonic detection and X-ray detection are image registered with the three-dimensional model obtained in step (2) respectively, and the defect point cloud is mapped to the corresponding position of the three-dimensional model.

[0005] In step (2), the detection object is divided into multiple square regions by the marker points, and the position information of the four corner points of each region corresponds one-to-one with the global coordinate system of the three-dimensional model through coordinate transformation relationship.

[0006] In step (3), laser ultrasonic testing uses a neural network to perform image recognition, obtains the coordinates of the defect relative to the image origin, and generates a point cloud.

[0007] After dividing the 3D model into multiple regions, laser ultrasonic testing is performed. The laser ultrasonic testing adopts a region-by-region scanning method, and each region is scanned sequentially. After the testing is completed, a point cloud map of each region is generated. The coordinate system angle is transformed by the positional relationship between the origin of each region and the two nearest positioning points to obtain the global location of the ultrasonic defect, thus obtaining the defect point cloud map.

[0008] In step (3), the defect image obtained by ray detection is preprocessed. The preprocessing method is as follows: first, spatial domain and frequency domain filtering is performed, then adaptive threshold binarization is performed, and then the image structure is optimized by morphological operations to fill in contour breaks and remove isolated noise points.

[0009] After the preprocessing is completed, edge extraction is performed to obtain the contour boundary of the defect area. Defect contour points are extracted and converted into a coordinate set with spatial attributes. The coordinate set is spatially aligned and mapped with the coordinates of the three-dimensional model to realize the three-dimensional projection and annotation of the defect area on the surface or inside the model.

[0010] In step (3), defects detected by infrared thermal imaging are extracted by edge extraction algorithm, and different layered defects are saved to separate point cloud files. The depth position of the detected layer is determined according to the temperature change characteristics of different depth layers under excitation. The temperature change of the surface of the detected object is obtained by edge extraction algorithm. The distorted matrix is ​​adjusted to a standard matrix by image segmentation and transformation. The planar defects are identified and the defect range is determined by neural network. Then, point cloudification is performed.

[0011] In step (4), feature points of the two images are detected for matching. The image transformation matrix is ​​solved based on the coordinate transformation of the successfully matched feature point pairs to complete image registration. If the matched image I x Image I to be matched y The pixel coordinates are I x (i, j) and I y If (i,j), then the correspondence after image registration is as follows:

[0012] Where h(i,j) is the coordinate transformation function of the spatial position of each pixel in the image.

[0013] Using projective transformation for coordinate transformation, the coordinates of the pixels in the image being matched and the pixels in the image to be matched are (X,Y) and (x,y) respectively. The coordinate transformation formula is:

[0014] Where k is a non-zero scaling factor and a is a scaling factor.

[0015] If the object to be inspected is a curved structure, a transformation from Cartesian coordinates to cylindrical coordinates is performed, and the defect information from infrared thermography, laser ultrasonic testing, and X-ray inspection is spatially transformed based on the visual projection relationship between planar defects and curved defects.

[0016] Compared with the prior art, the present invention has at least the following beneficial effects: (1) This invention achieves comprehensive detection of multi-layer composite structures by leveraging the complementary advantages of laser ultrasound, X-ray and infrared thermography, and improves the defect detection rate compared with single detection methods.

[0017] (2) In the preferred embodiment of the present invention, the positioning accuracy of different detection results is higher than that of traditional fusion methods by using marker registration and neural network feature extraction.

[0018] (3) In the preferred embodiment of the present invention, the spatial distribution and relative position of various defects are displayed intuitively through a three-dimensional visualization interface, which facilitates the comprehensive analysis and damage assessment by the inspection personnel.

[0019] (4) The present invention is preferably applied to the detection of various curved surface components by coordinate system transformation, and can be extended to the non-destructive testing of complex structures such as aerospace multilayer components and automotive composite materials. Attached Figure Description

[0020] Figure 1 This is a flowchart of the three-dimensional reconstruction process of the present invention; Figure 2 This is a schematic diagram of the image registration process of the present invention; Figure 3 This is a schematic diagram of the aluminum alloy adhesive heat-resistant structure in Embodiment 1 of the present invention; Figure 4 This is a diagram of the three-layer three-dimensional structure of Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of laser ultrasound region-by-region scanning in Embodiment 1 of the present invention; Figure 6 This is a diagram showing the laser ultrasound detection results of Embodiment 1 of the present invention; Figure 7 This is a diagram illustrating the process of determining the location and generating the point cloud of a laser ultrasonic defect in Embodiment 1 of the present invention. Figure 8 This is a diagram showing the extraction and location of defects detected by X-ray in Embodiment 1 of the present invention; Figure 9 This is a diagram showing the detection results of aluminum alloy debonding defects in Example 1 of the present invention; Figure 10 To identify the first layered defect map in Embodiment 1 of the present invention; Figure 11 This is a cloudification result diagram of the first debonding defect point in Embodiment 1 of the present invention; Figure 12 This is an example of importing the point cloud of the first debonding defect into the constructed 3D model in Embodiment 1 of the present invention. Figure 13 This is a comparison chart showing the results of importing all debonding defects into a three-dimensional model in Embodiment 1 of the present invention; Figure 14 This is a schematic diagram of the reconstruction result obtained by importing defects into a three-dimensional model in Embodiment 1 of the present invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments: like Figure 1 As shown, the technical solution provided by this invention is: First, an optimized combination of detection methods is used for different material layers and defect types. Laser ultrasound is used for the precise detection of internal folds and holes in aluminum alloy interfaces, X-rays are used for the detection of non-uniform defects inside heat-resistant structures, and infrared thermography is used for the detection of delamination defects in composite materials and delamination defects at adhesive interfaces.

[0022] Secondly, a network of precisely positioned markers is arranged on the surface of the sample to provide a unified spatial reference benchmark for multi-source data.

[0023] Then, advanced image processing algorithms and neural network technology are employed to perform feature enhancement, defect identification, and precise segmentation on various types of detection data. Specialized processing algorithms are used for different types of detection data, performing preprocessing and feature extraction. After acquiring infrared, laser ultrasound, and X-ray detection results, feature extraction algorithms are first used to enhance the features of the detection result images. Subsequently, neural networks are used to identify the enhanced defect features, obtaining defect size and location information. Specifically, infrared images use edge extraction algorithms to obtain the matrix of the composite material surface area, with the lower left corner of the material as the origin. Laser ultrasound data requires recording the position information of the detection area within the surface grid; each scan area should contain at least four positioning points. X-ray images, targeting typical defect areas identified in the image, undergo preprocessing through spatial and frequency domain filtering (such as median filtering, Gaussian filtering, Fourier denoising, etc.), followed by binarization and edge extraction to enhance defect features. Image segmentation is then used to segment and classify the multi-source raw detection images based on labeled data provided by the neural network.

[0024] During the data fusion stage, the processed defect information is converted into point cloud data with material properties, and a high-resolution three-dimensional mesh model is established according to the detection accuracy requirements.

[0025] Ultimately, through spatial mapping and coordinate transformation, the multi-source defect point cloud is accurately represented in the 3D model, and cylindrical coordinate transformation is supported to adapt to the imaging requirements of curved surface structures. This invention not only provides a visual representation of defects but also integrates defect quantification analysis and intelligent evaluation functions, providing a reliable basis for subsequent maintenance decisions.

[0026] The specific implementation of this invention can be divided into three main stages: data acquisition, data processing, and result integration.

[0027] During the data acquisition phase, the operating parameters of the three testing devices need to be rationally configured according to the material properties and structural characteristics of the object being tested. Taking infrared thermography as an example, when using an infrared camera, the field of view needs to be adjusted to completely cover the sample. For laser ultrasonic testing, the scanning step size and laser energy need to be set to ensure sensitivity for detecting small-scale defects. For radiographic testing, appropriate transmission parameters should be selected based on the material thickness of the heat-resistant structure. Laser ultrasonic, radiographic, and infrared thermography methods acquire multi-source testing data from different structural locations on the tested part.

[0028] In the data processing stage, the first step is to establish a 3D model of the sample. This involves defining the 3D dimensions of the sample and the resolution of the three types of detection results.

[0029] The second step is to extract and locate defect features. After defining the overall global coordinate system of the sample, when importing the multi-source detection results into the 3D imaging model, to ensure accurate defect location, the defects in the laser ultrasound, X-ray, and infrared thermography detection results need to be adjusted according to the actual modeling scale, and the detection image coordinates are converted into global coordinates. Because laser ultrasound has a small detection scale, the defect information is marked in the 3D model after determining its coordinates in the 3D model based on its relative position to the registration points. Labels and links are also added; clicking the links in the 3D model allows direct export of the laser ultrasound detection results. Since X-ray and infrared thermography images can directly acquire all surface information of the sample, after edge recognition and image segmentation, the entire image can be integrated into the global coordinate system of the 3D model.

[0030] The third step involves extracting, locating, and transforming the defect features detected by laser ultrasonic testing. Marker points are placed, precisely dividing the sample surface into multiple square regions. These marker points are used not only for spatial positioning but also to provide coordinate references for subsequent image stitching and result comparison. Laser ultrasonic testing is performed sequentially on each of these square regions, using a region-by-region scanning method to ensure comprehensive coverage of the entire target area. The positional information of the four corner points of each region can be mapped one-to-one with the global coordinate system through coordinate transformation, thereby accurately determining the spatial position of the scanned area within the overall sample. This method enables subsequent region-level defect analysis and further 3D reconstruction. Defect detection is performed using a laser ultrasonic testing system, and neural networks are used to identify the laser ultrasonic testing images, obtaining the coordinates of each defect relative to the image origin, and generating point clouds according to the global modeling resolution. After the point cloud positions are determined in the local coordinate system, coordinate system angle transformation is performed based on the positional relationship between the origin of the scanned area and the two nearest positioning points. Subsequently, the global position of the laser ultrasonic defect is determined based on the positional relationship between the detection image and the positioning points, and between the positioning points and the origin of the global coordinate system. In addition, laser ultrasound has a depth determination function, and the depth data is also imported into the point cloud file. The generated point cloud automatically has the spatial position coordinates of the global coordinate system.

[0031] The fourth step involves extracting, locating, and transforming the defect features obtained from ray detection. For the defect images acquired through ray detection, the original images are first preprocessed to improve the distinguishability of the defect area and the accuracy of subsequent contour extraction. Preprocessing operations include spatial and frequency domain filtering (such as median filtering, Gaussian filtering, and Fourier denoising) to suppress speckle noise and background interference during imaging. Then, adaptive threshold binarization is performed to enhance the contrast between the defect area and the background. Morphological operations (such as opening and closing operations) are then used to optimize the image structure, fill in contour breaks, and remove isolated noise points, ensuring that the defect area has complete and clear edge features. After image preprocessing, edge extraction is performed to obtain the contour boundaries of the defect area. The extracted defect contour points can be converted into a set of coordinates with spatial attributes and further imported into the structural digital geometric model. By setting a unified three-dimensional coordinate system, the image coordinates and the geometric model coordinates are spatially aligned and mapped, thereby realizing the three-dimensional projection and annotation of the defect area on or inside the model surface. This step not only restores the spatial location of defects in the actual structure but also enables subsequent quantitative analysis and visualization. For typical defect areas identified in the image, image preprocessing, binarization, and edge extraction were performed sequentially. Finally, the defect locations were accurately marked in the corresponding digital 3D geometric model, thus achieving accurate reconstruction of defects from 2D projection images to 3D structural space.

[0032] The fifth step involves extracting, locating, and transforming the defect features detected by infrared thermal imaging. For defects obtained from infrared thermal imaging, an edge extraction algorithm is used to extract the defects while maintaining the matrix scale. The detected layered defects are saved to separate point cloud files. Based on the pre-obtained temperature change characteristics of different depth layers under excitation, the depth position of the detected layers is determined. Since infrared thermal imaging can acquire temperature data of the entire material surface, an edge extraction algorithm can be used to obtain the temperature changes of the entire surface. The distorted matrix is ​​adjusted to a standard matrix through image segmentation and transformation. A neural network is used to identify planar defects and determine the defect range. The identified region is segmented and extracted, and then converted into a point cloud. The defect point cloud is imported into the constructed 3D model.

[0033] like Figure 2 As shown, in the results integration stage, a 3D mesh model is constructed based on the actual size of the tested part and the detection resolution. Image registration is performed by matching feature points from two images and solving the image transformation matrix based on the coordinate transformation of the successfully matched feature point pairs, thus completing the image registration. This accurately maps various defect point clouds to the corresponding positions in the 3D model and supports interactive viewing and analysis.

[0034] Default matched image I x Image I to be matched y Both are grayscale images, and their pixel coordinates are I... x (i, j) and I y If (i,j), then the correspondence between them after image registration is as follows:

[0035] In the formula, h(i,j) is the coordinate transformation function of the spatial position of each pixel in the image.

[0036] Let the coordinates of the pixel to be matched and the coordinates of the pixel in the image to be matched be (X,Y) and (x,y), respectively. A projective transformation is used to perform the coordinate transformation. The projective transformation is a composite transformation that includes both rigid body transformation and affine transformation. Its coordinate transformation formula is:

[0037] In the formula, k is a non-zero proportionality factor; In the formula, a is the scaling factor.

[0038] For special curved surface structures, a transformation from Cartesian coordinates to cylindrical coordinates is required to ensure the accuracy of defect projection. The point cloud defects obtained by the three methods are imported into a 3D model. After pre-mapping the sample to be tested, a 3D model of the curved surface sample is established using 3D modeling software. Based on the visual projection relationship between planar defects and curved surface defects, the obtained detection results are spatially transformed to obtain laser ultrasound / ray / infrared point cloud data of curved surface defects. After position calibration and coordinate transformation using the same method, 3D imaging of complex curved surface structures can be achieved.

[0039] Example 1 A typical application example of this invention is the detection of spacecraft heat protection structures. Through three-dimensional fusion imaging, the spatial distribution and interrelationships of various defects such as composite material delamination, aluminum alloy pores, and uneven heat protection layers can be displayed simultaneously.

[0040] Example of spacecraft heat protection structure testing: 1 Data Fusion Structure Model like Figure 3 As shown, the object to be tested is a three-layer structure consisting of aluminum alloy, adhesive surface, and heat protection structure. Infrared thermal waves are used to detect debonding of the composite material / aluminum alloy adhesive surface, laser ultrasound is used to detect delamination and folding defects in the aluminum alloy, and X-rays are used to detect inhomogeneity within the heat protection structure.

[0041] 2. Three-dimensional modeling scheme 2.1 Establishment of 3D Model of Sample Create a three-dimensional structure, as shown in the schematic diagram. Figure 4 As shown.

[0042] 2.2 Defect Feature Extraction and Localization After defining the global coordinate system of the entire sample, the multi-source detection results are imported into the three-dimensional imaging model. Defects in the laser ultrasound, X-ray and infrared thermography detection results are adjusted according to the actual modeling ratio, and the detection image coordinates are converted into global coordinates.

[0043] 2.2.1 Extraction, localization, and coordinate transformation of defect features in laser ultrasonic testing Marking points were placed, and the sample surface was precisely divided into multiple 10mm × 10mm square areas. Laser ultrasonic testing was performed sequentially on each of these square areas, using a region-by-region scanning method to ensure complete and thorough coverage of the entire target area. Figure 5 As shown.

[0044] Defect detection was performed using a laser ultrasonic testing system with a scanning range of 8mm × 8mm and a scanning step size of 0.02mm. The detection results are as follows: Figure 6 As shown.

[0045] A neural network is used to identify laser ultrasonic inspection images, obtain the coordinates of each defect relative to the image origin, and generate point clouds according to the global modeling resolution. The overall modeling process is as follows: Figure 7 As shown.

[0046] 2.2.2 Extraction, localization, and coordinate transformation of defect features in X-ray inspection For the three typical defect regions identified in the X-ray inspection images, image preprocessing, binarization, and edge extraction were performed sequentially. Finally, the defect locations were accurately marked in the corresponding digital 3D geometric models, such as... Figure 8 As shown.

[0047] 2.2.3 Extraction, localization, and coordinate transformation of defect features detected by infrared thermography After detecting the debonding defect between the aluminum alloy and the heat-resistant material, the distorted matrix was adjusted to the desired state through image segmentation and transformation. Figure 9 The two-dimensional detection image shown.

[0048] Using neural networks to identify planar defects and determine their extent. For example... Figure 10 As shown.

[0049] The identified region is segmented and extracted, and then converted into a point cloud. The resulting image after point cloudification of the first layered defect is shown below. Figure 11 As shown.

[0050] The effect of importing the first debonding defect point cloud into the constructed 3D model is as follows: Figure 12 As shown.

[0051] Import all three defect point clouds in the above order to obtain the 3D reconstruction effect, as shown below. Figure 13 As shown.

[0052] 2.3 Integration of 3D Reconstruction Results The point cloud-based simulation defects of aluminum alloy flat-bottom holes, aluminum alloy debonding defects, and heat-resistant structure defects were imported into the 3D model to obtain the reconstruction results, as shown below. Figure 14 As shown.

[0053] The above description is only the best specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the protection scope of the present invention.

[0054] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A three-dimensional fusion imaging method for laser, ultrasound, X-ray, and infrared data, characterized in that: Includes the following steps: (1) Perform infrared thermography, laser ultrasonic testing and X-ray testing on the object to be tested to obtain defect information respectively; (2) Establish a three-dimensional model of the object to be detected, set marker points on the three-dimensional model, and divide the object to be detected into multiple regions according to the marker points; (3) Extract, locate and transform the defect information obtained from infrared thermal imaging, laser ultrasonic detection and X-ray detection in step (1) respectively to obtain defect point cloud maps; (4) The defect point cloud maps obtained in step (3) by infrared thermal imaging, laser ultrasonic detection and X-ray detection are image registered with the three-dimensional model obtained in step (2) respectively, and the defect point cloud is mapped to the corresponding position of the three-dimensional model.

2. The three-dimensional fusion imaging method for laser ultrasonic X-ray infrared data according to claim 1, characterized in that: In step (2), the detection object is divided into multiple square regions by the marker points, and the position information of the four corner points of each region corresponds one-to-one with the global coordinate system of the three-dimensional model through coordinate transformation relationship.

3. The three-dimensional fusion imaging method for laser ultrasonic X-ray infrared data according to claim 1, characterized in that: In step (3), laser ultrasonic testing uses a neural network to perform image recognition, obtains the coordinates of the defect relative to the image origin, and generates a point cloud.

4. The three-dimensional fusion imaging method for laser ultrasonic X-ray infrared data according to claim 1, characterized in that: After dividing the 3D model into multiple regions, laser ultrasonic testing is performed. The laser ultrasonic testing adopts a region-by-region scanning method, and each region is scanned sequentially. After the testing is completed, a point cloud map of each region is generated. The coordinate system angle is transformed by the positional relationship between the origin of each region and the two nearest positioning points to obtain the global location of the ultrasonic defect, thus obtaining the defect point cloud map.

5. The three-dimensional fusion imaging method for laser, ultrasonic, and infrared data according to claim 1, characterized in that: In step (3), the defect image obtained by ray detection is preprocessed. The preprocessing method is as follows: first, spatial domain and frequency domain filtering is performed, then adaptive threshold binarization is performed, and then the image structure is optimized by morphological operations to fill in contour breaks and remove isolated noise points.

6. The three-dimensional fusion imaging method for laser, ultrasonic, and infrared data according to claim 5, characterized in that: After the preprocessing is completed, edge extraction is performed to obtain the contour boundary of the defect area. Defect contour points are extracted and converted into a coordinate set with spatial attributes. The coordinate set is spatially aligned and mapped with the coordinates of the three-dimensional model to realize the three-dimensional projection and annotation of the defect area on the surface or inside the model.

7. The three-dimensional fusion imaging method for laser, ultrasonic, and infrared data according to claim 1, characterized in that: In step (3), defects detected by infrared thermal imaging are extracted by edge extraction algorithm, and different layered defects are saved to separate point cloud files. The depth position of the detected layer is determined according to the temperature change characteristics of different depth layers under excitation. The temperature change of the surface of the detected object is obtained by edge extraction algorithm. The distorted matrix is ​​adjusted to a standard matrix by image segmentation and transformation. The planar defects are identified and the defect range is determined by neural network. Then, point cloudification is performed.

8. The three-dimensional fusion imaging method for laser ultrasonic X-ray infrared data according to claim 1, characterized in that: In step (4), feature points of the two images are detected for matching. The image transformation matrix is ​​solved based on the coordinate transformation of the successfully matched feature point pairs to complete image registration. If the matched image I x Image I to be matched y The pixel coordinates are I x (i, j) and I y If (i,j), then the correspondence after image registration is as follows: Where h(i,j) is the coordinate transformation function of the spatial position of each pixel in the image.

9. A three-dimensional fusion imaging method for laser, ultrasonic, and infrared data according to claim 8, characterized in that: Using projective transformation for coordinate transformation, the coordinates of the pixels in the image being matched and the pixels in the image to be matched are (X,Y) and (x,y) respectively. The coordinate transformation formula is: Where k is a non-zero scaling factor and a is a scaling factor.

10. A three-dimensional fusion imaging method for laser ultrasonic X-ray infrared data according to claim 1, characterized in that: If the object to be inspected is a curved structure, a transformation from Cartesian coordinates to cylindrical coordinates is performed, and the defect information from infrared thermography, laser ultrasonic testing, and X-ray inspection is spatially transformed based on the visual projection relationship between planar defects and curved defects.

Images