A large-scale composite material detection method based on infrared and ultrasonic data fusion

Abstract

The present application provides a large-scale composite material detection method based on infrared and ultrasonic data fusion, which utilizes the infrared large-scale rapid advantage to rapidly locate the impact marks on the surface of the composite material, cooperates with ultrasonic positioning detection to obtain detailed data of the impact defects, and finally realizes rapid positioning and accurate detection of the defects of the large-scale composite material structure.

Description

Technical Field

[0001] This invention relates to the fields of aerospace vehicle inspection, nondestructive testing, and data fusion technology, and in particular to a large-scale composite material inspection method based on infrared and ultrasonic data fusion. Background Technology

[0002] Data fusion inspection technology is a comprehensive inspection technique that combines multiple inspection methods for a unified target. It leverages the advantages of different inspection methods to obtain and integrate multifaceted information about the target, providing operators with more comprehensive parameters for decision-making. It primarily targets complex structures and multi-type composite materials, addressing the limitations of single-method inspections for all structural types. Currently, in the aerospace field, the inspection targets are mostly composite material structures, posing significant challenges to infrared, ultrasonic, and X-ray inspection methods.

[0003] With the deepening research on composite materials, they are widely used in the main structures of various aerospace vehicles in the aerospace field, effectively improving the overall performance of the structures. Currently, the main problems with composite material main structures for aerospace vehicles are as follows: 1. With the optimization of aerodynamic shapes, the outer surfaces of aerospace vehicles are smoother, making it difficult to locate defects during online inspection; 2. Composite material surfaces are usually covered with various heat-resistant and microwave-absorbing coatings, and the inner layers are connected to various heat-insulating and reinforcing structures, making it impossible to effectively inspect all material structures using a single inspection method; 3. Due to the unique application environment of aerospace equipment, all inspection work needs to be completed online, and in addition to the requirement for inspection speed, there is also an urgent need for rapid defect location.

[0004] In existing research, He Weifeng et al. from the Air Force Engineering University of the Chinese People's Liberation Army (patent number CN202111267118.7) proposed a method and system for detecting defects in composite materials based on the fusion of infrared and ultrasonic signals. The method and system include the following steps: collecting a dataset containing infrared and ultrasonic signals of the composite material; dividing the dataset into a training dataset and a validation dataset; constructing a deep learning-based signal feature learning and fusion classification model; inputting the training dataset into the trained model; and inputting the validation dataset into the trained model to obtain the composite material defect detection result. This method effectively solves the problems of ultrasonic detection, where defect type judgment is greatly affected by human factors and defect characterization is difficult, as well as the problems of low accuracy in defect type classification and inability to accurately reflect defect location in infrared thermal imaging detection. It achieves objective judgment of the type and location of composite material defects and improves the accuracy of defect type classification. However, this method uses detection data for database modeling, requiring a large accumulation of detection data, resulting in an overly complex algorithm. Furthermore, it can only detect material types contained in the training set and cannot meet the needs of online inspection of aerospace vehicles.

[0005] Gao Yuan et al. (patent number CN202210669263.6) of Tieling Power Supply Company, Liaoning Electric Power Co., Ltd., disclosed a method for joint localization of partial discharge sources in primary equipment of substations by fusing ultrasound and infrared. This method is more time-saving and labor-saving. By fusing image and signal spectra, this method can reduce the influence of adverse factors, improve the recognition of target sources, and thus ensure accurate localization of partial discharge sources. It has high detection efficiency and strong defect depth detection capability. It avoids the interference of subjective judgment factors, is more accurate than traditional detection methods, and the detection process is safer. The steps include: Step 1: Acquiring infrared images, ultrasonic spectra, and phase maps of primary equipment; Step 2: Preprocessing the infrared images and ultrasonic signal spectra of primary equipment acquired in Step 1; Step 3: Fusing the infrared images and ultrasonic signal spectra using a fusion rule based on maximum selection; Step 4: Achieving joint localization of partial discharge sources based on the fused infrared and ultrasonic images. This method is only suitable for defect detection of small components, and the scale of infrared and ultrasonic defect feature signals is similar. When applied to large-scale structures, it cannot achieve a high fusion effect when there is insufficient surface defect information that can participate in registration.

[0006] Fan Limei et al. (patent number CN202311528751.6) of the Shandong Institute of Non-metallic Materials disclosed a multi-source data fusion detection method for defects in stealth coating materials. The method includes: registering the positions of thermal wave grayscale images and ultrasonic grayscale images to obtain the maximum and minimum grayscale values ​​within the detection window of the thermal wave and ultrasonic grayscale images; calculating the absolute value of the difference between the grayscale values ​​of the center pixel and the remaining pixels within the window, extracting feature parameters and arranging them in order; substituting the extracted feature parameters into a basic confidence allocation function based on smoothness processing to obtain the ultrasonic basic confidence matrix; calculating the basic confidence matrix of the fused image using evidence theory; setting a decision threshold for defects in the stealth coating material to obtain a fused defect edge image of the stealth coating material after fusing the infrared and ultrasonic images. This method only obtains defect edge information and does not effectively utilize the high localization characteristics of ultrasound to determine the depth of the defect. Summary of the Invention

[0007] The purpose of this invention is to address the problems in existing technologies by proposing a large-scale composite material inspection method based on the fusion of infrared and ultrasonic data. The method proposes solutions for data fusion inspection of large-scale composite material structures in the aerospace field, where there are no obvious reference markers on the material surface; and for non-destructive testing of impact defects in large-scale composite material structures in the aerospace field, it proposes solutions based on the fusion of infrared and ultrasonic data.

[0008] This invention is achieved through the following technical solution: This invention proposes a large-scale composite material detection method based on the fusion of infrared and ultrasonic data, the method comprising the following steps:

[0009] Step 1: Obtain the three-dimensional coordinates of the detection system and the structure under test, and establish a three-dimensional coordinate system X with the base of the detection system as the origin;

[0010] Step 2: Use the infrared thermal wave module of the detection system to perform large-area imaging detection on the structure under test, record the position and orientation of the infrared camera in the X coordinate system, and determine the three-dimensional coordinates of the measured area corresponding to the field of view of the infrared camera based on the relative relationship between the infrared camera and the object under test in the same coordinate system.

[0011] Step 3: Obtain the infrared large-area imaging results, acquire the centroid coordinates of the defect, adjust the position of the infrared camera, approach the defect to perform small-area imaging detection, record the position and orientation of the infrared camera at this time, and determine the three-dimensional coordinates of the measured area corresponding to the small-area imaging field of view of the infrared camera.

[0012] Step 4: The infrared detection module returns to its initial position, controls the ultrasonic detection module to approach the defect location, performs a small-area scan centered on the defect location coordinates, obtains the ultrasonic detection results, and performs image stitching and positioning based on the built-in encoder of the ultrasonic probe module and the ultrasonic probe coordinates.

[0013] Step 5: Use Gaussian pyramid to extract feature vectors from the infrared and ultrasonic detection results. Use the feature vectors extracted from the infrared and ultrasonic defects to perform registration and image correction, and unify the image distortion of the infrared and ultrasonic detection results.

[0014] Step 6: Obtain and discretize the defect features in the obtained infrared and ultrasonic detection images, and calculate the defect depth based on the sound path information of the ultrasonic detection results;

[0015] Step 7: Determine the defect coordinates based on the positions of the infrared camera and ultrasonic probe in the three-dimensional coordinate system X. Project the obtained infrared and ultrasonic point cloud results onto the established three-dimensional model of the structure under test according to the obtained coordinates, and add defect labels to indicate the defect information.

[0016] This invention also proposes a large-scale composite material detection method based on the fusion of infrared and ultrasonic data, the method comprising the following steps:

[0017] Step 1: Establish the world coordinate system X of the testing environment with the fixed testing equipment base as the origin;

[0018] Step 2: Place the structure to be tested at a certain distance in front of the testing equipment to ensure that the testing equipment can control the ultrasonic testing module to approach the surface of the structure to be tested;

[0019] Step 3: Measure the distance between the testing equipment and the structure under test, calculate the precise distance between the equipment base and at least four points on the surface of the structure under test, and establish a three-dimensional model of the structure under test in the world coordinate system X;

[0020] Step 4: Based on the connection relationship between the equipment base, mechanical structure, and infrared camera, determine the position S of the infrared camera in the world coordinate system X. i1 With orientation I1, the infrared detection module is excited to the surface of the structure under test. The infrared camera records data and performs calculations. At this time, the position coordinates S of the material under test corresponding to the center of the infrared camera's field of view are determined in the established 3D model according to the orientation I1 of the infrared camera. n1 ; obtain with S n1 Infrared detection data centered on D i1 ;

[0021] Step 5: Obtain the defect coordinates X1 from the infrared detection results, and control the infrared detection module and the ultrasonic detection module to perform close-range detection on the area centered at X1; record the camera position S. i2 Camera orientation I2 and detection data Di2 Set the ultrasound scanning range R u1 The ultrasonic testing module controls the detection of a device with a side length of 2×R centered at X1. u1 A rectangular area was scanned to obtain ultrasound detection data D. u ;

[0022] Step Six: Process the obtained detection data D separately. i2 With D u Gaussian pyramids were used for feature extraction to obtain feature images of the defect locations detected by infrared and ultrasonic methods. Typical feature vectors generated by impact crack traces were used to register the two types of defects to ensure the accuracy of data fusion.

[0023] Step 7: After fusing the infrared and ultrasonic data according to the registration rules, load the point cloud fusion result into the built 3D model according to coordinate X1, and add the depth data and distribution obtained from the ultrasonic detection to the label of the model point cloud.

[0024] The beneficial effects of this invention are:

[0025] This invention proposes a large-scale composite material detection method based on the fusion of infrared and ultrasonic data. This method utilizes the rapid large-scale advantage of infrared technology to quickly locate impact marks on the surface of composite materials, and combines this with ultrasonic localization detection to obtain detailed data on impact defects, ultimately achieving rapid localization and accurate detection of defects in large-scale composite material structures. Attached Figure Description

[0026] Figure 1 This is a flowchart of a large-scale composite material detection method based on the fusion of infrared and ultrasonic data, as described in this invention.

[0027] Figure 2 This is a schematic diagram of the registration result based on the extraction of image feature vectors from the Gaussian pyramid. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Data fusion: Data fusion is the process of combining, correlating, and integrating data and information from multiple sensor sources to obtain more accurate location and identity estimations, thereby enabling real-time and comprehensive evaluation of the battlefield situation, threats, and their importance.

[0030] Infrared nondestructive testing: Infrared nondestructive testing is a non-contact testing technique that utilizes the characteristics of infrared radiation. It can assess internal defects in materials and components, such as cracks. Based on the relationship between the thermodynamic properties of an object and infrared radiation, it assesses the health status and potential defects of materials and structures by measuring and analyzing changes in infrared radiation on the object's surface.

[0031] Ultrasonic nondestructive testing: a technique that studies the reflected, transmitted, and scattered waves by interacting with ultrasonic waves, and performs macroscopic defect detection, geometric property measurement, detection and characterization of changes in microstructure and mechanical properties of the specimen, and further evaluates its specific application.

[0032] Probe: Typically consists of one or more transducers, used to transmit or receive ultrasonic waves, or both.

[0033] This invention proposes a large-scale composite material detection method based on the fusion of infrared and ultrasonic data. First, leveraging the advantages of infrared detection's large area and high speed, thermal imaging is performed on a large surface area of ​​the structure under test to obtain and locate shallow surface impact and crack defect signals. Then, utilizing the advantages of ultrasonic detection's high accuracy and good localization, close-range detection is performed based on provided coordinates, while simultaneously adjusting the infrared detection field of view to perform secondary refinement acquisition around the localization point. Next, Gaussian pyramids are used to extract the detected defect features, obtaining feature vectors, which are then used for image registration. Finally, based on the registration results, the defect distribution cloud maps obtained from infrared and ultrasonic detection, along with the localization depth, are projected into a three-dimensional twin structure.

[0034] For details, please refer to Figures 1-2 This invention proposes a large-scale composite material detection method based on the fusion of infrared and ultrasonic data, the method comprising the following steps:

[0035] Step 1: Obtain the three-dimensional coordinates of the detection system and the structure under test, and establish a three-dimensional coordinate system X with the base of the detection system as the origin;

[0036] Step 2: Use the infrared thermal wave module of the detection system to perform large-area imaging detection on the structure under test, record the position and orientation of the infrared camera in the X coordinate system, and determine the three-dimensional coordinates of the measured area corresponding to the field of view of the infrared camera based on the relative relationship between the infrared camera and the object under test in the same coordinate system.

[0037] Step 3: Obtain the infrared large-area imaging results, acquire the centroid coordinates of the defect, adjust the position of the infrared camera, approach the defect to perform small-area imaging detection, record the position and orientation of the infrared camera at this time, and determine the three-dimensional coordinates of the measured area corresponding to the small-area imaging field of view of the infrared camera.

[0038] Step 4: The infrared detection module returns to its initial position, controls the ultrasonic detection module to approach the defect location, performs a small-area scan centered on the defect location coordinates, obtains the ultrasonic detection results, and performs image stitching and positioning based on the built-in encoder of the ultrasonic probe module and the ultrasonic probe coordinates.

[0039] Step 5: Use Gaussian pyramid to extract feature vectors from the infrared and ultrasonic detection results. Use the feature vectors extracted from the infrared and ultrasonic defects to perform registration and image correction, and unify the image distortion of the infrared and ultrasonic detection results.

[0040] Step 6: Obtain and discretize the defect features in the infrared and ultrasonic detection images, and calculate the defect depth based on the acoustic path information of the ultrasonic detection results;

[0041] Step 7: Determine the defect coordinates based on the positions of the infrared camera and ultrasonic probe in the three-dimensional coordinate system X. Project the obtained infrared and ultrasonic point cloud results onto the established three-dimensional model of the structure under test according to the obtained coordinates, and add defect labels to indicate the defect information.

[0042] This invention also proposes a large-scale composite material detection method based on the fusion of infrared and ultrasonic data, the method comprising the following steps:

[0043] Step 1: Establish the world coordinate system X of the testing environment with the fixed base of the testing equipment as the origin;

[0044] Step 2: Place the structure to be tested at a certain distance in front of the testing equipment to ensure that the testing equipment can control the ultrasonic testing module to approach the surface of the structure to be tested;

[0045] Step 3: Measure the distance between the testing equipment and the structure under test, calculate the precise distance between the equipment base and at least four points on the surface of the structure under test, and establish a three-dimensional model of the structure under test in the world coordinate system X;

[0046] Step 4: Based on the connection relationship between the equipment base, mechanical structure, and infrared camera, determine the position S of the infrared camera in the world coordinate system X. i1 With orientation I1, the infrared detection module is excited to the surface of the structure under test. The infrared camera records data and performs calculations. At this time, the position coordinates S of the material under test corresponding to the center of the infrared camera's field of view are determined in the established 3D model according to the orientation I1 of the infrared camera. n1 ; obtain with S n1 Infrared detection data centered on D i1 ;

[0047] Step 5: Obtain the defect coordinates X1 from the infrared detection results, and control the infrared detection module and the ultrasonic detection module to perform close-range detection on the area centered at X1; record the camera position S. i2 Camera orientation I2 and detection data Di2 Set the ultrasound scanning range R u1 The ultrasonic testing module controls the detection of a device with a side length of 2×R centered at X1. u1 A rectangular area was scanned to obtain ultrasound detection data D. u ;

[0048] Step Six: Process the obtained detection data D separately. i2 With D u Gaussian pyramids were used for feature extraction to obtain feature images of the defect locations detected by infrared and ultrasonic methods. Typical feature vectors generated by impact crack traces were used to register the two types of defects to ensure the accuracy of data fusion.

[0049] Step 7: After fusing the infrared and ultrasonic data according to the registration rules, load the point cloud fusion result into the built 3D model according to coordinate X1, and add the depth data and distribution obtained from the ultrasonic detection to the label of the model point cloud.

[0050] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Anyone skilled in the art can make various modifications and alterations without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be determined by the claims.

Claims

1. A large-scale composite material detection method based on the fusion of infrared and ultrasonic data, characterized in that, The method Includes the following steps: Step 1: Obtain the three-dimensional coordinates of the detection system and the structure under test, and establish a three-dimensional coordinate system X with the base of the detection system as the origin; Step 2: Use the infrared thermal wave module of the detection system to perform large-area imaging detection on the structure under test, record the position and orientation of the infrared camera in the X coordinate system, and determine the three-dimensional coordinates of the measured area corresponding to the field of view of the infrared camera based on the relative relationship between the infrared camera and the object under test in the same coordinate system. Step 3: Obtain the infrared large-area imaging results, acquire the centroid coordinates of the defect, adjust the position of the infrared camera, approach the defect to perform small-area imaging detection, record the position and orientation of the infrared camera at this time, and determine the three-dimensional coordinates of the measured area corresponding to the small-area imaging field of view of the infrared camera. Step 4: The infrared detection module returns to its initial position, controls the ultrasonic detection module to approach the defect location, performs a small-area scan centered on the defect location coordinates, obtains the ultrasonic detection results, and performs image stitching and positioning based on the built-in encoder of the ultrasonic probe module and the ultrasonic probe coordinates. Step 5: Use Gaussian pyramid to extract feature vectors from the infrared and ultrasonic detection results. Use the feature vectors extracted from the infrared and ultrasonic defects to perform registration and image correction, and unify the image distortion of the infrared and ultrasonic detection results. Step 6: Obtain and discretize the defect features in the infrared and ultrasonic detection images, and calculate the defect depth based on the acoustic path information of the ultrasonic detection results; Step 7: Determine the defect coordinates based on the positions of the infrared camera and ultrasonic probe in the three-dimensional coordinate system X. Project the obtained infrared and ultrasonic point cloud results onto the established three-dimensional model of the structure under test according to the obtained coordinates, and add defect labels to indicate the defect information. Utilize the large-scale and rapid advantages of infrared technology to quickly locate impact marks on the surface of the composite material, and combine it with ultrasonic positioning detection to obtain detailed data on impact defects, ultimately achieving rapid location and accurate detection of defects in large-scale composite material structures.

2. A large-scale composite material detection method based on the fusion of infrared and ultrasonic data, characterized in that, The method includes the following steps: Step 1: Establish the world coordinate system X of the testing environment with the fixed base of the testing equipment as the origin; Step 2: Place the structure to be tested at a certain distance in front of the testing equipment to ensure that the testing equipment can control the ultrasonic testing module to approach the surface of the structure to be tested; Step 3: Measure the distance between the testing equipment and the structure under test, calculate the precise distance between the equipment base and at least four points on the surface of the structure under test, and establish a three-dimensional model of the structure under test in the world coordinate system X; Step 4: Based on the connection relationship between the equipment base, mechanical structure, and infrared camera, determine the position of the infrared camera in the world coordinate system X. Orientation The infrared detection module is controlled to excite the surface of the structure under test, and an infrared camera is used to record and calculate data. At this time, the data is then incorporated into the established 3D model based on the orientation of the infrared camera. Determine the position coordinates of the material under test at the center of the infrared camera's field of view. ; obtain with Infrared detection data centered on ; Step 5: Obtain the defect coordinates from the infrared detection results The infrared detection module and the ultrasonic detection module are controlled respectively to detect the infrared detection module and the ultrasonic detection module. Close-range detection is performed on the central area; camera positions are recorded. Camera orientation and test data Set the ultrasonic scanning range Control the ultrasonic testing module to perform ultrasound detection on the The side length of the center is 2× A rectangular area was scanned to obtain ultrasound detection data. ; Step Six: Process the obtained detection data separately. and Gaussian pyramids were used for feature extraction to obtain feature images of the defect locations detected by infrared and ultrasonic methods. Typical feature vectors generated by impact crack traces were used to register the two types of defects to ensure the accuracy of data fusion. Step 7: After fusing the infrared and ultrasonic data according to the registration rules, the point cloud fusion result is then processed according to coordinates. add The data is loaded into the existing 3D model and added to the model's point cloud based on the depth data and distribution obtained from ultrasonic testing. The label describes how infrared technology can be used to quickly locate impact marks on the surface of composite materials, and how ultrasonic positioning detection can be used to obtain detailed data on impact defects, ultimately achieving rapid location and accurate detection of defects in large-scale composite material structures.

Images