Large-scale composite material detection method based on infrared and ultrasonic data fusion

Abstract

According to the large-scale composite material detection method based on infrared and ultrasonic data fusion provided by the invention, the impact trace on the surface of the composite material is rapidly positioned by using the rapid advantage of infrared large scale, and the detailed data of impact defects are obtained by cooperating with ultrasonic positioning detection; and finally, rapid defect positioning and accurate detection of the large-scale composite material structure are realized.

Description

Technical Field

[0001] The present invention relates to the fields of aerospace vehicle detection, nondestructive testing, and data fusion technology, and particularly relates to a large-scale composite material detection method based on infrared and ultrasonic data fusion. Background Art

[0002] The data fusion detection technology is a comprehensive detection technology that combines multiple detection methods for a unified target. By taking advantage of the strengths of different detection means, it obtains multi-faceted information of the target and integrates it to obtain more comprehensive parameters of the target for the operator to make decisions. It mainly targets the current complex structures and multi-type composite materials to solve the need that a single means cannot effectively handle the detection of all types of structures. Currently, in the aerospace field, the detection objects are mainly composite material-based comprehensive structures, which pose great challenges to detection methods such as infrared, ultrasonic, and ray detection.

[0003] With the in-depth research on composite materials, in the aerospace field, composite materials are widely used in the main structures of various aerospace vehicles, effectively improving the comprehensive performance of the structures. Currently, there are mainly the following problems with the composite material main structures of aerospace vehicles: 1. With the optimization of the aerodynamic shape, the outer surface of aerospace vehicles is smoother, and it is difficult to locate the defect positions during the on-line detection process; 2. The surface of composite materials is usually covered with various heat protection and wave absorption coatings, and the inner layer is connected to various heat insulation and strengthening structures. A single detection means cannot effectively detect all material structures; 3. Due to its unique application environment, aerospace equipment needs to complete all detection work on-line. Based on the requirement of detection speed, there is also an urgent need for rapid defect location.

[0004] In existing research, He Weifeng et al. from the Air Force Engineering University of the Chinese People's Liberation Army (Patent No. CN202111267118.7) proposed a composite material defect detection method and system based on the fusion of infrared and ultrasonic signals. The steps of a composite material defect detection method and system based on the fusion of infrared and ultrasonic signals include: collecting a data set containing the infrared signals and ultrasonic signals of the composite material, and dividing the data set into a training data set and a validation data set; constructing a signal feature learning and fusion classification model based on deep learning, and inputting the training data set into the signal feature learning and fusion classification model for training; inputting the validation data set into the trained signal feature learning and fusion classification model to obtain the composite material defect detection result. It effectively solves the problems that the judgment of defect types in ultrasonic detection is greatly affected by human factors and it is difficult to qualitatively determine defects, as well as the problems that the classification accuracy of defect types in infrared thermal imaging detection is not high and the defect location cannot be well reflected, and realizes an objective judgment of the composite material defect type and location, improving the classification accuracy of defect types. This method uses detection data for database modeling, requires a large amount of detection data accumulation, the algorithm is too complex, and it can only detect the material types included in the training set and cannot meet the needs of on-line detection of aerospace vehicles.

[0005] Gao Yuan et al. from Tieling Power Supply Company of Liaoning Electric Power Co., Ltd. (Patent No. CN202210669263.6) disclosed a method for joint localization of partial discharge sources of primary substation equipment by fusing ultrasonic and infrared. This method is more time-saving and labor-saving. By using this method to fuse images and signal spectrograms, it can reduce the influence of adverse factors, improve the recognition degree of the target source, and then ensure the accurate localization of the partial discharge source, with high detection efficiency and strong defect depth detection ability. It avoids the interference of subjective judgment factors, is more accurate than traditional detection methods, and the detection process is safer. The steps include: Step 1: Obtaining the infrared image, ultrasonic spectrogram and phase diagram of the primary equipment; Step 2: Preprocessing the infrared image and ultrasonic signal spectrogram of the primary equipment obtained in Step 1; Step 3: Fusing the infrared image and ultrasonic signal spectrogram using the fusion rule based on maximum selection; Step 4: Realizing the joint localization of the partial discharge source based on the fused infrared and ultrasonic image. This method is only for the defect detection of small components, and the scales of the infrared and ultrasonic defect feature signals are similar. When applied to large-scale structures, a high fusion effect cannot be achieved in the case of insufficient defect information available for registration on the surface.

[0006] Fan Limei et al. from Shandong Institute of Non-Metallic Materials (Patent No. CN202311528751.6) disclosed a multi-source data fusion detection method for defects in stealth coating materials, including: registering the positions of the thermal wave grayscale image and the ultrasonic grayscale image to obtain the maximum and minimum grayscale values within the detection windows of the thermal wave grayscale image and the ultrasonic grayscale image; calculating the absolute values of the differences between the grayscale values of the central pixel and the remaining pixels within the window, extracting and arranging the obtained characteristic parameters in sequence; substituting the extracted characteristic parameters into the basic belief assignment function based on smoothness processing to obtain the ultrasonic basic belief matrix; calculating using the evidence theory to obtain the fused image basic belief matrix; setting the decision threshold for the defects in the stealth coating materials to obtain the fused defect edge image of the stealth coating materials after fusing the infrared image and the ultrasonic image. This method can only obtain the defect edge information and does not make good use of the high positioning characteristics of ultrasonic waves to achieve the depth determination of defects. Summary of the Invention

[0007] The object of the present invention is to solve the problems in the prior art and propose a large-scale composite material detection method based on the fusion of infrared and ultrasonic data. The method respectively proposes solutions in the direction of data fusion detection for large-scale composite material structures in the aerospace field where there are no obvious reference marking points on the material surface, and proposes solutions in the direction of non-destructive detection of the fusion of infrared and ultrasonic waves for impact defects in composite materials for large-scale composite material structures in the aerospace field.

[0008] The present invention is realized through the following technical solutions. The present invention proposes a large-scale composite material detection method based on the fusion of infrared and ultrasonic data, and the method includes the following steps:

[0009] Step 1: Obtain the three-dimensional coordinates of the detection system and the structure to be measured, and establish a three-dimensional coordinate system X with the base of the detection system as the origin;

[0010] Step 2: Use the infrared thermal wave module of the detection system to perform large-area imaging detection on the structure to be measured, record the position and direction of the infrared camera in the coordinate system X, and determine the three-dimensional coordinates of the measured area corresponding to the field of view of the infrared camera according to the relative relationship between the infrared camera and the object to be measured in the same coordinate system;

[0011] Step 3: Obtain the infrared large-area imaging result, obtain the centroid coordinates of the defect, adjust the position of the infrared camera, approach the defect for small-area imaging detection, record the position and direction of the infrared camera at this time, and determine the three-dimensional coordinates of the measured area corresponding to the small-area imaging field of view of the infrared camera;

[0012] Step 4: The infrared detection module returns to the initial position, controls the ultrasonic detection module to approach the defect position, performs a small-area scan centered on the defect position coordinates, obtains the ultrasonic detection result, and performs image stitching and positioning according to the built-in encoder of the ultrasonic probe module and the ultrasonic probe coordinates;

[0013] Step 5: Use the Gaussian pyramid to extract feature vectors from the infrared and ultrasonic detection results, perform registration and image correction using the feature vectors extracted from the infrared defects and ultrasonic detection defects, and unify the image distortion of the infrared and ultrasonic detection results;

[0014] Step 6: Obtain and discretize the defect features in the obtained infrared and ultrasonic detection images, and calculate the defect depth according to the sound path information of the ultrasonic detection result;

[0015] Step 7: Determine the defect coordinates according to the positions of the infrared camera and the ultrasonic probe in the three-dimensional coordinate system X, project the obtained infrared and ultrasonic result point clouds onto the established three-dimensional model of the measured structure according to the obtained coordinates, and add defect labels to indicate the defect information.

[0016] The present invention also proposes a large-scale composite material detection method based on infrared and ultrasonic data fusion, and the method includes the following steps:

[0017] Step 1: Taking the fixed detection equipment base as the origin, establish the world coordinate system X of the detection environment;

[0018] Step 2: Park the structure to be measured at a certain distance in front of the detection equipment, and ensure that the detection equipment can control the ultrasonic detection module to approach the surface of the structure to be measured;

[0019] Step 3: Measure the distance between the detection equipment and the measured structure, calculate the accurate distances between the equipment base and at least four points on the surface of the measured structure, and establish a three-dimensional model of the measured structure in the world coordinate system X;

[0020] Step 4: According to the connection relationship of the equipment base - mechanical structure - infrared camera, determine the position S of the infrared camera in the world coordinate system X i1 and the orientation I 1 , control the infrared detection module to excite the surface of the structure to be measured, use the infrared camera to record data and calculate. At this time, in the established three-dimensional model, according to the infrared camera orientation I 1 determine the position coordinates S of the center of the infrared camera field of view corresponding to the material to be measured n1 ; obtain the infrared detection data D centered on S n1 ; i1 ;

[0021] Step 5: Obtain the defect coordinates X in the infrared detection result 1 , respectively control the infrared detection module and the ultrasonic detection module to take X 1Perform proximity detection on the area centered on [X]; record the camera position S i2 , the camera orientation I 2 , and the detection data D i2 ; Set the ultrasonic scanning range R u1 , and control the ultrasonic detection module to scan a rectangular area with a side length of 2×R centered on X 1 to obtain ultrasonic detection data D u1 ; u ;

[0022] Step Six: Respectively perform feature extraction on the obtained detection data D i2 and D u using the Gaussian pyramid to obtain the feature images of the infrared and ultrasonic detection defect positions, and register the two types of defects using the typical feature vectors generated by the impact crack traces to ensure the accuracy of data fusion;

[0023] Step Seven: After fusing the infrared and ultrasonic data according to the registration rules, load the point-clouded fusion result into the pre-built three-dimensional model according to the coordinate X 1 and add it to the label of the model point cloud according to the depth data and distribution obtained by ultrasonic detection.

[0024] The beneficial effects of the present invention are:

[0025] The present invention proposes a large-scale composite material detection method based on the fusion of infrared and ultrasonic data. This method uses the rapid large-scale advantage of infrared to quickly locate the impact traces on the surface of the composite material, and cooperates with ultrasonic positioning detection to obtain detailed data on impact defects, ultimately realizing the rapid positioning and accurate detection of defects in the large-scale composite material structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of the large-scale composite material detection method based on the fusion of infrared and ultrasonic data described in the present invention.

[0027] Figure 2 is a schematic diagram of the registration result of extracting the image feature vector based on the Gaussian pyramid. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0029] Data fusion: Data fusion is a process of combining, correlating, and integrating data and information from multiple sensor information sources to obtain more accurate position estimates and identity estimates, thereby achieving real-time and complete evaluation of the battlefield situation, threats, and their importance levels.

[0030] Infrared non-destructive testing: Infrared non-destructive testing is a non-contact testing technology that utilizes infrared radiation characteristics. It can evaluate defects inside materials and components, such as cracks. It is based on the relationship between the thermodynamic properties of objects and infrared radiation, and assesses the health status and potential defect problems of materials and structures by measuring and analyzing the changes in infrared radiation on the object surface.

[0031] Ultrasonic non-destructive testing: A technology that studies the reflected, transmitted, and scattered waves through the interaction between ultrasonic waves and specimens, conducts macroscopic defect detection, geometric property measurement, detection and characterization of changes in organizational structure and mechanical properties on specimens, and further evaluates their specific applications.

[0032] Probe: Usually composed of one or more transducers, it is an electro-acoustic conversion device used to transmit or receive, or both transmit and receive ultrasonic waves.

[0033] The present invention proposes a large-scale composite material detection method based on the fusion of infrared and ultrasonic data. First, utilize the advantages of large detection area and fast detection speed of infrared detection to perform thermal wave imaging on the large-scale surface of the structure to be measured, obtain the impact and crack defect signals in the shallow surface layer of the large-scale structure body and locate them; subsequently, utilize the advantages of high detection accuracy and good positioning of ultrasonic detection to perform close-range detection according to the provided coordinates, and at the same time adjust the infrared detection field of view to perform secondary refined acquisition on the periphery of the positioning point; then use the Gaussian pyramid to extract the defect features obtained from the detection to obtain feature vectors, and use the obtained vector values for image registration; finally, project the defect distribution cloud map obtained from infrared detection, the defect distribution cloud map obtained from ultrasonic detection, and the positioning depth into the three-dimensional twin structure body according to the registration result.

[0034] Specifically, referring to Figure 1 - Figure 2 The present invention proposes a large-scale composite material detection method based on the fusion of infrared and ultrasonic data, and the method includes the following steps:

[0035] Step 1: Obtain the three-dimensional coordinates of the detection system and the structure to be measured, and establish a three-dimensional coordinate system X with the base of the detection system as the origin.

[0036] Step 2: Use the infrared thermal wave module of the detection system to perform large-area imaging detection on the structure to be measured, record the position and direction of the infrared camera in the coordinate system X, and determine the three-dimensional coordinates of the detected area corresponding to the infrared camera's field of view according to the relative relationship between the infrared camera and the object to be measured in the same coordinate system.

[0037] Step 3: Obtain the infrared large-area imaging result, acquire the centroid coordinates of the defect, adjust the position of the infrared camera, approach the defect for small-area imaging detection, record the position and direction of the infrared camera at this time, and determine the three-dimensional coordinates of the area corresponding to the small-area imaging field of view of the infrared camera on the measured area;

[0038] Step 4: The infrared detection module returns to the initial position, control the ultrasonic detection module to approach the defect position, perform a small-area scan centered on the defect position coordinates, obtain the ultrasonic detection result, and perform image stitching and positioning according to the built-in encoder of the ultrasonic probe module and the ultrasonic probe coordinates;

[0039] Step 5: Use the Gaussian pyramid to extract feature vectors from the infrared and ultrasonic detection results, and use the feature vectors extracted from the infrared defects and ultrasonic detected defects for registration and image correction to unify the image distortion of the infrared and ultrasonic detection results;

[0040] Step 6: Acquire and discretize the defect features in the obtained infrared and ultrasonic detection images, and calculate the defect depth according to the sound path information of the ultrasonic detection result;

[0041] Step 7: Determine the defect coordinates according to the positions of the infrared camera and the ultrasonic probe in the three-dimensional coordinate system X, project the obtained infrared and ultrasonic result point clouds onto the established three-dimensional model of the measured structure according to the obtained coordinates, and add defect labels to indicate defect information.

[0042] The present invention also proposes a large-scale composite material detection method based on infrared and ultrasonic data fusion, and the method includes the following steps:

[0043] Step 1: Take the fixed detection equipment base as the origin to establish the world coordinate system X of the detection environment;

[0044] Step 2: Park the structure to be measured at a certain distance in front of the detection equipment to ensure that the detection equipment can control the ultrasonic detection module to approach the surface of the structure to be measured;

[0045] Step 3: Measure the distance between the detection equipment and the measured structure, calculate the accurate distances between the equipment base and at least four points on the surface of the measured structure, and establish a three-dimensional model of the measured structure in the world coordinate system X;

[0046] Step 4: According to the connection relationship of the equipment base - mechanical structure - infrared camera, determine the position S i1 and orientation I 1 of the infrared camera in the world coordinate system X, control the infrared detection module to excite the surface of the structure to be measured, use the infrared camera to record data and calculate. At this time, in the established three-dimensional model, according to the infrared camera orientation I 1 determine the position coordinates S n1 of the center of the infrared camera field of view corresponding to the material to be measured; obtain the position with S n1Infrared detection data D centered on i1 ;

[0047] Step Five: Obtain the defect coordinates X in the infrared detection result 1 , respectively control the infrared detection module and the ultrasonic detection module to perform close detection on the area centered on X 1 ; Record the camera position S i2 , the camera orientation I 2 and the detection data D i2 ; Set the ultrasonic scanning range R u1 , control the ultrasonic detection module to scan a rectangular area with a side length of 2×R 1 centered on X, and obtain the ultrasonic detection data D u1 ; u ;

[0048] Step Six: Respectively perform feature extraction on the obtained detection data D i2 and D u using the Gaussian pyramid to obtain the feature images of the infrared and ultrasonic detection defect positions, and register the two types of defects using the typical feature vectors generated by the impact crack traces to ensure the accuracy of data fusion;

[0049] Step Seven: After fusing the infrared and ultrasonic data according to the registration rules, load the point-clouded fusion result into the pre-built three-dimensional model according to the coordinate X 1 , and add it to the label of the model point cloud according to the depth data and distribution obtained by ultrasonic detection.

[0050] Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Anyone familiar with this technology can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention should be defined by the claims.

Claims

1. A large-scale composite material detection method based on infrared and ultrasonic data fusion, characterized in that: The method comprises the following steps: Step 1: Obtain the three-dimensional coordinates of the detection system and the structure to be measured, and establish a three-dimensional coordinate system X with the base of the detection system as the origin; Step 2: Use the infrared thermal wave module of the detection system to perform large-area imaging detection on the structure to be tested, record the position and direction of the infrared camera in the coordinate system X, and determine the three-dimensional coordinates of the tested area corresponding to the infrared camera's field of view based on the relative relationship between the infrared camera and the object to be tested in the same coordinate system; Step 3: Get the infrared large-area imaging result, obtain the coordinates of the defect centroid, adjust the position of the infrared camera, approach the defect for small-area imaging detection, record the position and direction of the infrared camera at this time, and determine the three-dimensional coordinates of the small-area imaging field of view of the infrared camera corresponding to the measured area; Step 4: The infrared detection module returns to the initial position, controls the ultrasonic detection module to approach the defect position, performs a small area scan with the defect position coordinates as the center, obtains the ultrasonic detection result, and performs image stitching and positioning according to the built-in encoder of the ultrasonic probe module and the ultrasonic probe coordinates; Step 5: Use Gaussian pyramid to extract feature vectors from infrared and ultrasonic detection results, use feature vectors extracted from infrared defects and ultrasonic detection defects to perform registration and image correction, and unify image distortion of infrared and ultrasonic detection results; Step 6: Acquire and discretize the defect features in the infrared and ultrasonic detection images, and calculate the defect depth based on the sound path information of the ultrasonic detection results; Step 7: Determine the defect coordinates according to the positions of the infrared camera and the ultrasonic probe in the three-dimensional coordinate system X, project the infrared and ultrasonic result point clouds obtained according to the obtained coordinates into the established three-dimensional model of the measured structure, and add defect labels to indicate the defect information.

2. A large-scale composite material detection method based on infrared and ultrasonic data fusion, characterized in that: The method comprises the following steps: Step 1: Take the fixed test equipment base as the origin and establish the world coordinate system X of the test environment; Step 2: Park the structure to be tested at a certain distance in front of the testing equipment to ensure that the testing equipment can control the ultrasonic testing module to approach the surface of the structure to be tested; Step 3: Measure the distance between the testing equipment and the structure to be tested, calculate the precise distance between the base of the equipment and at least four points on the surface of the structure to be tested, and establish a three-dimensional model of the structure to be tested in the world coordinate system X; Step 4: Determine the position S of the infrared camera in the world coordinate system X according to the connection relationship between the equipment base, mechanical structure and infrared camera i1 With the orientation I1, the infrared detection module is controlled to stimulate the surface of the structure to be tested, and the infrared camera is used to record data and calculate. At this time, in the established three-dimensional model, the position coordinate S of the center of the infrared camera field of view corresponding to the material to be tested is determined according to the orientation I1 of the infrared camera n1 ; Get S n1 The infrared detection data D i1 ; Step 5: Obtain the defect coordinate X1 in the infrared detection result, and control the infrared detection module and the ultrasonic detection module to perform close detection on the area centered on X1; record the camera position S i2 , camera orientation I2 and detection data D i2 ; Set the ultrasonic scanning range R u1 , control the ultrasonic detection module to the side length of 2×R centered at X1 u1 Scan the rectangular area to obtain ultrasonic detection data D u ; Step 6: Get the test data D i2 With D u Gaussian pyramid is used for feature extraction to obtain the characteristic images of the defect positions detected by infrared and ultrasonic detection. The typical characteristic vectors generated by impact crack traces are used to align the two defects to ensure the accuracy of data fusion. Step 7: After the infrared and ultrasonic data are fused according to the registration rules, the point cloud fusion result is loaded into the built 3D model according to the coordinate X1, and the depth data and distribution obtained by ultrasonic detection are added to the label of the model point cloud.

Images