A detection method for aviation composite materials based on multimodal information

Abstract

The invention discloses a detection method for aviation composite materials based on multimodal information, comprising capturing images, performing damage recognition according to the images, obtaining quantitative information of internal damage through T channel information based on the recognition results, and outputting a three-dimensional visualized damage report, performing damage assessment, and generating a maintenance work card. The RGB-T multimodal image composed of RGB and thermal infrared images is used to detect and segment significant target areas in the image, effectively recognize detailed features of different damages, realize rapid recognition and classification of composite material structure damage, determine defect spatial geometric features, accurately locate defects and measure defect sizes. The method has the advantages of rapid, non-contact, high-precision detection, and supports subsequent maintenance decisions, making up for the shortcomings of traditional infrared non-destructive testing technology, and has important industrial application value.

Description

Technical Field

[0001] The present invention relates to the technical field of composite material detection, and in particular to a detection method for aviation composite materials based on multimodal information. Background Art

[0002] Advanced composite materials are widely used in aerospace, automotive, chemical and textile industries due to their high specific stiffness, high specific strength, stable corrosion resistance, excellent vibration reduction and fatigue resistance. With the development of new high-performance composite materials, their proportion in aircraft structures is increasing, mainly used in aircraft radomes, landing gear doors, spoilers, flaps, ailerons, fairings, elevators, rudders, engine pods and other structural parts to bear and transmit the aircraft's aerodynamic loads.

[0003] Composite materials have special structures and complex manufacturing processes. Due to the influence of various factors, they may produce various types of damage defects during molding and service, such as pore defects, poor or rich adhesive, incomplete curing, matrix cracking, delamination, surface scratches and cracks, debonding and debonding, honeycomb core damage, etc. The defect forms of composite materials are relatively complex and changeable. Daily inspection and maintenance work shows that their defects often manifest as composite defects with multiple damage forms. The use of accurate and efficient non-destructive testing technology to timely determine the existence and expansion of such defects is of great significance to ensure the reliability of composite structural parts and the safe operation of in-service aircraft.

[0004] At present, the commonly used non-destructive testing technologies for aircraft include visual inspection, ultrasonic scanning, laser speckle interferometry, X-ray method, etc., but these detection methods have their own limitations. Visual inspection can observe the more obvious damage defects on the surface of aircraft parts, but it cannot detect defects inside the parts; ultrasonic detection requires the use of coupling agents, and ultrasonic attenuation leads to poor signal-to-noise of the detection signal, and different defects require different detection probes; laser speckle technology has the problem of poor sensitivity to defects in materials and difficulty in quantification; X-ray method has environmental pollution; infrared thermal wave non-destructive testing technology is widely used in internal defects, bonding quality and skin riveting of composite materials of aviation aircraft, but the original infrared thermal images usually have blurred edges, low signal-to-noise ratio, low detection accuracy and other problems. Therefore, a detection method for aviation composite materials based on multimodal information is urgently needed to solve the above technical problems. Summary of the invention

[0005] The present invention provides a detection method for aviation composite materials based on multimodal information, which can quickly, non-contact and high-precision detect composite material structures, to solve the problems existing in the above-mentioned prior art.

[0006] To achieve the above object, the present invention provides the following technical solution: a detection method for aviation composite materials based on multimodal information, comprising the following steps:

[0007] S1, capture images, inspect composite material structures through RGB cameras and infrared thermal imagers, and obtain visible light video stream information and multi-frame infrared images;

[0008] S2. Damage identification, including:

[0009] The damage is visible to the naked eye, and the RGB single channel information is used in conjunction with the YOLOV3 network to directly identify the damaged target;

[0010] Damage that is not visible to the eye is detected. Images are acquired based on RGB-T dual-channel information. The acquired images are preprocessed, and then saliency detection is used to obtain suspected target areas in infrared images. The YOLOV3 network is used to perform multi-level structural fault recognition on the suspected target areas.

[0011] S3. Based on the recognition results, quantitative information of internal damage is obtained through T channel information, including damage area and damage depth;

[0012] S4. Output a 3D visual damage report, conduct damage assessment, and generate a maintenance work card.

[0013] Preferably, in step S1, the composite material passes through a detection area formed by an RGB camera and an infrared thermal imager at a speed of ≤1.5 km / h, wherein the infrared thermal imager is located in the horizontal and vertical directions of the target area, and ensures that the field of view of the infrared thermal imager is uniform.

[0014] Preferably, in step S2, the image preprocessing is specifically:

[0015] (1) Improve the threshold and perform denoising on the infrared image using the wavelet denoising method, where the threshold is α = 0.006;

[0016] (2) Use adaptive histogram equalization to perform image enhancement processing;

[0017] (3) Use fusion strategy to obtain the fusion result of infrared image and visible light image.

[0018] Preferably, in step S3, the damage area S 损 For: S 损 =N·p, where N is the number of white pixels in the binarized infrared thermal image, and p is the actual area represented by each pixel in the infrared image;

[0019] in, s0 is the field of view of the lens, f0 is the focal length of the infrared thermal imager lens, f is the imaging focal length of the lens, and pixel is the resolution of the infrared thermal imager.

[0020] Preferably, in step S3, the damage depth d is: α is the thermal diffusivity of the material being tested, and t is the time of the frame with the highest temperature.

[0021] Preferably, in step S4, during damage assessment, the damage report is compared with the structural manual in XML format, wherein structural faults are classified as being within the scope of the manual or outside the scope of the manual, and a corresponding maintenance worker card is generated.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows: the RGB-T multimodal image composed of RGB and thermal infrared images is used to detect and segment the significant target area in the image, the detailed features of different damages are effectively identified, the rapid identification and classification of composite material structure damages are realized, the spatial geometric features of defects are determined, the defects are accurately located and the defect sizes are measured, it has the advantages of fast, non-contact and high-precision detection, and supports subsequent maintenance decisions, which makes up for the shortcomings of traditional infrared non-destructive testing technology and has important industrial application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0024] In the attached picture:

[0025] Figure 1 is a flow chart of composite material structure fault detection of the present invention;

[0026] Figure 2 is a flow chart of composite material structure fault identification according to the present invention;

[0027] Figure 3 is a damage detection and maintenance decision flow chart of the present invention;

[0028] Figure 4 It is a flow chart of aviation composite material structure fault identification of the present invention. DETAILED DESCRIPTION

[0029] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0030] Example: Figure 1-3 As shown, a detection method for aviation composite materials based on multimodal information includes the following steps:

[0031] S1. Capture images, inspect the composite material structure through an RGB camera and an infrared thermal imager, and obtain visible light video stream information and multi-frame infrared images; wherein the composite material component passes through the detection area formed by the RGB camera and the infrared thermal imager at a speed of ≤1.5km / h, wherein the infrared thermal imager is located in the horizontal and vertical directions of the target area, and ensure that the infrared thermal imager has a uniform field of view;

[0032] S2. Damage identification, including:

[0033] The damage is visible to the naked eye, and the RGB single channel information is used in conjunction with the YOLOV3 network to directly identify the damaged target;

[0034] Damage that is not visible to the eye is detected. Images are acquired based on RGB-T dual-channel information. The acquired images are preprocessed, and then saliency detection is used to obtain suspected target areas in infrared images. The YOLOV3 network is used to perform multi-level structural fault recognition on the suspected target areas.

[0035] Among them, the image preprocessing is specifically as follows:

[0036] (1) Improve the threshold and perform denoising on the infrared image using the wavelet denoising method, where the threshold is α = 0.006;

[0037] (2) Use adaptive histogram equalization to perform image enhancement processing;

[0038] (3) Using fusion strategy to obtain the fusion result of infrared image and visible light image;

[0039] S3. Based on the recognition results, quantitative information of internal damage is obtained through T channel information, including damage area and damage depth, where:

[0040] Damage area S 损 For: S 损 =N·p, where N is the number of white pixels in the binarized infrared thermal image, and p is the actual area represented by each pixel in the infrared image;

[0041] in, s0 is the field of view of the lens, f0 is the focal length of the infrared thermal imager lens, f is the imaging focal length of the lens, and pixel is the resolution of the infrared thermal imager;

[0042] The damage depth d is: α is the thermal diffusivity of the material being tested, and t is the time of the frame with the highest temperature.

[0043] S4. Output a three-dimensional visualized damage report, conduct damage assessment, and generate a maintenance worker card; wherein, during the damage assessment, the damage report is compared with the structural manual in XML format, wherein the structural faults are classified as being within the scope of the manual or outside the scope of the manual, and a corresponding maintenance worker card is generated.

[0044] refer to Figure 4 , inspect the composite structure of the aircraft, build an inspection door, install the RGB camera and the infrared thermal imager on the inspection door, and the aircraft passes through the inspection door by a ground tractor at a speed of ≤1.5km / h. According to the inspection requirements, adjust the position of the infrared thermal imager and ensure that the position of the inspection target area is at the center of the field of view of the infrared thermal imager to ensure the accuracy and completeness of the defect display in the infrared image obtained later, and use the above steps to identify, locate and measure damage. In this embodiment, a planar pulse heat source is used as a pulse flash, the resolution pixel of the infrared thermal imager is 640*480, the thermal sensitivity is 0.06℃, each infrared thermal image has 640*480 independent temperature measurement points, and the corresponding collected infrared video field of view is S, then: p=S / 640*480, the imaging focal length f (cm) of the standard lens and the field of view S (mm 2 ) is: S / S0=f 2 / f0 2 , so S=S0*f 2 / f0 2 In this embodiment, the infrared thermal imager lens uses a 35cm standard lens, the focal length f0 = 30cm, and the lens field of view is S0 = 130 × 100 = 13000 (mm 2 ). When the lens imaging focal length is adjusted between 30 and 40 cm during the test, In this example, when f = 35 cm, the number of white pixels in the infrared thermal image after binarization is 179032, and the damage area S 损 Loss: S 损 =N·p=4.702×10 -5 ×350 2 ×179032=1.03×10 7 mm 2 The thermal diffusivity of the material is 5.337×10 -7 , the time t for the maximum temperature difference is 0.136s, and the damage depth is

[0045] Finally, it should be noted that the above description is only a preferred example of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A detection method for aviation composite materials based on multimodal information, It is characterized in that The steps include: S1, capture images, inspect composite material structures through RGB cameras and infrared thermal imagers, and obtain visible light video stream information and multi-frame infrared images; S2. Damage identification, including: The damage is visible to the naked eye, and the RGB single channel information is used in conjunction with the YOLOV3 network to directly identify the damaged target; Damage that is not visible to the eye is detected. Images are acquired based on RGB-T dual-channel information. The acquired images are preprocessed, and then saliency detection is used to obtain suspected target areas in infrared images. The YOLOV3 network is used to perform multi-level structural fault recognition on the suspected target areas. S3. Based on the recognition results, quantitative information of internal damage is obtained through T channel information, including damage area and damage depth; Damage area S 损 For: S 损 =N·p, where N is the number of white pixels in the infrared thermal image after binarization, and p is the actual area represented by each pixel in the infrared image; in, s 0 is the field of view of the lens, f 0 is the focal length of the infrared thermal imager lens, f is the imaging focal length of the lens, and pixel is the resolution of the infrared thermal imager; The damage depth d is: α is the thermal diffusivity of the material being tested, and t is the time of the frame with the highest temperature; S4. Output a 3D visual damage report, conduct damage assessment, and generate a maintenance work card.

2. The method for detecting aviation composite materials based on multimodal information according to claim 1, Features: In step S1, the composite material passes through the detection area formed by the RGB camera and the infrared thermal imager at a speed of ≤1.5km / h, wherein the infrared thermal imager is located in the horizontal and vertical directions of the target area, and the field of view of the infrared thermal imager is ensured to be uniform.

3. The method for detecting aviation composite materials based on multimodal information according to claim 1, Features: In step S2, the image preprocessing is specifically as follows: (1) Improve the threshold and perform denoising on the infrared image using the wavelet denoising method, where the threshold is α = 0.006; (2) Use adaptive histogram equalization to perform image enhancement processing; (3) Use fusion strategy to obtain the fusion result of infrared image and visible light image.

4. The method for detecting aviation composite materials based on multimodal information according to claim 1, Features: In step S4, during damage assessment, the damage report is compared with the structural manual in XML format, wherein structural faults are classified as being within the scope of the manual or outside the scope of the manual, and a corresponding maintenance worker card is generated.

Images