Infrared-based rapid detection method for surface structure damage of whole aircraft

Abstract

The invention belongs to the technical field of aircraft surface structure damage detection, and relates to an infrared-based rapid aircraft surface structure damage detection method. According to the method, rapid global capture of infrared images is realized through automatic equipment, a high-precision three-dimensional grid model is established based on multi-view visible light images, the spatial poses of the infrared images are solved, and an infrared image data set with three-dimensional coordinate information is constructed; intelligent recognition of structural damage is realized by adopting a deep learning algorithm, and damage types and areas are automatically recognized and marked; and three-dimensional accurate positioning of the damage is realized through a space mapping technology, an identification result is accurately projected to the surface of the three-dimensional grid model, and a three-dimensional damage marking model containing damage space distribution is generated. According to the invention, through deep fusion of infrared rapid imaging, an intelligent identification algorithm and a three-dimensional reconstruction technology, technology expansion from single-point, two-dimensional and artificial technology to global, three-dimensional and intelligent technology is realized, and detection efficiency, identification accuracy and positioning precision are improved.

Description

Technical Field

[0001] This invention belongs to the field of aircraft surface structure damage detection technology, and in particular, it is a rapid detection method for surface structure damage of an aircraft based on infrared radiation. Background Technology

[0002] The surface structure of an aircraft is crucial for ensuring the safe operation of special-purpose aircraft. After a high-speed flight, the surface structure often suffers damage such as cracking, debonding, and surface ablation due to intense thermal cycling. This type of damage not only threatens flight safety, but its lengthy inspection and maintenance cycle also severely restricts the aircraft's support capabilities. Improving support capabilities requires first addressing the efficiency bottleneck in the damage detection process. Currently, damage detection methods for surface structures can be categorized into contact and non-contact methods based on their operation. Contact methods include manual pulling, acoustic vibration detection, and built-in sensor technology. These methods typically require direct contact with the structure under test or pre-embedding of sensors: pulling detection may cause secondary damage, acoustic vibration detection is insensitive to deeply buried defects, and both are highly dependent on the operator's experience. Furthermore, while built-in sensor technology can achieve automatic monitoring, the requirement for pre-embedding increases the complexity of the structural design and the overall weight. Non-contact methods include manual visual inspection, X-ray, and infrared detection. However, manual visual inspection cannot detect internal damage; X-ray detection is costly and carries radiation risks. These methods all have significant limitations in rapid field support applications. Against this backdrop, infrared thermal imaging technology utilizes the inherent internal and external temperature differences caused by the thermal inertia of the surface structure after a spacecraft's return. It detects damage by capturing the abnormal temperature field distribution caused by the damaged area, eliminating the need for additional thermal excitation devices and demonstrating its unique applicability in outdoor fields. This technology offers advantages such as non-contact and rapid imaging, enabling rapid imaging of large areas with high detection efficiency. It also eliminates the cost of expensive equipment and protective measures required by X-ray methods and overcomes the operational dependence of ultrasonic methods on coupling agents and contact scanning. However, existing infrared detection equipment for outdoor applications is still primarily traditional handheld, which has the following limitations: the detection process relies on manual image capture and subjective interpretation, providing only isolated two-dimensional images and failing to automatically identify damage types and achieve three-dimensional spatial localization; scanning efficiency is low and speed is slow, making it difficult to achieve comprehensive coverage of complex structures such as curved surfaces and edges. This working mode cannot meet the needs of modern rapid field support. Developing a novel infrared detection method suitable for outdoor conditions, capable of rapid, intelligent identification and three-dimensional localization of surface structural damage, has become an urgent technical requirement. Summary of the Invention

[0003] The purpose of this invention is to overcome the problems of low efficiency and inability to achieve rapid damage identification and accurate three-dimensional positioning in existing infrared detection methods for the surface structure of special aircraft. This invention deeply integrates rapid infrared imaging, intelligent recognition algorithms, and three-dimensional reconstruction technology, achieving a technological expansion from "single-point, two-dimensional, manual" to "global, three-dimensional, and intelligent." Compared with traditional manual visual inspection and tapping methods, it achieves improvements in detection efficiency, identification accuracy, and positioning precision.

[0004] The technical solution of this invention is: a rapid field detection method for surface structural damage of an aircraft based on infrared imaging. First, rapid global acquisition of infrared images is achieved through automated equipment. Second, a high-precision three-dimensional mesh model is established based on multi-view visible light images, and the spatial pose of the infrared images is solved to construct an infrared image dataset with three-dimensional coordinate information. On this basis, a deep learning algorithm is used to achieve intelligent identification of structural damage, automatically identifying and marking damage types and regions. Finally, spatial mapping technology is used to achieve precise three-dimensional localization of the damage, accurately projecting the identification results onto the surface of the three-dimensional mesh model to generate a three-dimensional damage marking model containing the spatial distribution of the damage.

[0005] The rapid detection method is specifically as follows: Step 1: Calibration of a dual-light camera calibration system based on a composite calibration plate and a blackbody: Construct a dual-light camera calibration system, which includes a composite calibration plate, a standard blackbody radiation source, a three-dimensional translation stage for precisely controlling the relative pose of the calibration plate and the camera, and a data processing unit for executing calibration algorithms and parameter calculations; Perform visible light camera calibration, infrared camera calibration, joint calibration, and non-uniformity parameter calibration on the dual-light camera, and establish a unified spatial coordinate relationship between the infrared image and the visible light image; Step 2, Detection path planning based on the 3D geometric model of the aircraft and safety distance constraints: Based on the 3D geometric model of the aircraft to be detected, plan a full-coverage detection path that meets the safety constraints; Step 3, Hardware-synchronized dual-light image sequence acquisition: Control the automated equipment equipped with dual-light cameras to acquire images along the detection path planned in Step 2, and obtain visible light image sequence and infrared image sequence of the aircraft surface for subsequent 3D reconstruction and damage identification. Step 4: Geometric distortion and non-uniformity correction of visible light and infrared images: Preprocess the acquired raw visible light and infrared images, including distortion correction and non-uniformity correction, to eliminate systematic errors; first, perform distortion correction on the visible light and infrared images to obtain distorted visible light and infrared images; then, perform non-uniformity correction on the distorted infrared images to obtain non-uniformity corrected infrared images. Step 5: 3D Gaussian Splash Reconstruction and 3D Mesh Model Generation Based on Visible Light Images: Based on the preprocessed visible light image sequence, a high-precision 3D scene model of the aircraft surface is constructed using the 3D Gaussian splash reconstruction method, and the 3D spatial pose information P of each visible light image in the global coordinate system is output. vis =[ | The method is used to solve the spatial pose of infrared images and provides a three-dimensional model basis for the spatial mapping and visualization of damage markers. The 3D Gaussian splash reconstruction method includes scene initialization and parameterization, covariance parameterization, differential optimization, and three-dimensional mesh model generation. Step 6: Global spatial pose calculation of infrared images based on the spatial transformation model: The precise camera pose P of each visible light image is calculated using the spatial transformation model R and T obtained in Step 1 and the 3D reconstruction obtained in Step 5. vis Calculate the spatial pose P of each infrared image in the global coordinate system. ir Finally, a spatial pose dataset of infrared image sequences is obtained, which is used to map the identified damage from the two-dimensional infrared image to the surface of the three-dimensional model to achieve three-dimensional localization of the damage. Step 7: Pixel-level damage segmentation mask generation based on encoder-decoder architecture: First, the infrared image sequence is standardized to adjust each image to a fixed size of 512×512 pixels suitable for the segmentation network; then, the standardized infrared image sequence is input into the damage intelligent recognition model based on encoder-decoder architecture to obtain the 512×512 pixel damage segmentation mask output by the model; finally, the size restoration operation is performed on the output damage segmentation mask to generate an original-size damage segmentation mask aligned with the original input image space. Step 8: Fusion rendering of damage segmentation mask and original infrared image: The damage segmentation mask output in step 7 is fused with the corresponding original infrared image to generate a synthetic diagnostic image with color damage markers. Step 9: Generation of a 3D damage marking model based on texture projection and multi-view fusion: Based on the spatial pose dataset of the infrared image sequence established in Step 6, the synthetic diagnostic image output in Step 8 is mapped to the corresponding surface area of ​​the 3D mesh model generated in Step 5, and finally a 3D infrared damage marking model with both 3D geometry and damage marking information is generated.

[0006] The substrate of the composite calibration plate in step 1 is made of metal. One side of the plate is a black and white checkerboard pattern for calibrating the geometric parameters of the visible light camera. The other side is a coated surface with an emissivity of 0.95-0.98 for infrared calibration. To achieve physical spatial uniformity of the feature points of the two cameras, a micro-cylindrical recess is machined at the position of the coated surface corresponding to each corner point of the black and white checkerboard pattern. The diameter of the recess is 0.5-1mm and the depth is 0.1-0.3mm. The center of the recess is aligned with the center of the corner point of the checkerboard pattern on the front side in three-dimensional space.

[0007] The visible light camera calibration in step 1 uses a black and white checkerboard pattern on a composite calibration plate. A three-dimensional translation stage is controlled to acquire image groups from N different viewpoints, where N is between 20 and 30. The Zhang Zhengyou calibration method is used to solve for the intrinsic parameter matrix of the visible light camera. and distortion coefficient The intrinsic parameter matrix Defined as:

[0008]

[0009] in, The equivalent focal length of the visible light camera in the x-direction of the image coordinate system, in pixels; The equivalent focal length of the visible light camera in the y-direction of the image coordinate system, in pixels; The x-component of the principal point coordinates of the visible light camera, in pixels; The y-component of the principal point coordinates of the visible light camera, in pixels; , The radial distortion coefficient of the visible light camera; denoted as the tangential distortion coefficient of a visible light camera.

[0010] The infrared camera calibration in step 1 involves: using a high-emissivity coating on a composite calibration plate, acquiring image sets from N different viewpoints under the same three-dimensional translation stage trajectory as the visible light calibration; and using a nonlinear optimization algorithm to solve for the intrinsic parameter matrix of the infrared camera. and distortion coefficient The intrinsic parameter matrix Defined as:

[0011]

[0012] in, The equivalent focal length of the infrared camera in the x-direction of the image coordinate system, in pixels; The equivalent focal length of the infrared camera in the y-direction of the image coordinate system, in pixels; The x-component of the principal point coordinates of the infrared camera, in pixels; The y-component of the principal point coordinates of the infrared camera, in pixels; The radial distortion coefficient of the infrared camera; denoted as the tangential distortion coefficient of the infrared camera.

[0013] The system joint calibration in step 1 involves: using a hand-eye calibration method to solve the relative pose relationship between the visible light camera and the infrared camera; and using the synchronized observation image sets of the two cameras on the composite calibration board acquired during the visible light camera calibration and the infrared camera calibration to obtain multiple pairs of physical points with fixed spatial constraints. Each pair of physical points contains two physical points with a defined three-dimensional spatial relationship: one is a checkerboard corner point P on the front of the composite calibration board. f The other is the center point P of the micro-frustum-shaped depression on its back. b The relative position vector d between two points in the calibration plate coordinate system is a known constant. Based on the camera intrinsic parameters and corresponding image coordinates obtained from calibration, and combined with the known three-dimensional structure of the calibration plate, point P is calculated using the perspective n-point method. f The three-dimensional coordinates P in the visible light camera coordinate system vis and point P b The three-dimensional coordinates P in the infrared camera coordinate system ir ; Utilizing multiple sets of corresponding and Establish the spatial transformation models R and T between the two camera coordinate systems, with the following formulas:

[0014] Among them, P vis Let P be the point f Three-dimensional coordinates in the visible light camera coordinate system, unit: millimeters; P ir Let P be the point b The three-dimensional coordinates in the infrared camera coordinate system, in millimeters; R is a 3×3 rotation matrix describing the rotation transformation from the infrared camera coordinate system to the visible light camera coordinate system; T is a 3×1 translation vector, in millimeters, describing the translation transformation from the infrared camera coordinate system to the visible light camera coordinate system.

[0015] The non-uniformity parameter calibration in step 1 is performed using a standard blackbody radiation source at high temperature. and low temperature Multiple frames of uniform field infrared images were acquired at two temperature points, and the gain correction coefficient G(x,y) and offset correction coefficient B(x,y) for each pixel were calculated:

[0016]

[0017] in, and Each pixel High temperature and low temperature The average grayscale value of multiple frames of images; For pixels Gain correction factor, unit: gray level / ; For pixels Offset correction coefficient, unit: gray level.

[0018] Step 2, planning the full-coverage detection path: First, calculate the optimal working distance range [D] for the dual-light camera to achieve clear imaging on the entire surface structure. min D max Then, using this as a constraint, a reciprocating scanning coverage path planning algorithm is adopted to generate an initial path on the surface of the 3D model that meets the requirement of 70% planning overlap rate; finally, cubic B-spline curves are used to smooth and optimize the path, improving the smoothness of the path while maintaining the coverage effect.

[0019] The optimal working distance range [D] min D max The imaging requirements and safety constraints must be met simultaneously; the specific calculation method is as follows: First, calculate the minimum theoretical distance D. min_theory D min_theory =The closest focusing distance specified by the lens; Then calculate the maximum theoretical distance D. max_theory ;

[0020] in: Camera focal length; : The size of the damage to the object being measured; The maximum ground sampling distance required to identify this damage; Then, take a safe distance D. secure =0.5-1 meter, this value is a safety margin set comprehensively considering factors such as the motion control accuracy of the detection equipment, external environmental disturbances and structural form and position deviations, to ensure that the detection equipment maintains a safe distance from the surface of the aircraft and does not interfere with any protruding structures; Finally, according to , and The relationship determines the final working distance range: if ,but , ; if ,but , ; if This indicates that the current lens configuration cannot meet the imaging requirements at a safe distance, and the equipment parameters need to be adjusted or a new lens needs to be selected.

[0021] The planned overlap rate refers to the lateral overlap rate: that is, the overlap rate between images acquired by adjacent parallel paths in the direction perpendicular to the detection path. It is ensured by controlling the spatial distance between adjacent parallel path segments, and its calculation formula is as follows:

[0022] in: This refers to the lateral overlap rate. The distance between adjacent parallel paths. The width of a single frame of image coverage on the object plane.

[0023] The automated equipment in step 3 adopts a multi-degree-of-freedom parallel mechanism with six degrees of freedom of motion capability, which can achieve precise control of the acquisition angle. During the image acquisition process, the 70% acquisition overlap rate is guaranteed by controlling the movement speed and acquisition frame rate of the automated equipment.

[0024] The acquisition overlap rate refers to the forward overlap rate: that is, the overlap rate between two adjacent frames of images along the detection path. Its calculation formula is as follows:

[0025] in: For heading overlap, For the forward speed of the motion platform, For image acquisition frame rate, The length of a single frame image coverage on the object plane.

[0026] The dual-light camera includes a hardware synchronization trigger module; both the visible light and infrared image sequences are embedded with precise timestamps and location encoding information.

[0027] Step 4, distortion correction, involves performing geometric distortion correction on visible light and infrared images using the Brown-Conrady lens distortion correction model. The correction model is expressed as follows:

[0028]

[0029] in: and Represents the coordinates of the original image; and Indicates the corrected image coordinates; , , Indicates the radial distortion coefficient; , Indicates the tangential distortion coefficient; The radial distance to the image center is represented by the formula: .

[0030] Step 4, non-uniformity correction, is based on the obtained gain correction coefficient G(x,y) and offset correction coefficient B(x,y). A two-point correction method is used to perform pixel-level response compensation on the infrared image. The correction formula is as follows:

[0031] in For the original infrared image in pixels grayscale value at that location This is the offset correction coefficient for that pixel. This is the gain correction coefficient for that pixel. This is the corrected grayscale value.

[0032] Step 5, scene initialization and parameterization, firstly involves processing the sparse point cloud obtained from the visible light image sequence using motion reconstruction technology and the initial camera pose P'. vis= [ | Subsequently, each 3D point in the sparse point cloud is initialized as a 3D Gaussian function, and its core parameters are defined for each function: the position parameter μ ∈ R³ determines its spatial position, and the covariance matrix Σ ∈ R³ x ³ It controls the initial shape and spatial orientation; finally, it outputs an initial 3D Gaussian scene representation composed of a set of parameterized Gaussian functions, providing a high-precision three-dimensional coordinate mapping basis for the intelligent recognition results of the damaged area.

[0033] Step 5, covariance parameterization, involves decomposing the covariance matrix Σ into a rotation matrix R and a scaling matrix S. Its mathematical expression is as follows: Σ = RSS T R T By optimizing the rotation and scaling parameters respectively, a refined reconstruction of the complex geometric features such as curved surfaces and edges of the aircraft's overall surface structure can be achieved.

[0034] Step 5, differential optimization, involves projecting the 3D Gaussian scene onto a 2D image plane using a differentiable Gaussian splash renderer to generate a composite view. All Gaussian function parameters and camera pose are iteratively optimized by minimizing the difference between the composite view and the corresponding real visible light image. The composite loss function used is: L = (1 - λ)L1+λL D-SSIM Where L1 is the L1 loss, L D-SSIM The structural similarity loss is represented by λ, which is the balancing weight. During optimization, an adaptive density control strategy is employed, involving cloning, splitting, and pruning operations based on the position gradient and transparency of the Gaussian function to improve the reconstruction capability of structural details. Through this optimization process, the accurate camera pose P in the global coordinate system is obtained. vis .

[0035] Step 5, generating the 3D mesh model, involves obtaining a 3D scene model composed of an optimized set of 3D Gaussian functions after differential optimization. Using the Poisson surface reconstruction algorithm, a 3D mesh model that maintains the original geometric accuracy is generated from the optimized set of 3D Gaussian functions, serving as the unified spatial basis for subsequent 3D damage mapping. The 3D mesh model is a watertight triangular mesh model, which has the geometric characteristics of regular topological structure, no self-intersection, and no holes, ensuring the stability and accuracy of subsequent texture projection and fusion processes.

[0036] The spatial pose P of the infrared image in step 6 in the global coordinate system ir =[ | The calculation formula is as follows:

[0037]

[0038] in: For the transition from the world coordinate system to the visible light camera coordinate system Rotation matrix; For the transition from the world coordinate system to the visible light camera coordinate system Translation vector, unit: millimeters; and The rotation matrix and translation vector from the infrared camera coordinate system to the visible light camera coordinate system obtained in step 1; Rotation matrix The inverse matrix represents the rotational transformation from the visible light camera coordinate system to the infrared camera coordinate system; and This refers to the pose parameters obtained from the world coordinate system to the infrared camera coordinate system.

[0039] The standardization process in step 7 uses a fill scaling method that maintains the aspect ratio of the image. Specifically, it involves: calculating the original height H and width W of the image, and determining the scaling ratio scale = 512 / max(H, W); scaling the image according to this ratio to obtain a new height H' = round(H * scale) and width W' = round(W * scale); and then filling the image with pixels on both sides or top and bottom to make the final image size 512×512 pixels.

[0040] The pixel padding uses a specific value of -1, which is significantly different from the valid image area, to identify invalid regions in subsequent network processing; during this process, the scaling ratio (scale) and the horizontal padding amount are recorded. = (512 - W') / 2 and vertical fill amount = (512 - H') / 2 is used as a transformation parameter for subsequent size restoration operations.

[0041] The damage intelligent recognition model in step 7 is a trained deep convolutional neural network with an encoder-decoder structure. The model takes a standardized infrared image as input. The encoder consists of multiple downsampling stages, which are used to extract and compress multi-scale abstract features from the input image layer by layer. The decoder consists of corresponding upsampling stages, which gradually restore the spatial resolution of the feature maps by fusing the feature maps of each layer of the encoder, and finally outputs a 512×512 pixel single-channel segmentation mask. The integer value of each pixel is used to uniquely identify the damage category corresponding to the location: 0 represents the background, 1 represents cracking, 2 represents debonding, and 3 represents surface ablation.

[0042] The size restoration operation in step 7 employs an inverse mask transformation method corresponding to the normalization process: this method first uses the recorded pad... x and pad y The padding region in the standardized segmentation mask is removed to obtain an intermediate mask of size (H', W'), which corresponds to the scaled original image content. Then, this intermediate mask is restored to the original input image size (H, W). The restoration process is achieved by establishing a coordinate mapping relationship from the target size (H, W) to the intermediate size (H', W'): for any integer coordinate position (x...) in the target mask... dst , y dst ), through the mapping relationship x src =x dst ·(W' / W) and y src = y dst · (H' / H) Calculate its corresponding source coordinates (x) in the intermediate mask. src,y src ), and use the nearest neighbor interpolation algorithm to calculate the distance (x) src , y src The category label value of the nearest integer coordinate pixel is assigned to the target pixel (x). dst , y dst The transformation is completed by traversing all target pixels, ultimately generating a damage segmentation mask that is pixel-aligned and has the same spatial resolution as the original input image.

[0043] Step 8, the fusion process, uses the original infrared image as the background substrate. According to the preset color mapping relationship, the pixel areas of different damage categories in the mask are rendered as corresponding specific colors, where the background is transparent, cracks are green, debonding is red, and surface ablation is blue. Through channel duplication, the single-channel original infrared image is converted into a three-channel image. Using an image overlay algorithm, the colored damage marking area is covered with the corresponding position of the background image with a transparency parameter of 50%-80%, and finally a synthetic diagnostic image that can intuitively display the damage distribution, geometric contours, and category attributes is formed.

[0044] The diagnostic image synthesized in step 8 comprises two information layers: the bottom layer is a single-channel original infrared grayscale image, preserving its original infrared radiation information; the upper layer is a color damage marker layer generated based on damage segmentation masks, capable of characterizing the contour of the damage region and the damage category; the image overlay algorithm employs the α-mixing algorithm, the mathematical expression of which is:

[0045] in, The output is a synthetic diagnostic image. For color-marked damage images, This is a three-channel infrared image obtained by channel replication. For transparency parameters, For RGB color channel indexing; the channel copying is performed by copying the original single-channel infrared image. The grayscale value of each pixel at position (x, y) is simultaneously assigned to the three-channel image. The conversion is achieved through the R, G, and B channels, and their conversion relationship is expressed as follows: = = =

[0046] in, This represents a three-channel infrared image obtained through channel replication. Represents the image pixel coordinates, where R, G, and B represent the red, green, and blue color channels, respectively. Raw single-channel infrared image.

[0047] The mapping process in step 9 includes texture projection and multi-view fusion; The texture projection is based on the 3D coordinates of each vertex of the 3D mesh model and the pose P of the infrared image. ir =[ | The texture coordinate correspondence between the mesh and the synthetic diagnostic image is established through a perspective projection model; for any point P on the mesh surface... w Its projected coordinates (u,v) on the i-th synthetic diagnostic image are determined by the following projection relationship:

[0048] in: The projection depth factor; The homogeneous coordinates of points on the 3D network surface in the global coordinate system, in millimeters; For the first The camera pose matrix corresponding to the infrared image, where for Rotation matrix, for Translation vector, unit: millimeters This is the intrinsic parameter matrix of the infrared camera; The multi-view fusion process calculates fusion weights based on the viewing angle θ and camera distance d for multiple synthetic diagnostic images covering the same grid area. Weighted fusion eliminates stitching artifacts, generating a seamless panoramic infrared texture map, which is then bound to a 3D mesh model, ultimately producing a complete 3D infrared damage marking model. The fusion weight calculation formula is as follows:

[0049] in: The angle between the surface normal vector and the camera's viewing direction, in radians; The Euclidean distance from the camera's optical center to the surface electric field is expressed in millimeters. The normalized fusion weights have a value range of [value range missing]. .

[0050] The beneficial effects of this invention lie in providing a rapid infrared detection method for surface structural damage of an entire machine that integrates automated acquisition, intelligent identification, and three-dimensional positioning. This method represents a technological expansion from "single-point, two-dimensional, and manual" to "global, three-dimensional, and intelligent." It can comprehensively and rapidly detect surface structural damage across the entire machine, automatically identify damage types, accurately calibrate three-dimensional spatial locations, and generate intuitive and visual three-dimensional damage marking models. This significantly improves the objectivity, efficiency, and reliability of the detection, effectively solving the technical shortcomings of traditional infrared detection methods in outdoor environments, such as reliance on manual subjective interpretation and the inability to achieve rapid damage identification and accurate three-dimensional positioning. It possesses promising prospects for equipment application. Attached Figure Description

[0051] Figure 1 This is a flowchart of the detection method described in this invention.

[0052] Figure 2 This is a schematic diagram of step 1 of the present invention.

[0053] Figure 3 This is a schematic diagram of step 2 of the present invention.

[0054] Figure 4 This is a schematic diagram of step 3 of the present invention.

[0055] Figure 5 This is a schematic diagram of step 4 of the present invention.

[0056] Figure 6 This is a schematic diagram of step 5 of the present invention.

[0057] Figure 7 This is a schematic diagram of step 7 of the present invention.

[0058] Figure 8 This is a schematic diagram of step 8 of the present invention.

[0059] Figure 9 This is a schematic diagram of step 9 of the present invention.

[0060] In the diagram: 100, Composite calibration plate; 101, 3D translation stage; 102, Two-light camera; 103, Synchronous observation image set; 104, Visible light camera coordinate system; 105, Infrared camera coordinate system; 106, Standard blackbody radiation source; 107, High temperature. and low temperature Multiple frames of uniform field infrared images acquired at two temperature points; 108. 3D translation stage motion trajectory; 200. Initial detection path; 201. Path after smoothing optimization; 202. Images acquired from adjacent parallel paths; 301. Automated equipment; 302. Two adjacent frames of images along the detection path forward direction; 400. Infrared image sequence; 401. Visible light image sequence; 402. Infrared image sequence after distortion correction; 403. Infrared image sequence after non-uniform correction; 404. Visible light image sequence after distortion correction; 501. Sparse point cloud; 502. 3D high 503. Covariance matrix Σ; 504. Rotation matrix R; 505. Scaling matrix S; 506. Composite view; 507. Cloning, splitting, and trimming operations; 508. High-precision 3D scene model of aircraft surface; 509. 3D mesh model; 702. Scaled image sequence; 703. Normalized infrared image sequence; 704. Intelligent damage recognition model; 705. Original damage segmentation mask; 706. Intermediate mask; 707. Damage segmentation mask; 801. Composite diagnostic image; 901. Multiple composite diagnostic images covering the same mesh area. Detailed Implementation

[0061] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0062] like Figure 1 As shown, a rapid field detection method for surface structural damage of an aircraft based on infrared imaging is proposed. First, rapid global acquisition of infrared images is achieved through automated equipment. Second, a high-precision three-dimensional mesh model is established based on multi-view visible light images, and the spatial pose of the infrared images is solved to construct an infrared image dataset with three-dimensional coordinate information. On this basis, a deep learning algorithm is used to achieve intelligent identification of structural damage, automatically identifying and marking the damage type and region. Finally, spatial mapping technology is used to achieve precise three-dimensional localization of the damage, and the identification results are accurately projected onto the surface of the three-dimensional mesh model to generate a three-dimensional damage marking model containing the spatial distribution of damage.

[0063] The specific implementation steps are as follows: Step 1, as follows Figure 2As shown, the calibration of a dual-light camera calibration system based on a composite calibration plate and a blackbody is performed. The dual-light camera calibration system includes a composite calibration plate (100), a standard blackbody radiation source (106), a three-dimensional translation stage (101) for precisely controlling the relative pose of the calibration plate and the camera, and a data processing unit for executing calibration algorithms and parameter calculations. Visible light camera calibration, infrared camera calibration, joint calibration, and non-uniformity parameter calibration are performed on the dual-light camera (102) to correct inherent lens distortion, establish a unified spatial coordinate relationship between the infrared and visible light images, and ensure the consistency of the infrared image pixel response. The composite calibration plate has a metal substrate. One side of the plate is a high-precision black and white checkerboard surface for geometric parameter calibration of the visible light camera; the other side is a coated surface with an emissivity of 0.95-0.98 for infrared calibration. To achieve physical spatial uniformity of feature points from both cameras, a micro-cylindrical recess is precisely machined at the high-emissivity coating position corresponding to each corner point of the high-precision black-and-white checkerboard pattern. The diameter of this recess is 0.5-1 mm, and its depth is 0.1-0.3 mm. This structural design allows for the formation of stable high-temperature or low-temperature feature points in the recessed area under infrared thermal imaging due to the difference in heat capacity and radiation characteristics between the recessed area and the surrounding coating. Furthermore, the center of each recess is strictly aligned in three-dimensional space with the center of the corner point of the checkerboard pattern on the front side, thus providing a unique and precise set of physical control points for joint calibration.

[0064] The visible light camera calibration employs a high-precision black and white checkerboard pattern on a composite calibration plate (100). A three-dimensional translation stage (101) is used to acquire image groups from N different viewpoints, where N is between 20 and 30. The Zhang Zhengyou calibration method is used to solve for the intrinsic parameter matrix K_vis and distortion coefficients of the visible light camera. The intrinsic parameter matrix Defined as:

[0065]

[0066] in, The equivalent focal length of the visible light camera in the x-direction of the image coordinate system (unit: pixels); The equivalent focal length of the visible light camera in the y-direction of the image coordinate system (unit: pixels); The x-component of the principal point coordinates of the visible light camera (unit: pixels); The y-component of the principal point coordinates of the visible light camera (unit: pixels). , It is the radial distortion coefficient of a visible light camera, used to correct the radial distortion caused by the lens shape. It is the tangential distortion coefficient for visible light cameras, used to correct distortions that are consistent with the lens tangent direction caused by lens manufacturing and installation errors.

[0067] The infrared camera calibration uses a high-emissivity coating on a composite calibration plate (100). Under the same three-dimensional translation stage trajectory (108) as the visible light calibration, image sets are acquired from N different viewpoints. A nonlinear optimization algorithm is used to solve for the intrinsic parameter matrix of the infrared camera. and distortion coefficients Dir. The intrinsic parameter matrix Kir is defined as:

[0068]

[0069] in, The equivalent focal length of the infrared camera in the x-direction of the image coordinate system (unit: pixels); The equivalent focal length of the infrared camera in the y-direction of the image coordinate system (unit: pixels); The x-component of the principal point coordinates of the infrared camera (unit: pixels); The y-component of the principal point coordinates of the infrared camera (unit: pixels). This is the radial distortion coefficient of the infrared camera, used to correct the radial distortion caused by the lens shape. This is the tangential distortion coefficient for infrared cameras, used to correct distortions that occur in the same direction as the lens tangent due to lens manufacturing and installation errors.

[0070] The system joint calibration uses a hand-eye calibration method to solve the relative pose relationship between the visible light camera and the infrared camera. Using the synchronized observation image set (103) of the composite calibration board (100) acquired during the visible light camera calibration and the infrared camera calibration, multiple pairs of physical points with fixed spatial constraints are obtained. Each pair of physical points contains two physical points with a defined three-dimensional spatial relationship: one is a checkerboard corner point P on the front of the composite calibration board (100). f The other is the center point P of the micro-frustum-shaped depression on its back. b The relative position vector d of the two points in the calibration plate coordinate system is a known constant. Based on the camera intrinsic parameters and corresponding image coordinates obtained from calibration, and combined with the known three-dimensional structure of the composite calibration plate (100), point P is calculated using the perspective n-point (PnP) method. f The three-dimensional coordinates P in the visible light camera coordinate system (104) vis and point P b The three-dimensional coordinates P in the infrared camera coordinate system (105) ir Using multiple sets of corresponding ( , Establish spatial transformation models R and T between the two camera coordinate systems, with the following formulas:

[0071] Among them, P vis Let Pf be the three-dimensional coordinates of point Pf in the visible light camera coordinate system (unit: millimeters); P ir Let P be the point b The three-dimensional coordinates (unit: mm) in the infrared camera coordinate system; R is a 3×3 rotation matrix describing the rotation transformation from the infrared camera coordinate system to the visible light camera coordinate system; T is a 3×1 translation vector (unit: mm) describing the translation transformation from the infrared camera coordinate system to the visible light camera coordinate system.

[0072] The non-uniformity parameter was calibrated using a standard blackbody radiation source (106) at high temperature. and low temperature Multiple frames of uniform field infrared images were acquired at two temperature points (107), and the gain correction coefficient G(x,y) and offset correction coefficient B(x,y) for each pixel were calculated:

[0073]

[0074] in, and Each pixel High temperature and low temperature The average grayscale value of multiple frames of images; For pixels Gain correction factor (unit: gray level / ); For pixels Offset correction coefficient (unit: gray level).

[0075] Step 2, as follows Figure 3 As shown, a detection path is planned based on the three-dimensional geometric model of the aircraft and safety distance constraints. According to the three-dimensional geometric model of the aircraft to be inspected, a full-coverage detection path that satisfies safety constraints is planned to guide automated equipment in orderly and comprehensive image acquisition of the entire surface structure of the aircraft. The planning of the full-coverage detection path first calculates the optimal working distance range [D] of the dual-light camera to achieve clear imaging of the entire surface structure. min D maxThen, using this as a constraint, a Boustrophedon coverage path planning algorithm is adopted to generate an initial detection path (200) on the surface of the 3D model that meets the requirement of 70% planning overlap rate, ensuring complete coverage of the entire machine surface structure; finally, a cubic B-spline curve is used to smooth and optimize the path, improving the smoothness of the path while maintaining the coverage effect, ensuring the stability of the detection equipment movement and the image acquisition quality, and obtaining the smoothed and optimized path (201). The optimal working distance interval [D] min D max The imaging requirements and safety constraints must be met simultaneously. The specific calculation method is as follows: First, calculate the minimum theoretical distance D. min_theory D min_theory =The closest focusing distance specified by the lens to ensure image sharpness.

[0076] Then calculate the maximum theoretical distance D. max_theory This ensures that the image resolution meets the requirements for damage detection.

[0077] in: Camera focal length; : The dimensions of the object being measured (damaged); The maximum ground sampling distance required to identify this damage; Then, take a safe distance D. secure =0.5-1 meter. This value is a safety margin set by taking into account factors such as the motion control accuracy of the detection equipment, external environmental disturbances, and structural shape and position deviations, to ensure that the detection equipment maintains a safe distance from the surface of the aircraft and does not interfere with any protruding structures.

[0078] Finally, according to , and The relationship determines the final working distance range: if ,but , .

[0079] if ,but , .

[0080] if This indicates that the current lens configuration cannot meet the imaging requirements at a safe distance, and the equipment parameters need to be adjusted or a new lens needs to be selected.

[0081] The planned overlap rate refers to the lateral overlap rate: that is, the overlap rate between adjacent parallel path acquisition images (202) in the direction perpendicular to the detection path. It is guaranteed by controlling the spatial distance between adjacent parallel path segments, and its calculation formula is:

[0082] in: This refers to the lateral overlap rate. The distance between adjacent parallel paths. Width of single-frame image coverage on the object plane Step 3, as follows Figure 4 As shown, hardware synchronously triggered dual-light image sequence acquisition. The automated equipment (301) equipped with a dual-light camera (102) is controlled to acquire images along the detection path planned in step 2, obtaining visible light image sequences and infrared image sequences of the aircraft surface for subsequent 3D reconstruction and damage identification. The automated equipment adopts a multi-degree-of-freedom parallel mechanism, possessing six degrees of freedom motion capability, enabling precise control of the acquisition perspective. During the image acquisition process, the movement speed and acquisition frame rate of the automated equipment are controlled to ensure a 70% acquisition overlap rate. The acquisition overlap rate refers to the forward overlap rate: that is, the overlap rate between two adjacent frames (302) along the detection path forward direction, and its calculation formula is:

[0083] in: For heading overlap, For the forward speed of the motion platform, For image acquisition frame rate, The single-frame image coverage length on the object plane The dual-light camera includes a hardware synchronization trigger module to ensure strict synchronization of visible light and infrared image acquisition. Both the visible light and infrared image sequences are embedded with precise timestamps and location encoding information, providing complete metadata support for subsequent data processing.

[0084] Step 4, as follows Figure 5As shown, geometric distortion and non-uniformity correction are performed on visible light and infrared images. The acquired visible light image sequence (401) and infrared image sequence (400) are preprocessed, including distortion correction and non-uniformity correction, to eliminate systematic errors. First, distortion correction is performed on the visible light image sequence (401) and infrared image sequence (400) to obtain the distorted visible light image sequence (404) and infrared image sequence (402). Then, non-uniformity correction is performed on the distorted infrared image sequence (402) to obtain the non-uniformly corrected infrared image sequence (403). The distortion correction is performed using the Brown-Conrady lens distortion correction model to correct the geometric distortion of the visible light image sequence (401) and infrared image sequence (400). The correction model is expressed as:

[0085]

[0086] in: and Represents the coordinates of the original image; and Indicates the corrected image coordinates; , , Indicates the radial distortion coefficient; , Indicates the tangential distortion coefficient; The radial distance to the image center is represented by the formula: .

[0087] The non-uniformity correction, based on the gain correction coefficient G(x,y) and offset correction coefficient B(x,y) obtained in step one, uses a two-point correction method to perform pixel-level response compensation on the distortion-corrected infrared image sequence (402). The correction formula is as follows:

[0088] in For the original infrared image in pixels grayscale value at that location This is the offset correction coefficient for that pixel. This is the gain correction coefficient for that pixel. This is the corrected grayscale value.

[0089] Step 5, as follows Figure 6As shown, 3D Gaussian splash reconstruction and 3D mesh model generation are performed based on visible light images. Based on the distortion-corrected visible light image sequence (404), a high-precision 3D scene model of the aircraft surface (508) is constructed using the 3D Gaussian splash reconstruction method, and the 3D spatial pose information P of each visible light image in the global coordinate system is output. vis =[ | This method is used to solve the spatial pose of infrared images and provides a 3D model basis for the spatial mapping and visualization of damage markers. The 3D Gaussian splash reconstruction method includes scene initialization and parameterization, covariance parameterization, differential optimization, and 3D mesh model generation.

[0090] The scene initialization and parameterization first involve processing the sparse point cloud (501) obtained from the visible light image sequence using motion reconstruction technology and the initial camera pose P'. vis= [ | Subsequently, each 3D point in the sparse point cloud is initialized as a 3D Gaussian function (502), and its core parameters are defined for each function: the position parameter μ ∈ R³ determines its spatial position, and the covariance matrix Σ ∈ R³ x ³ It controls the initial shape and spatial orientation. Finally, it outputs an initial 3D Gaussian scene representation composed of a set of parameterized Gaussian functions, providing a high-precision three-dimensional coordinate mapping basis for the intelligent recognition of damaged areas.

[0091] The covariance parameterization decomposes the covariance matrix Σ (503) into a rotation matrix R (504) and a scaling matrix S (505), the mathematical expression of which is: Σ = RSS T R T By optimizing the rotation and scaling parameters respectively, a refined reconstruction of the complex geometric features such as curved surfaces and edges of the aircraft's overall surface structure can be achieved.

[0092] The differential optimization involves projecting a 3D Gaussian scene onto a 2D image plane using a differentiable Gaussian splash renderer to generate a composite view (506). All Gaussian function parameters and camera pose are iteratively optimized by minimizing the difference between the composite view (506) and the corresponding real visible light image (404). The composite loss function used is: L = (1 - λ)L1+λL D-SSIM Where L1 is the L1 loss, L D-SSIMλ represents the structural similarity loss and the balancing weight. During optimization, an adaptive density control strategy is employed, performing cloning, splitting, and pruning operations on the Gaussian function based on its position gradient and transparency (507) to improve the reconstruction capability of structural details. Through this optimization process, the accurate camera pose P in the global coordinate system is obtained. vis .

[0093] The generation of the 3D mesh model involves obtaining a 3D scene model (508) composed of an optimized set of 3D Gaussian functions after differential optimization. To achieve subsequent texture mapping, a 3D mesh model (509) that maintains the original geometric accuracy is generated from the optimized set of 3D Gaussian functions using a Poisson surface reconstruction algorithm, serving as the unified spatial basis for subsequent damaged 3D mapping. The 3D mesh model (509) is a watertight triangular mesh model, which possesses regular topological structure, no self-intersections, and no holes, ensuring the stability and accuracy of subsequent texture projection and fusion processes.

[0094] Step 6: Global Spatial Pose Calculation of Infrared Images Based on Spatial Transformation Model. Using the spatial transformation model R and T obtained in Step 1 and the accurate camera pose Pvis of each visible light image obtained from the 3D reconstruction in Step 5, calculate the spatial pose P of each infrared image in the global coordinate system. ir Finally, a spatial pose dataset of infrared image sequences is obtained, which is used to map the identified damage from the two-dimensional infrared images to the surface of a three-dimensional model, thereby achieving three-dimensional localization of the damage. The spatial pose P of the infrared images in the global coordinate system is... ir =[ | The calculation formula is as follows:

[0095]

[0096] in: For the transition from the world coordinate system to the visible light camera coordinate system Rotation matrix; For the transition from the world coordinate system to the visible light camera coordinate system Translation vector (unit: millimeters); and The rotation matrix and translation vector from the infrared camera coordinate system to the visible light camera coordinate system obtained in step 1; Rotation matrix The inverse matrix represents the rotational transformation from the visible light camera coordinate system to the infrared camera coordinate system; and This refers to the desired pose parameters from the world coordinate system to the infrared camera coordinate system. Step 7, as follows Figure 7 As shown, pixel-level damage segmentation mask generation is based on an encoder-decoder architecture. First, the non-uniformity-corrected infrared image sequence (403) is standardized to adjust each image to a fixed size of M×N pixels suitable for the segmentation network, where M=N=512. Then, the standardized infrared image sequence (703) is input into the damage intelligent recognition model (704) based on the encoder-decoder architecture to obtain the original damage segmentation mask (705) of P×Q pixels output by the model, where P=Q=512. Finally, a size restoration operation is performed on the output original damage segmentation mask to generate an original-size damage segmentation mask (707) aligned with the original input image space, which is used for subsequent damage visualization rendering.

[0097] The standardization process employs a fill scaling method that maintains the image aspect ratio. Specifically, it involves calculating the original image height H and width W, determining the scaling ratio scale = 512 / max(H, W), scaling the non-uniformity-corrected infrared image sequence (403) according to this ratio, and obtaining a scaled image sequence (702) with height H' = round(H * scale) and width W' = round(W * scale). Then, pixel padding is performed on both sides or top and bottom of the scaled image sequence to obtain a standardized infrared image sequence (703) with an image size of 512 × 512 pixels. The fill pixel value is a specific value -1 that is significantly different from the effective image area, used to identify invalid areas in subsequent network processing. During this process, the scaling ratio scale and the horizontal fill amount are recorded. = (512 - W') / 2 and vertical fill amount = (512 - H') / 2 is used as a transformation parameter for subsequent size restoration operations.

[0098] The damage intelligent recognition model (704) is a trained deep convolutional neural network with an encoder-decoder architecture. The model takes a standardized infrared image as input; the encoder consists of multiple downsampling stages, which are used to extract and compress multi-scale abstract features from the input image layer by layer; the decoder consists of corresponding upsampling stages, which gradually restore the spatial resolution of the feature maps by fusing the feature maps of each layer of the encoder, and finally outputs a 512×512 pixel single-channel segmentation mask, where the integer value of each pixel is used to uniquely identify the damage category corresponding to that location: 0 represents the background, 1 represents cracking, 2 represents debonding, and 3 represents surface ablation.

[0099] The size restoration operation employs an inverse mask transformation method corresponding to the normalization process. This method first uses the recorded pad... x and pady The padding region in the normalized segmentation mask is removed to obtain an intermediate mask (706) of size (H', W'), which corresponds to the scaled original image content. This intermediate mask is then restored to the original input image size (H, W). The restoration process is achieved by establishing a coordinate mapping from the target size (H, W) to the intermediate size (H', W'): for any integer coordinate position (x...) in the target mask... dst , y dst ), through the mapping relationship x src =x dst ·(W' / W) and y src = y dst · (H' / H) Calculate its corresponding source coordinates (x) in the intermediate mask. src ,y src ), and use the nearest neighbor interpolation algorithm to calculate the distance (x) src , y src The category label value of the nearest integer coordinate pixel is assigned to the target pixel (x). dst , y dst The transformation is completed by traversing all target pixels, and finally a damage segmentation mask (707) is generated that is completely consistent with the spatial resolution of the original input image and is pixel-level aligned.

[0100] Step 8, as follows Figure 8 As shown, the damage segmentation mask and the original infrared image are fused and rendered. The damage segmentation mask (707) output in step 7 is fused with the corresponding non-uniformly corrected infrared image (403) to generate a synthetic diagnostic image (801) with colored damage markers, which is used for spatial mapping and three-dimensional positioning. The fusion process uses the original infrared image as the background substrate; according to the preset color mapping relationship, the pixel areas of different damage categories in the mask are rendered with corresponding specific colors, where the background is transparent, cracking is green, debonding is red, and surface ablation is blue; through channel duplication, the single-channel original infrared image is converted into a three-channel image; using an image overlay algorithm, the colored damage marker area is covered with the corresponding position of the background image with a transparency parameter of 50%-80%, and finally a synthetic diagnostic image that can intuitively display the damage distribution, geometric contour and category attributes is formed. The synthetic diagnostic image contains two information layers: the bottom layer is the single-channel original infrared grayscale image, which retains its original infrared radiation information; the upper layer is a colored damage marker layer generated based on the damage segmentation mask, which can characterize the contour of the damage area and the damage category. The image overlay algorithm uses the α-mixing algorithm, and its mathematical expression is:

[0101] in, The output is a synthetic diagnostic image. For color-marked damage images, This is a three-channel infrared image obtained by channel replication. For transparency parameters, For RGB color channel indexing. The channel copying is performed by copying the original single-channel infrared image. The grayscale value of each pixel at position (x, y) is simultaneously assigned to the three-channel image. The conversion is achieved through the R, G, and B channels, and their conversion relationship is expressed as follows: = = =

[0102] in, This represents a three-channel infrared image obtained through channel replication. Represents the image pixel coordinates, where R, G, and B represent the red, green, and blue color channels, respectively. Raw single-channel infrared image.

[0103] Step 9, as follows Figure 9 As shown, a three-dimensional damage marking model is generated based on texture projection and multi-view fusion. According to the spatial pose dataset of the infrared image sequence established in step 6, the synthetic diagnostic image (801) output in step 8 is mapped to the corresponding surface area of ​​the three-dimensional mesh model (509) generated in step 5, and finally a three-dimensional infrared damage marking model with both three-dimensional geometry and damage marking information is generated, so as to realize the accurate positioning and three-dimensional visualization of the surface structural damage of the whole machine in three-dimensional space.

[0104] The mapping process includes texture projection and multi-view fusion.

[0105] The texture projection establishes a texture coordinate correspondence between the mesh and the synthesized diagnostic image based on the 3D coordinates of each vertex of the 3D mesh model and the infrared image pose Pir obtained in step 6, using a perspective projection model. For any point Pw on the mesh surface, its projected coordinates (u,v) on the i-th synthesized diagnostic image are determined by the following projection relationship:

[0106] in: is the projection depth factor (dimensionless). The homogeneous coordinates of points on the 3D network surface in the global coordinate system (unit: millimeters); For the first The camera pose matrix corresponding to the infrared image, where for Rotation matrix, for Translation vector (unit: millimeters) This is the intrinsic parameter matrix of the infrared camera, defined in step 1.

[0107] The multi-view fusion process involves calculating fusion weights based on the viewing angle θ and camera distance d from multiple synthetic diagnostic images (901) covering the same grid area. Weighted fusion is then used to eliminate stitching artifacts, generating a seamless panoramic infrared texture map, which is then bound to the 3D mesh model, ultimately producing a complete 3D infrared damage marking model. The fusion weight calculation formula is as follows:

[0108] in: The angle between the surface normal vector and the camera viewing direction (unit: radians). The Euclidean distance from the camera's optical center to the surface electric field (unit: millimeters); The normalized fusion weights have a value range of [value range missing]. .

Claims

1. A rapid detection method for surface structural damage of an aircraft based on infrared radiation, characterized in that, First, rapid global acquisition of infrared images is achieved through automated equipment. Second, a high-precision three-dimensional mesh model is established based on multi-view visible light images, and the spatial pose of the infrared images is solved to construct an infrared image dataset with three-dimensional coordinate information. On this basis, deep learning algorithms are used to achieve intelligent identification of structural damage, automatically identifying and marking damage types and regions. Finally, spatial mapping technology is used to achieve precise three-dimensional localization of damage, and the identification results are accurately projected onto the surface of the three-dimensional mesh model to generate a three-dimensional damage marking model containing the spatial distribution of damage.

2. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, The rapid detection method is specifically as follows: Step 1: Calibration of a dual-light camera calibration system based on a composite calibration plate and a blackbody: Construct a dual-light camera calibration system, which includes a composite calibration plate, a standard blackbody radiation source, a three-dimensional translation stage for precisely controlling the relative pose of the calibration plate and the camera, and a data processing unit for executing calibration algorithms and parameter calculations; Perform visible light camera calibration, infrared camera calibration, joint calibration, and non-uniformity parameter calibration on the dual-light camera, and establish a unified spatial coordinate relationship between the infrared image and the visible light image; Step 2, Detection path planning based on the 3D geometric model of the aircraft and safety distance constraints: Based on the 3D geometric model of the aircraft to be detected, plan a full-coverage detection path that meets the safety constraints; Step 3, Hardware-synchronized dual-light image sequence acquisition: Control the automated equipment equipped with dual-light cameras to acquire images along the detection path planned in Step 2, and obtain visible light image sequence and infrared image sequence of the aircraft surface for subsequent 3D reconstruction and damage identification. Step 4: Geometric distortion and non-uniformity correction of visible light and infrared images: Preprocess the acquired raw visible light and infrared images, including distortion correction and non-uniformity correction, to eliminate systematic errors; first, perform distortion correction on the visible light and infrared images to obtain distorted visible light and infrared images; then, perform non-uniformity correction on the distorted infrared images to obtain non-uniformity corrected infrared images. Step 5: 3D Gaussian Splash Reconstruction and 3D Mesh Model Generation Based on Visible Light Images: Based on the preprocessed visible light image sequence, a high-precision 3D scene model of the aircraft surface is constructed using the 3D Gaussian splash reconstruction method, and the 3D spatial pose information P of each visible light image in the global coordinate system is output. vis =[ | The method is used to solve the spatial pose of infrared images and provides a three-dimensional model basis for the spatial mapping and visualization of damage markers. The 3D Gaussian splash reconstruction method includes scene initialization and parameterization, covariance parameterization, differential optimization, and three-dimensional mesh model generation. Step 6: Global spatial pose calculation of infrared images based on the spatial transformation model: The precise camera pose P of each visible light image is calculated using the spatial transformation model R and T obtained in Step 1 and the 3D reconstruction obtained in Step 5. vis Calculate the spatial pose P of each infrared image in the global coordinate system. ir Finally, a spatial pose dataset of infrared image sequences is obtained, which is used to map the identified damage from the two-dimensional infrared image to the surface of the three-dimensional model to achieve three-dimensional localization of the damage. Step 7: Pixel-level damage segmentation mask generation based on encoder-decoder architecture: First, the infrared image sequence is standardized to adjust each image to a fixed size of 512×512 pixels suitable for the segmentation network; then, the standardized infrared image sequence is input into the damage intelligent recognition model based on encoder-decoder architecture to obtain the 512×512 pixel damage segmentation mask output by the model; finally, the size restoration operation is performed on the output damage segmentation mask to generate an original-size damage segmentation mask aligned with the original input image space. Step 8: Fusion rendering of damage segmentation mask and original infrared image: The damage segmentation mask output in step 7 is fused with the corresponding original infrared image to generate a synthetic diagnostic image with color damage markers. Step 9: Generation of a 3D damage marking model based on texture projection and multi-view fusion: Based on the spatial pose dataset of the infrared image sequence established in Step 6, the synthetic diagnostic image output in Step 8 is mapped to the corresponding surface area of ​​the 3D mesh model generated in Step 5, and finally a 3D infrared damage marking model with both 3D geometry and damage marking information is generated.

3. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, The substrate of the composite calibration plate in step 1 is made of metal. One side of the plate is a black and white checkerboard pattern for calibrating the geometric parameters of the visible light camera. The other side is a coated surface with an emissivity of 0.95-0.98 for infrared calibration. To achieve physical spatial uniformity of the feature points of the two cameras, a micro-cylindrical recess is machined at the position of the coated surface corresponding to each corner point of the black and white checkerboard pattern. The diameter of the recess is 0.5-1mm and the depth is 0.1-0.3mm. The center of the recess is aligned with the center of the corner point of the checkerboard pattern on the front side in three-dimensional space.

4. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, The visible light camera calibration in step 1 uses a black and white checkerboard pattern on a composite calibration plate. A three-dimensional translation stage is controlled to acquire image groups from N different viewpoints, where N is between 20 and 30. The Zhang Zhengyou calibration method is used to solve for the intrinsic parameter matrix of the visible light camera. and distortion coefficient The intrinsic parameter matrix Defined as: in, The equivalent focal length of the visible light camera in the x-direction of the image coordinate system, in pixels; The equivalent focal length of the visible light camera in the y-direction of the image coordinate system, in pixels; The x-component of the principal point coordinates of the visible light camera, in pixels; The y-component of the principal point coordinates of the visible light camera, in pixels; , The radial distortion coefficient of the visible light camera; denoted as the tangential distortion coefficient of a visible light camera.

5. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, The infrared camera calibration in step 1 involves: using a high-emissivity coating on a composite calibration plate, acquiring image sets from N different viewpoints under the same three-dimensional translation stage trajectory as the visible light calibration; and using a nonlinear optimization algorithm to solve for the intrinsic parameter matrix of the infrared camera. and distortion coefficient The intrinsic parameter matrix Defined as: in, The equivalent focal length of the infrared camera in the x-direction of the image coordinate system, in pixels; The equivalent focal length of the infrared camera in the y-direction of the image coordinate system, in pixels; The x-component of the principal point coordinates of the infrared camera, in pixels; The y-component of the principal point coordinates of the infrared camera, in pixels; The radial distortion coefficient of the infrared camera; denoted as the tangential distortion coefficient of the infrared camera.

6. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, The system joint calibration in step 1 involves: using a hand-eye calibration method to solve the relative pose relationship between the visible light camera and the infrared camera; and using the synchronized observation image sets of the two cameras on the composite calibration board acquired during the visible light camera calibration and the infrared camera calibration to obtain multiple pairs of physical points with fixed spatial constraints. Each pair of physical points contains two physical points with a defined three-dimensional spatial relationship: one is a checkerboard corner point P on the front of the composite calibration board. f The other is the center point P of the micro-frustum-shaped depression on its back. b The relative position vector d of the two points in the calibration plate coordinate system is a known constant. Based on the camera intrinsic parameters and corresponding image coordinates obtained from calibration, and combined with the known three-dimensional structure of the calibration plate, the three-dimensional coordinates P of point Pf in the visible light camera coordinate system are calculated using the perspective n-point method. vis and point P b The three-dimensional coordinates P in the infrared camera coordinate system ir ; Utilizing multiple sets of corresponding and Establish the spatial transformation models R and T between the two camera coordinate systems, with the following formulas: Among them, P vis Let P be the point f Three-dimensional coordinates in the visible light camera coordinate system, unit: millimeters; P ir Let P be the point b The three-dimensional coordinates in the infrared camera coordinate system, in millimeters; R is a 3×3 rotation matrix describing the rotation transformation from the infrared camera coordinate system to the visible light camera coordinate system; T is a 3×1 translation vector, in millimeters, describing the translation transformation from the infrared camera coordinate system to the visible light camera coordinate system.

7. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, The non-uniformity parameter calibration in step 1 is performed using a standard blackbody radiation source at high temperature. and low temperature Multiple frames of uniform field infrared images were acquired at two temperature points, and the gain correction coefficient G(x,y) and offset correction coefficient B(x,y) for each pixel were calculated: in, and Each pixel High temperature and low temperature The average grayscale value of multiple frames of images; For pixels Gain correction factor, unit: gray level / ; For pixels Offset correction coefficient, unit: gray level.

8. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, Step 2, planning the full-coverage detection path: First, calculate the optimal working distance range [D] for the dual-light camera to achieve clear imaging on the entire surface structure. min D max Then, using this as a constraint, a reciprocating scanning coverage path planning algorithm is adopted to generate an initial path on the surface of the 3D model that meets the requirement of 70% planning overlap rate; finally, cubic B-spline curves are used to smooth and optimize the path, improving the smoothness of the path while maintaining the coverage effect.

9. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 8, characterized in that, The optimal working distance range [D] min D max The imaging requirements and safety constraints must be met simultaneously; the specific calculation method is as follows: First, calculate the minimum theoretical distance D. min_theory D min_theory =The closest focusing distance specified by the lens; Then calculate the maximum theoretical distance D. max_theory ; in: Camera focal length; : The size of the damage to the object being measured; The maximum ground sampling distance required to identify this damage; Then, take a safe distance D. secure =0.5-1 meter, this value is a safety margin set comprehensively considering factors such as the motion control accuracy of the detection equipment, external environmental disturbances and structural form and position deviations, to ensure that the detection equipment maintains a safe distance from the surface of the aircraft and does not interfere with any protruding structures; Finally, according to , and The relationship determines the final working distance range: if ,but , ; if ,but , ; if This indicates that the current lens configuration cannot meet the imaging requirements at a safe distance, and the equipment parameters need to be adjusted or a new lens needs to be selected.

10. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 8, characterized in that, The planned overlap rate refers to the lateral overlap rate: that is, the overlap rate between images acquired by adjacent parallel paths in the direction perpendicular to the detection path. It is ensured by controlling the spatial distance between adjacent parallel path segments, and its calculation formula is as follows: in: This refers to the lateral overlap rate. The distance between adjacent parallel paths. The width of a single frame of image coverage on the object plane.

11. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, The automated equipment in step 3 adopts a multi-degree-of-freedom parallel mechanism with six degrees of freedom of motion capability, which can achieve precise control of the acquisition angle. During the image acquisition process, the 70% acquisition overlap rate is guaranteed by controlling the movement speed and acquisition frame rate of the automated equipment.

12. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 11, characterized in that, The acquisition overlap rate refers to the forward overlap rate: that is, the overlap rate between two adjacent frames of images along the detection path. Its calculation formula is as follows: in: For heading overlap, For the forward speed of the motion platform, For image acquisition frame rate, The length of a single frame image coverage on the object plane.

13. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 2, characterized in that, The dual-light camera includes a hardware synchronization trigger module; both the visible light and infrared image sequences are embedded with precise timestamps and location encoding information.

14. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, Step 4, distortion correction, involves performing geometric distortion correction on visible light and infrared images using the Brown-Conrady lens distortion correction model. The correction model is expressed as follows: in: and Represents the coordinates of the original image; and Indicates the corrected image coordinates; , , Indicates the radial distortion coefficient; , Indicates the tangential distortion coefficient; The radial distance to the image center is represented by the formula: .

15. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 7, characterized in that, Step 4, non-uniformity correction, is based on the obtained gain correction coefficient G(x,y) and offset correction coefficient B(x,y). A two-point correction method is used to perform pixel-level response compensation on the infrared image. The correction formula is as follows: in For the original infrared image in pixels grayscale value at that location This is the offset correction coefficient for that pixel. This is the gain correction coefficient for that pixel. This is the corrected grayscale value.

16. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, Step 5, scene initialization and parameterization, firstly involves processing the sparse point cloud obtained from the visible light image sequence using motion reconstruction technology and setting the initial camera pose P'vis=[ | Subsequently, each 3D point in the sparse point cloud is initialized as a 3D Gaussian function, and its core parameters are defined for each function: the position parameter μ ∈ R³ determines its spatial position, and the covariance matrix Σ ∈ R³ x ³ It controls the initial shape and spatial orientation; finally, it outputs an initial 3D Gaussian scene representation composed of a set of parameterized Gaussian functions, providing a high-precision three-dimensional coordinate mapping basis for the intelligent recognition results of the damaged area.

17. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, Step 5, covariance parameterization, involves decomposing the covariance matrix Σ into a rotation matrix R and a scaling matrix S. Its mathematical expression is as follows: Σ = RS ST RT By optimizing the rotation and scaling parameters respectively, a refined reconstruction of the complex geometric features such as curved surfaces and edges of the aircraft's overall surface structure can be achieved.

18. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, Step 5, differential optimization, involves projecting the 3D Gaussian scene onto a 2D image plane using a differentiable Gaussian splash renderer to generate a composite view. All Gaussian function parameters and camera pose are iteratively optimized by minimizing the difference between the composite view and the corresponding real visible light image. The composite loss function used is: L = (1 − λ) L1 + λ L D-SSIM Where L1 is the L1 loss, L D-SSIM The structural similarity loss is represented by λ, which is the balancing weight. During optimization, an adaptive density control strategy is employed, involving cloning, splitting, and pruning operations based on the position gradient and transparency of the Gaussian function to improve the reconstruction capability of structural details. Through this optimization process, the accurate camera pose P in the global coordinate system is obtained. vis .

19. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, Step 5, generating the 3D mesh model, involves obtaining a 3D scene model composed of an optimized set of 3D Gaussian functions after differential optimization. Using the Poisson surface reconstruction algorithm, a 3D mesh model that maintains the original geometric accuracy is generated from the optimized set of 3D Gaussian functions, serving as the unified spatial basis for subsequent 3D damage mapping. The 3D mesh model is a watertight triangular mesh model, which has the geometric characteristics of regular topological structure, no self-intersection, and no holes, ensuring the stability and accuracy of subsequent texture projection and fusion processes.

20. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, The spatial pose of the infrared image in step 6, Pir, is given by […]. | The calculation formula is as follows: in: For the transition from the world coordinate system to the visible light camera coordinate system Rotation matrix; For the transition from the world coordinate system to the visible light camera coordinate system Translation vector, unit: millimeters; and The rotation matrix and translation vector from the infrared camera coordinate system to the visible light camera coordinate system obtained in step 1; Rotation matrix The inverse matrix represents the rotational transformation from the visible light camera coordinate system to the infrared camera coordinate system; and This refers to the pose parameters obtained from the world coordinate system to the infrared camera coordinate system.

21. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, The standardization process in step 7 uses a fill scaling method that maintains the aspect ratio of the image. Specifically, it involves: calculating the original height H and width W of the image, and determining the scaling ratio scale = 512 / max(H, W); scaling the image according to this ratio to obtain a new height H' = round(H * scale) and width W' = round(W * scale); and then filling the image with pixels on both sides or top and bottom to make the final image size 512×512 pixels.

22. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 21, characterized in that, The pixel padding uses a specific value of -1, which is significantly different from the valid image area, to identify invalid regions in subsequent network processing; during this process, the scaling ratio (scale) and the horizontal padding amount are recorded. = (512 - W') / 2 and vertical fill amount = (512 - H') / 2 is used as a transformation parameter for subsequent size restoration operations.

23. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, The damage intelligent recognition model in step 7 is a trained deep convolutional neural network with an encoder-decoder structure. The model takes a standardized infrared image as input. The encoder consists of multiple downsampling stages, which are used to extract and compress multi-scale abstract features from the input image layer by layer. The decoder consists of corresponding upsampling stages, which gradually restore the spatial resolution of the feature maps by fusing the feature maps of each layer of the encoder, and finally outputs a 512×512 pixel single-channel segmentation mask. The integer value of each pixel is used to uniquely identify the damage category corresponding to the location: 0 represents the background, 1 represents cracking, 2 represents debonding, and 3 represents surface ablation.

24. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 22, characterized in that, The size restoration operation in step 7 employs an inverse mask transformation method corresponding to the normalization process: this method first uses the recorded pad... x and pad y The padding region in the normalized segmentation mask is removed to obtain an intermediate mask of size (H', W'), which corresponds to the scaled original image content. Then, this intermediate mask is restored to the original input image size (H, W). The restoration process is achieved by establishing a coordinate mapping relationship from the target size (H, W) to the intermediate size (H', W'): for any integer coordinate position (x...) in the target mask... ds t, y dst ), through the mapping relationship x src = x dst ·(W' / W) and y src = y dst · (H' / H) Calculate its corresponding source coordinates (x) in the intermediate mask. src , y src ), and use the nearest neighbor interpolation algorithm to calculate the distance (x) src , y src) The category label value of the nearest integer coordinate pixel is assigned to the target pixel (x). dst , y dst The transformation is completed by traversing all target pixels, ultimately generating a damage segmentation mask that is pixel-aligned and has the same spatial resolution as the original input image.

25. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, Step 8, the fusion process, uses the original infrared image as the background substrate. According to the preset color mapping relationship, the pixel areas of different damage categories in the mask are rendered as corresponding specific colors, where the background is transparent, cracks are green, debonding is red, and surface ablation is blue. Through channel duplication, the single-channel original infrared image is converted into a three-channel image. Using an image overlay algorithm, the colored damage marking area is covered with the corresponding position of the background image with a transparency parameter of 50%-80%, and finally a synthetic diagnostic image that can intuitively display the damage distribution, geometric contours, and category attributes is formed.

26. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, The diagnostic image synthesized in step 8 comprises two information layers: the bottom layer is a single-channel original infrared grayscale image, preserving its original infrared radiation information; the upper layer is a color damage marker layer generated based on damage segmentation masks, capable of characterizing the contour of the damage region and the damage category; the image overlay algorithm employs the α-mixing algorithm, the mathematical expression of which is: in, The output is a synthetic diagnostic image. For color-marked damage images, This is a three-channel infrared image obtained by channel replication. For transparency parameters, For RGB color channel indexing; the channel copying is performed by copying the original single-channel infrared image. The grayscale value of each pixel at position (x, y) is simultaneously assigned to the three-channel image. The conversion is achieved through the R, G, and B channels, and their conversion relationship is expressed as follows: = = = in, This represents a three-channel infrared image obtained through channel replication. Represents the image pixel coordinates, where R, G, and B represent the red, green, and blue color channels, respectively. Raw single-channel infrared image.

27. The rapid detection method for surface structural damage of an aircraft based on infrared radiation according to claim 1, characterized in that, The mapping process in step 9 includes texture projection and multi-view fusion; The texture projection is based on the 3D coordinates of each vertex of the 3D mesh model and the infrared image pose Pir=[ | The texture coordinate correspondence between the mesh and the synthetic diagnostic image is established through a perspective projection model; for any point Pw on the mesh surface, its projected coordinates (u,v) on the i-th synthetic diagnostic image are determined by the following projection relationship: in: The projection depth factor; The homogeneous coordinates of points on the 3D network surface in the global coordinate system, in millimeters; For the first The camera pose matrix corresponding to the infrared image, where for Rotation matrix, for Translation vector, unit: millimeters This is the intrinsic parameter matrix of the infrared camera; The multi-view fusion process calculates fusion weights based on the viewing angle θ and camera distance d for multiple synthetic diagnostic images covering the same grid area. Weighted fusion eliminates stitching artifacts, generating a seamless panoramic infrared texture map, which is then bound to a 3D mesh model, ultimately producing a complete 3D infrared damage marking model. The fusion weight calculation formula is as follows: in: The angle between the surface normal vector and the camera's viewing direction, in radians; The Euclidean distance from the camera's optical center to the surface electric field is expressed in millimeters. The normalized fusion weights have a value range of [value range missing]. .

Images