Infrared nondestructive testing lamp shadow processing method based on target guidance

Abstract

The invention discloses an infrared nondestructive testing lamp shadow processing method based on target guidance, which is used for repairing artifacts or highlight areas caused by high reflection or direct light in acquired infrared image data and performing defect detection, and comprises the following steps: S1, acquiring an infrared image, and performing tensor feature processing; s2, based on spatial position information guidance, lamp shadow area target positioning is carried out on the infrared image after tensor feature processing is completed, and a lamp shadow mask image is generated; s3, inputting the lamp shadow mask image and the infrared image into a Fourier convolution feature extraction lamp shadow repair network based on a target guidance mechanism to obtain a lamp shadow repair image; and S4, inputting the lamp shadow repair image into a defect detection network for nondestructive defect detection. The method is compatible with various types of test pieces, high in adaptability, high in processing speed and high in detection precision, and can be widely applied to structural health monitoring and defect analysis in the fields of aerospace, rail transit, wind power and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of infrared nondestructive testing, and in particular to an infrared nondestructive testing lamp shadow processing method based on target guidance. BACKGROUND

[0002] With the wide application of high-performance composites in various industrial equipment, the development of structural nondestructive testing technology is particularly critical. As a technique for evaluating defects using temperature field response, infrared thermal imaging detection has been widely used in practical engineering applications due to its advantages of non-contact, high efficiency, visualization, etc. The basic principle of infrared nondestructive testing is to use external excitation sources (such as halogen lamps, lasers or electric heaters) to apply transient or periodic heating to the surface of the measured object, and then collect the thermal response image of the material surface through an infrared thermal imager, and analyze the temperature distribution difference to determine whether there are cracks, delamination, cavities or other defects inside.

[0003] However, in actual detection process, when the thermal reflectivity of the measured object is high or the surface geometry is complex, the strong reflected light generated by the excitation source will be received by the infrared camera, forming a so-called "lamp shadow" artifact. This kind of artifact usually has strong contrast, high brightness, and symmetrical distribution, which can cover the real defect information, leading to missed detection, false detection and even misjudgment, and seriously interfering with the quantitative evaluation of defects.

[0004] Current research methods for lamp shadow problems mainly focus on the following categories:

[0005] (1) Image enhancement and filtering methods: such as histogram equalization, high-pass filtering, wavelet transform, etc. These methods can improve the image contrast to some extent, but are prone to loss of image details and cannot distinguish between real defects and lamp shadows.

[0006] (2) Physical modeling methods: remove optical reflection by modeling light source distribution and material reflection characteristics, but model establishment is difficult, calculation is complex, and adaptability is poor.

[0007] (3) Deep learning repair methods: have been gradually applied in the field of image repair in recent years, but there is no special model system for infrared lamp shadow processing at present, especially in the trade-off between defect preservation and artifact elimination.

[0008] Therefore, there is an urgent need for an infrared image lamp shadow interference processing method that can maximize the preservation of defect thermal response characteristics based on the identification of lamp shadow areas, and improve the quality of infrared images and subsequent defect recognition capabilities. SUMMARY

[0009] The purpose of the present application is to overcome the shortcomings of the prior art and provide an infrared nondestructive testing lamp shadow processing method based on target guidance.

[0010] The object of the application is achieved by the following technical solutions:

[0011] In a first aspect, the application provides an infrared non-destructive detection lamp shadow processing method based on target guidance, which repairs and detects defects in the pseudo shadow or highlight area caused by high reflection or direct light in the collected infrared image data, and includes the following steps:

[0012] S1: Collecting infrared images and performing tensor feature processing;

[0013] S2: Based on the spatial position information guidance, the lamp shadow area target positioning is performed on the infrared image after tensor feature processing, and a lamp shadow mask image is generated;

[0014] S3: The lamp shadow mask image and the infrared image are input into the Fourier convolution feature extraction lamp shadow repair network based on the target guidance mechanism to obtain a lamp shadow repair image;

[0015] S4: The lamp shadow repair image is input into the defect detection network for non-destructive defect detection.

[0016] Further, step S1 includes the following sub-steps:

[0017] S11: Apply periodic thermal excitation or pulse thermal excitation to the surface of the material to be detected through an externally controllable photo-thermal excitation system;

[0018] S12: Collect time series of infrared images using an infrared thermal imager, and construct a multi-frame image matrix containing thermal diffusion space-time change information as input for subsequent processing;

[0019] S13: Use PCA algorithm or independent component analysis algorithm or matrix tensor decomposition algorithm of L4 algorithm to weaken the noise interference of thermal distribution image sequence, and obtain the infrared image after tensor feature processing.

[0020] Further, step S2 includes the following sub-steps:

[0021] S21: Priori positioning of potential lamp shadow area based on the excitation source layout of the photo-thermal excitation system;

[0022] S22: Image enhancement and coarse segmentation processing, including the following sub-steps:

[0023] S221: Image enhancement processing is performed on the candidate region defined in step S21, including histogram equalization, local contrast enhancement and edge preserving filter;

[0024] S222: Dynamic delimitation of high reflection area boundary is performed using adaptive threshold segmentation algorithm, and the connected domain set corresponding to the lamp shadow area is extracted;

[0025] S23: Multi-constraint refined shadow region identification, the connected domain set extracted in S222 is subjected to spatial geometric consistency constraint screening to obtain a generated shadow mask image map; wherein the spatial geometric consistency constraint screening includes the following sub-steps:

[0026] S231: Horizontal and vertical spacing constraint:

[0027] The constraint condition is set as and respectively limit the minimum spacing of the transverse distribution and the longitudinal distribution of the connected domain, and remove high-density false points; wherein W is the image width, H is the image height, and respectively represent the transverse and longitudinal distance threshold factors, which are determined by KS test, represents the distance between the i-th and j-th region center points in the horizontal direction, represents the distance between the i-th and j-th region center points in the vertical direction, respectively represent the horizontal coordinate values of the i-th and j-th region center points in the image coordinates, respectively represent the vertical coordinate values of the i-th and j-th region center points in the image coordinates, i and j represent the region index, and c represents the subscript identifier of the center point;

[0028] S232: Horizontal and vertical alignment constraint:

[0029] Calculate the horizontal and vertical coordinate deviations of the connected domain center, and set the maximum allowed deviation threshold to remove misaligned regions;

[0030] S233: Area consistency constraint of the region:

[0031] According to the prior setting of the size of the shadow region, the upper and lower limits are set to remove large or small interference targets, specifically: in the horizontal direction , in the vertical direction ; wherein is the pixel area of the connected domain , is the horizontal candidate set that meets the horizontal spacing constraint condition in S231, is the vertical candidate set that meets the vertical spacing constraint condition in S231, , represents the connected domain pair in the horizontal direction, represents the vertical connected domain pair, respectively represent the pixel area of each connected domain;

[0032] ​​​​​S234: Verification of the geometric intersection of the centerlines:

[0033] An intersection relationship judgment model is established using the vectors connecting the center points of regions to filter out regions that do not meet the symmetry condition; assuming that the optimal result is obtained by filtering the horizontally connected components in step S233 ( , () center line Parameterized as: ,in Indicates the line connecting the center points of the horizontally connected domain. Indicates a connection any point on, ( , ) indicates a region The center coordinates, ( , ) indicates a region center coordinates This represents the normalized interpolation parameters of the horizontal centerline between two points; in step S233, the vertical connected components are used to select the optimal result. , () center line Parameterized as: ,in Represents the line connecting the center points of the vertically connected domain. Indicates a connection any point on, ( , ) indicates a region The center coordinates, ( , ) indicates a region center coordinates This represents the normalized interpolation parameters of the longitudinal centerline between two points; the intersection point is calculated using the vector cross product. ,when When intersecting, and These represent parameterized paths. and The index of the intersection point.

[0034] Furthermore, in step S21, when inspecting a planar structural specimen, a central region division method is adopted; based on the axisymmetric physical characteristics of the handheld optical excitation device, a spatial mapping relationship between the image coordinate system and the excitation source is established; the infrared image size is defined as H×W, i.e., height × width, and the boundary coordinates of the central positioning region R are calculated:

[0035]

[0036] In the formula and is the center region size relative to the image height-width;

[0037] When detecting the special-shaped structure test piece, a target detection network based on deep learning is used to position the lamp shadow area target.

[0038] Further, the Fourier convolution feature extraction lamp shadow repair network based on the target guidance mechanism in step S3 comprises:

[0039] The gated convolution down-sampling module: after the infrared image and the lamp shadow mask lamp shadow mask image are spliced in the channel dimension, the spliced image is input into the gated convolution down-sampling module, and the spliced image is down-sampled to obtain an initial feature map , which has a three-dimensional tensor with , wherein is the number of channels, and respectively represent the height and width of the spliced feature map, and for each channel, the two-dimensional time domain signal thereof is denoted as , wherein and respectively represent the row index and column index in the spatial domain;

[0040] The parallel feature extraction module: receives the initial feature map output by the gated convolution down-sampling module, and inputs in parallel to extract global frequency domain information of a fast Fourier convolution module and local spatial information of a conventional convolution module, so as to jointly extract multi-scale spatial context features;

[0041] The global-local feature fusion module: element-wise adds the global frequency domain information and the local spatial information in the channel or spatial position to obtain preliminary fusion features: ; then applies batch normalization BN and ReLU activation to the preliminary fusion features in sequence to obtain fused features: ;

[0042] The skip connection and feature enhancement module: directly adds the shallow features saved in the down-sampling process of the gated convolution down-sampling module to the fused features to obtain enhanced features : ;

[0043] The gated convolution up-sampling module: restores the enhanced features to the original image resolution through gated deconvolution to obtain a lamp shadow repair image.

[0044] Further, the fast Fourier convolution module in the parallel feature extraction module comprises:

[0045] The two-dimensional discrete Fourier transform is performed on to obtain a frequency domain representation , wherein , are a frequency domain row index and a frequency domain column index, respectively;

[0046] After the transformation, a set of learnable frequency domain convolution kernels are multiplied element by element to obtain a convolution result ;

[0047] Subsequently, the inverse Fourier transform is used to restore to the time domain to generate the feature of the branch output of the fast Fourier convolution module, that is, the global frequency domain information , and the expression is as follows:

[0048]

[0049] The regular convolution module in the parallel feature extraction module comprises:

[0050] A series of learnable spatial convolution kernels and biases are applied to the initial feature map , denoted as .

[0051] Further, the gated convolution up-sampling module comprises:

[0052] S351: Gated weight generation:

[0053]

[0054] In the formula, and are a learnable convolution kernel and a bias, respectively; is a Sigmoid activation, and the output is a gating mask ;

[0055] S352: Deconvolution up-sampling:

[0056]

[0057] wherein Deconv is a normal deconvolution operator or a transpose convolution operator, and the output is ;

[0058] S353: Gated fusion:

[0059]

[0060] In the formula, the up-sampled feature with dynamic weights ;

[0061] S354: Iteration layer by layer:

[0062] Repeat the above process, upsample multiple times until the original image size is restored, and finally output the repair result .

[0063] Further, the Fourier convolution feature extraction lamp shadow repair network based on the target guidance mechanism is trained through a multi-loss function target joint optimization mechanism, specifically including:

[0064] Generator Receive mask condition input, that is, infrared image And lamp shadow mask image , output lamp shadow repair image ; Discriminator Adopt PatchGAN structure, and give the probability distribution of the authenticity of the input image by discriminating the local image area;

[0065] The training target is to make The generated As much as possible The discriminator is true, while retaining the similarity of the original infrared image without lamp shadow In the pixel and perceptual feature level; pixel reconstruction loss, adversarial loss, feature matching loss and high receptive field perceptual loss are used for training.

[0066] Further, the pixel reconstruction loss , ensures that The known lamp shadow-free area is consistent with the real image ;

[0067] The adversarial loss Includes discriminator loss And generator loss , described as , wherein Is the gradient penalty, Indicates the weight coefficient of the generator gradient smoothing loss term, Indicates the expectation or batch mean of the sample, Indicates the probability distribution of the discriminator output that the image Is a real image, Indicates the gradient operation of the discriminator output on the image , used for generator smoothing loss;

[0068] The generator loss Is:

[0069] ;

[0070] feature matching loss including the discrimination between the discriminators based on true and false samples loss, denoted as:

[0071]

[0072] in the formula, is the output of the intermediate layer of the discriminator, is the number of intermediate layers, denotes the original undamaged image corresponding to , i.e. the "correct answer", denotes the output result of the generator;

[0073] high receptive field perception loss including pixel-by-pixel feature similarity calculation by a pre-trained network, denoted as: , in the formula is a Fourier convolution pre-trained encoding network;

[0074] The final loss function is obtained by synthesizing the above loss functions:

[0075]

[0076] in the formula, , , , respectively represent the weights of the pixel reconstruction loss, the adversarial loss, the feature matching loss and the high receptive field perception loss.

[0077] Further, the defect detection network adopts a deep segmentation model including CANet or U-Net to extract the boundary of the real defect and quantify the thermal response feature, and outputs parameters including defect position, area and severity level for final evaluation.

[0078] The beneficial effects of the present application are:

[0079] In an example embodiment of the present application, the collected infrared image is first processed for tensor feature, and the lamp shadow area is locked; then the lamp shadow area target positioning is performed to generate a high-precision binary lamp shadow mask image; subsequently, the lamp shadow mask image and the infrared image are input into a Fourier convolution feature extraction lamp shadow repair network based on a target guiding mechanism to obtain a lamp shadow repair image; finally, nondestructive defect detection is realized. The present application is compatible with planar and special-shaped structure test pieces, has strong adaptability, fast processing speed, high detection precision, and can be widely applied to structural health monitoring and defect analysis in the fields of aerospace, rail transportation, wind power and the like. BRIEF DESCRIPTION OF DRAWINGS

[0080] Figure 1 a flowchart of a method in an exemplary embodiment of the present application;

[0081] Figure 2 a schematic diagram of priori positioning of potential shadow region based on excitation source layout in an exemplary embodiment of the present application;

[0082] Figure 3 a schematic diagram of image enhancement and coarse segmentation processing in an exemplary embodiment of the present application;

[0083] Figure 4 a schematic diagram of multi-constrained refined shadow region identification in an exemplary embodiment of the present application;

[0084] Figure 5 a schematic diagram of target guided Fourier convolution shadow processing algorithm framework in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0085] The technical solutions of the present application will be described clearly and completely below with reference to the drawings. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0086] In the description of the present application, it should be noted that the directions or positional relationships indicated by "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like are described based on the directions or positional relationships described in the drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application. In addition, "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0087] In the description of the present application, it should be noted that, unless otherwise explicitly specified and limited, "mounting", "connection", "connecting" should be understood in a broad sense, for example, can be fixed connection, can also be detachable connection, or integral connection; can be mechanical connection, can also be electrical connection; can be directly connected, can also be indirectly connected through an intermediate medium, can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0088] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as there is no conflict.

[0089] Reference is made to Figure 1 ,Figure 1 An infrared non-destructive detection lamp shadow processing method based on target guidance provided in an exemplary embodiment of the present application is shown. The method repairs and detects defects in the pseudo shadow or highlight area caused by high reflection or direct light in the collected infrared image data, including the following steps:

[0090] S1: collect infrared images and perform tensor feature processing;

[0091] S2: based on spatial position information guidance, target positioning of the infrared image after tensor feature processing is performed, and a lamp shadow mask image is generated;

[0092] S3: input the lamp shadow mask image and the infrared image into a Fourier convolution feature extraction lamp shadow repair network based on a target guidance mechanism to obtain a lamp shadow repair image;

[0093] S4: input the lamp shadow repair image into a defect detection network for non-destructive defect detection.

[0094] Specifically, in the present exemplary embodiment, the collected infrared images are first processed by tensor feature processing to lock the lamp shadow area; then target positioning of the lamp shadow area is performed to generate a high-precision binary lamp shadow mask; subsequently, the lamp shadow mask image and the infrared image are input into a Fourier convolution feature extraction lamp shadow repair network based on a target guidance mechanism to obtain a lamp shadow repair image; finally, non-destructive defect detection is achieved. The method of the present application is compatible with planar and special-shaped structure test pieces, has strong adaptability, fast processing speed, high detection accuracy, and can be widely applied in the fields of aerospace, rail transportation, wind power, etc. for structure health monitoring and defect analysis.

[0095] The following will describe each step in detail:

[0096] More preferably, in an exemplary embodiment, step S1 locks the lamp shadow area by combining spatial prior positioning with a target detection network.

[0097] Specifically, step S1 includes the following sub-steps:

[0098] S11: apply periodic thermal excitation or pulsed thermal excitation to the surface of the material to be detected by an externally controllable light-thermal excitation system;

[0099] S12: use an infrared thermal imager to collect time series of infrared images and construct a multi-frame image matrix containing thermal diffusion spatiotemporal variation information as input for subsequent processing;

[0100] S13: use PCA algorithm or independent component analysis algorithm or matrix tensor decomposition algorithm of L4 algorithm to weaken thermal distribution image sequence noise interference to obtain infrared images after tensor feature processing.

[0101] When the PCA algorithm is used, the noise interference of the thermal distribution image sequence is weakened, and the formula is: ;

[0102] represents the column vector in the new space obtained after orthogonal transformation, represents the original thermal distribution image column vector, represents the orthogonal transformation.

[0103] The matrix tensor decomposition algorithm in step S13 is used to calculate the position information of the defect area in the infrared thermal image.

[0104] More preferably, in an exemplary embodiment, step S2 generates candidate connected domains using histogram equalization, local contrast enhancement, and adaptive threshold segmentation, and introduces multiple constraints such as horizontal / vertical spacing, alignment, area consistency, and geometric intersection to quickly eliminate false points and obtain a high-precision binary mask.

[0105] Specifically, step S2 includes the following sub-steps:

[0106] S21: Based on the layout of the excitation source of the photo-thermal excitation system, the potential lamp shadow area is located in advance; in a preferred exemplary embodiment, when detecting a planar structure sample, a central region division method is used; according to the axial symmetry physical property of the handheld light excitation device, the spatial mapping relationship between the image coordinate system and the excitation source is established; the infrared image size is defined as HxW, i.e., height x width, and the boundary coordinates of the central positioning area R are calculated:

[0107]

[0108] wherein and are the center region size relative to the height and width of the image;

[0109] When detecting a special-shaped structure sample, a target detection network based on deep learning is used to locate the lamp shadow area.

[0110] The schematic diagram of the prior positioning of the potential lamp shadow area based on the layout of the excitation source is shown in Figure 2 .

[0111] S22: Image enhancement and coarse segmentation processing, including the following sub-steps:

[0112] S221: Image enhancement processing is performed on the candidate region defined in step S21, including histogram equalization, local contrast enhancement, and edge preserving filtering;

[0113] S222: An adaptive threshold segmentation algorithm is used to dynamically define the boundary of the high reflection area, and the connected domain set corresponding to the lamp shadow area is extracted; ​

[0114] An image enhancement and coarse segmentation process is shown in Figure 3 .

[0115] S23: Multi-constraint fine-tuning shadow region identification, spatial geometric consistency constraint screening is performed on the connected domain set extracted in step S222 to obtain a generated shadow mask image; wherein the spatial geometric consistency constraint screening includes the following sub-steps:

[0116] S231: Horizontal and vertical spacing constraints:

[0117] The constraint condition is set as and , respectively limiting the minimum spacing of the transverse distribution and the longitudinal distribution of the connected domain, and removing high-density artifact points; wherein W is the image width, H is the image height, and respectively represent the transverse and longitudinal distance threshold factors, which are determined by KS test, represents the distance between the i-th and j-th region center points in the horizontal direction, represents the distance between the i-th and j-th region center points in the vertical direction, , respectively represent the horizontal coordinate values of the i-th and j-th region center points in the image coordinates, , respectively represent the vertical coordinate values of the i-th and j-th region center points in the image coordinates, i and j represent the region index, and c represents the subscript identifier of the center point;

[0118] S232: Horizontal and vertical alignment constraints:

[0119] Calculate the horizontal and vertical coordinate deviations of the connected domain center, and set the maximum allowed deviation threshold to remove misaligned regions;

[0120] S233: Area consistency constraint of the region:

[0121] According to the prior setting of the size of the shadow region, the upper and lower limits are set to remove large or small interference targets, specifically: the horizontal direction , the vertical direction ; wherein is the pixel area of the connected domain , is the horizontal candidate set that meets the horizontal spacing constraint condition in step S231, is the vertical candidate set that meets the vertical spacing constraint condition in step S231, , represents the connected domain pair in the horizontal direction, represents the vertical connected domain pair, , , , These represent the pixel areas of each connected component;

[0122] S234: Verification of the geometric intersection of the centerlines:

[0123] An intersection relationship judgment model is established using the vectors connecting the center points of regions to filter out regions that do not meet the symmetry condition; assuming that the optimal result is obtained by filtering the horizontally connected components in step S233 ( , () center line Parameterized as: ,in Indicates the line connecting the center points of the horizontally connected domain. Indicates a connection any point on, ( , ) indicates a region The center coordinates, ( , ) indicates a region center coordinates This represents the normalized interpolation parameters of the horizontal centerline between two points; in step S233, the vertical connected components are used to select the optimal result. , () center line Parameterized as: ,in Represents the line connecting the center points of the vertically connected domain. Indicates a connection any point on, ( , ) indicates a region The center coordinates, ( , ) indicates a region center coordinates This represents the normalized interpolation parameters of the longitudinal centerline between two points; the intersection point is calculated using the vector cross product. ,when When intersecting, and These represent parameterized paths. and The index of the intersection point.

[0124] A schematic diagram of refined light and shadow region recognition with multiple constraints is shown below. Figure 4 As shown.

[0125] After this step, a binary mask image M is generated, which serves as the input for the target area in subsequent lamp shadow restoration.

[0126] More preferably, in an exemplary embodiment, step S3 concatenates the mask and the original heatmap and inputs them into a multi-scale encoder-decoder structure, embedding a Fast Fourier Convolution (FFC) module to achieve collaborative reconstruction of global and local features. Using the light shadow mask image M obtained in step S2 as the input condition for the repair target, the original image I and the mask M are concatenated and input into the following network module. A schematic diagram of the target-guided Fourier convolution light shadow processing algorithm framework is shown below. Figure 5 As shown, the encoder corresponds to the gated convolution downsampling module in S31, the decoder corresponds to the gated convolution upsampling module in S34, and the FFC corresponds to S32 and S33.

[0127] Specifically, the Fourier convolutional feature extraction and shadow restoration network based on the target guidance mechanism in step S3 includes:

[0128] S31: Gated Convolution Downsampling Module (Encoder): The infrared image and the lamp shadow mask image are concatenated along the channel dimension and then input into the gated convolution downsampling module to downsample the concatenated image and obtain the initial feature map. This process can be represented as: In the formula, GConv represents gated convolution, which can dynamically enhance important regional features and suppress irrelevant information; where the initial feature map... have The three-dimensional tensor, in which For the number of channels, and Let the height and width of the spliced ​​feature map be represented respectively. For each channel, its two-dimensional time-domain signal is denoted as... , here as well as These are the row index and column index of the spatial domain, respectively; in, for example... Figure 5 In the exemplary embodiment shown, a progressively increasing channel count design is employed to enhance feature extraction capabilities and adapt to information fusion at different scales. Specifically, the first layer has 64 channels to extract initial low-level features of the image; the second layer increases the number of channels to 128, further deepening the network's perception of image structure; and the third layer expands to 256 channels to capture more complex and abstract global semantic information.

[0129] S32: Parallel Feature Extraction Module (one of the embedded Fast Fourier Convolution (FFC) modules): Receives the preliminary feature map output by the gated convolution downsampling module. ,Will Parallel input extraction of global frequency domain information Fast Fourier Convolution module for extracting local spatial information The conventional convolutional modules work together to extract multi-scale spatial context features;

[0130] In a specific exemplary embodiment, S321: the fast Fourier convolution module in the parallel feature extraction module includes:

[0131] right Perform a two-dimensional discrete Fourier transform to obtain the frequency domain representation. ,in , These are the frequency domain row index and the frequency domain column index, respectively.

[0132] After transformation, a set of learnable frequency domain convolution kernels are used. Multiplying it element-wise yields the convolution result. ;

[0133] Subsequently, the inverse Fourier transform was used to... Returning to the time domain, the features output by the branches of the Fast Fourier Convolution module are generated, which are the global frequency domain information. The expression is as follows:

[0134]

[0135] S322: The conventional convolution module in the parallel feature extraction module includes:

[0136] For the initial feature map Applying a series of learnable spatial convolution kernels and biases, denoted as .

[0137] S33: Global-Local Feature Fusion Module (one of the embedded Fast Fourier Convolution (FFC) modules): This module fuses global frequency domain information. and local spatial information By adding elements one by one along the channel or in spatial location, preliminary fusion characteristics are obtained: In the formula The output of the Fast Fourier Convolution branch contains global frequency domain information; For the output of a regular convolutional branch, local spatial details are preserved; This represents the initial fusion features; subsequently, the initial fusion features were analyzed. By applying Batch Normalization (BN) and ReLU activation sequentially, the fused features are obtained: In the formula Normalize the features of each channel and perform stable training; Increase the nonlinear expressive power of the network; The fused features will contain both global context and local details for subsequent enhancement.

[0138] S34: Skip Connections and Feature Enhancement Module: To prevent deep features from losing low-level details, the shallow features saved during the downsampling process of the gated convolution downsampling module are used after fusion. Features after fusion Direct addition yields enhanced features. : In the formula The shallow features saved during the downsampling process contain richer texture information; the addition operation ensures that the network inherits the details of the shallow layers while preserving the global semantics of the deep layers. The enhanced features will be used as input to the upsampling module to further restore the spatial resolution.

[0139] S35: Gated Convolution Upsampling Module (Decoder): Enhances features through gated deconvolution. The original image resolution is restored to obtain the lamp shadow restoration image.

[0140] In one specific exemplary embodiment, the gated convolutional upsampling module includes:

[0141] S351: Gating weight generation:

[0142]

[0143] In the formula, and These are the learnable convolutional kernel and the bias, respectively. Activate for Sigmoid and output the gate mask. ;

[0144] S352: Deconvolution upsampling:

[0145]

[0146] Where Deconv is a regular deconvolution operator or a transpose convolution operator, the output is... ;

[0147] S353: Gated Fusion

[0148]

[0149] In the formula, the upsampled features with dynamic weights are obtained. ;

[0150] S354: Layer-by-layer iteration:

[0151] Repeat the above process, upsampling multiple times until the image size is restored to the original size, and finally output the repaired result. .

[0152] By using gated deconvolution, upsampling can not only increase spatial resolution, but also... Dynamic noise suppression and enhancement of important areas lay the foundation for generating high-quality, shadow-free images.

[0153] It should be noted that, Figure 5 The example shown is a generator. During the training phase, a discriminator is required. The discriminator used in this exemplary embodiment is a PatchGAN structure. By judging the realism of local image regions, it provides a probability distribution of the real / false values ​​of the input image, guiding the generator to generate more natural and realistic details in the repaired area. After the model is trained, in actual use, the discriminator is not required. Only the trained weight file needs to be used to process the shadow processing network (generator) part introduced in this exemplary embodiment.

[0154] More preferably, in an exemplary embodiment, during the training process of the Fourier convolutional feature extraction network for lamp shadow restoration based on a target-guided mechanism, a joint loss function including pixel-level mean square error, local adversarial loss, perceptual loss, and semantic consistency is employed to ensure the detail and structural continuity of the restoration result. The generator... With discriminator By placing it within an adversarial training framework, competitive optimization can both improve the quality of the repair and ensure the realism of the generated results.

[0155] Specifically, the Fourier convolutional feature extraction and shadow restoration network based on the target guidance mechanism is trained through a multi-loss function target joint optimization mechanism, which includes:

[0156] Generator Receive mask condition input, i.e., infrared image With light shadow mask image Output image with light and shadow restoration Discriminator The PatchGAN structure is adopted. By judging the authenticity of local image regions, the probability distribution of the true / false values ​​of the input image is given, which guides the generator to generate more natural and realistic details in the repaired area.

[0157] The training goal is to make generated Make as much as possible The result is determined to be true, while retaining the original infrared image without light shadows. Similarity at the pixel and perceptual feature levels; training is performed using pixel reconstruction loss, adversarial loss, feature matching loss, and high receptive field perceptual loss.

[0158] More preferably, in an exemplary embodiment, pixel reconstruction loss ,ensure With real images Maintain consistency in areas where there is no light or shadow.

[0159] Combat losses Including discriminator loss and generator loss , is described as In the formula For gradient penalty, This represents the weight coefficients of the generator gradient smoothing loss term. This represents the expected value or batch mean of the sample. Indicates the discriminator's effect on the image The probability distribution of the output being a "real image" Indicates the image The gradient operation performed on the discriminator output is used to smooth the generator loss; (the two discriminators D here, D and D of the feature matching loss, are the same network structure, but their input information is different. The discriminator input of the adversarial loss is the final image generated by the generator, and the discriminator input of the feature matching loss is the high-dimensional information of the intermediate feature layer.)

[0160] Discriminator loss ;

[0161] Generator loss for:

[0162] ;

[0163] Feature matching loss Including discriminators based on real and fake samples Loss is expressed as:

[0164]

[0165] In the formula, The output of the discriminator's intermediate layer, The number of intermediate layers. Indicates correspondence The original, undamaged image, i.e., the "correct answer". This represents the output of the generator; here, both the generator's output and the real label are used as outputs. Through the generator's encoding process, the high-dimensional feature loss of each intermediate layer is compared.

[0166] High receptive field perception loss This includes calculating pixel-wise feature similarity using a pre-trained network, expressed as: In the formula For Fourier convolutional pre-trained encoding networks;

[0167] Combining the above loss functions, the final loss function is:

[0168]

[0169] In the formula, , , , These represent the weights of pixel reconstruction loss, adversarial loss, feature matching loss, and high receptive field perception loss, respectively.

[0170] Specifically, in actual training, the network not only needs to achieve accurate reconstruction at the pixel level and local features, but also needs to fully understand the global structure and semantic relationships of the image. Therefore, the pixel reconstruction loss weight is set to... =1 to ensure basic pixel recovery; set the adversarial loss weight to 1. =0.05, to achieve a balance between GAN training stability and repair quality; set the weight of the high receptive field perceptual loss to 0.05. =1, enabling the network to capture overall texture and structural coherence at a global scale; gradient penalty is applied to the discriminator. =15, used to constrain its continuity and prevent training collapse.

[0171] More preferably, in an exemplary embodiment, the final step S5 involves extracting the true defect boundaries and quantifying features through deep segmentation or manual evaluation, thereby achieving high-fidelity elimination of lamp shadow artifacts and complete preservation of defect information.

[0172] The defect detection network uses a deep segmentation model, including CANet or U-Net, to extract the boundaries of real defects and quantify their thermal response features. It outputs parameters, including defect location, area, and severity level, for final evaluation.

[0173] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A target-guided infrared non-destructive testing light shadow processing method for repairing artifacts or bright areas caused by high reflectivity or direct light in acquired infrared image data and detecting defects, characterized in that: Includes the following steps: S1: Acquire infrared images and perform tensor feature processing; S2: Based on spatial location information guidance, target localization of the light shadow region is performed on the infrared image after tensor feature processing, and a light shadow mask image is generated; S3: Input the light shadow mask image and infrared image into the light shadow restoration network based on the target guidance mechanism Fourier convolution feature extraction to obtain the light shadow restoration image; S4: Input the lamp shadow restoration image into the defect detection network for non-destructive defect detection.

2. The target-guided infrared non-destructive testing light shadow processing method according to claim 1, characterized in that: Step S1 includes the following sub-steps: S11: Apply periodic or pulsed thermal excitation to the surface of the material to be tested through an externally controllable photothermal excitation system; S12: Use an infrared thermal imager to acquire time-series infrared images and construct a multi-frame image matrix containing information on the spatiotemporal changes in thermal diffusion, which will then be used as input for subsequent processing. S13: Using the PCA algorithm, independent component analysis algorithm, or L4 algorithm for matrix tensor decomposition, noise interference in the thermal distribution image sequence is reduced to obtain the infrared image with tensor feature processing completed.

3. The target-guided infrared non-destructive testing light shadow processing method according to claim 2, characterized in that: Step S2 includes the following sub-steps: S21: Based on the layout of the excitation source of the photothermal excitation system, the potential lamp shadow area is located a priori. S22: Image enhancement and coarse segmentation processing, including the following sub-steps: S221: Perform image enhancement processing on the candidate regions previously defined in step S21, including histogram equalization, local contrast enhancement, and edge-preserving filtering; S222: An adaptive threshold segmentation algorithm is used to dynamically define the boundary of the high-reflectivity region and extract the set of connected components corresponding to the lamp shadow region; S23: Refined light shadow region identification with multiple constraints. The set of connected components extracted in step S222 is subjected to spatial geometric consistency constraint filtering to obtain the generated light shadow mask image. The spatial geometric consistency constraint filtering includes the following sub-steps: S231: Horizontal and vertical spacing constraints: Set the constraints as follows and Minimum spacing is applied to connected components with horizontal and vertical distributions to remove high-density artifacts; where W is the image width, H is the image height, and τ is the image height. h and τ v Δx represents the horizontal and vertical distance threshold factors, determined through a sample KS test. ij Δy represents the horizontal distance between the center points of the i-th and j-th regions. ij This represents the vertical distance between the center points of the i-th and j-th regions. These represent the x-coordinates of the center points of the i-th and j-th regions in the image coordinate system, respectively. Let represent the ordinate values ​​of the center points of the i-th and j-th regions in the image coordinate system, where i and j represent the region indices and c represents the subscript identifier of the center point; S232: Horizontal and vertical alignment constraints: Calculate the deviation of the x and y coordinates of the center of the connected domain, set the maximum allowable deviation threshold, and remove misaligned regions; S233: Region area consistency constraint: Based on the prior setting of upper and lower limits for the size of the light shadow area, excessively large or excessively small interfering targets are eliminated, specifically in the horizontal direction. vertical direction Where A k For connected components The pixel area, Ψ h To satisfy the horizontal spacing constraint in step S231, Ψ v To provide a vertical candidate set that satisfies the vertical spacing constraint in step S231, k = i, j, m, n. Represents horizontally connected pairs. A represents a vertically connected pair. i A j A m A n These represent the pixel areas of each connected component; S234: Verification of the geometric intersection of the centerlines: An intersection relationship judgment model is established using the vectors connecting the center points of regions to filter out regions that do not meet the symmetry condition; let the horizontal connected components in step S233 be the optimal results for screening. center line L h Parameterized as: Where L h p(t) represents the line connecting the center points of the horizontally connected domain, and p(t) represents the line L. h any point on the top Indicates the area center coordinates Indicates the area The center coordinates of the point and t represent the normalized interpolation parameters of the horizontal centerline between the two points; in step S233, the vertical connected components are used to select the optimal results. center line L v Parameterized as: Where L v q(s) represents the line connecting the center points of the vertically connected domain, and q(s) represents the line connecting the center points. v any point on the top Indicates the area center coordinates Indicates the area The center coordinates of the two points are denoted by s, which represents the normalized interpolation parameter of the longitudinal centerline between them. The intersection point p(t0) = q(s0) is calculated by vector cross product, when (t0,s0)∈[0,1]. 2 Intersection is determined by time, where t0 and s0 represent the intersection indexes on the parameterized paths p(t0) and q(s0), respectively.

4. The target-guided infrared non-destructive testing light shadow processing method according to claim 3, characterized in that: In step S21, when inspecting a planar structural specimen, a central region division method is adopted; based on the axisymmetric physical characteristics of the handheld optical excitation device, a spatial mapping relationship between the image coordinate system and the excitation source is established; the infrared image size is defined as H×W, i.e., height × width, and the boundary coordinates of the central positioning region R are calculated: In the formula, ΔH and ΔW are the dimensions of the central region relative to the height and width of the image; When inspecting irregularly shaped specimens, a deep learning-based target detection network is used to locate targets in the light shadow area.

5. The target-guided infrared non-destructive testing light shadow processing method according to claim 3 or 4, characterized in that: The Fourier convolutional feature extraction and shadow restoration network based on the target guidance mechanism in step S3 includes: Gated convolutional downsampling module: The infrared image and the lamp shadow mask image are stitched together in the channel dimension and then input into the gated convolutional downsampling module. The stitched image is downsampled to obtain the initial feature map F0, which has a three-dimensional tensor of C×M×N, where C is the number of channels, and M and N represent the height and width of the stitched feature map, respectively. For each channel, its two-dimensional time domain signal is denoted as X(m,n), where m = 0, 1, ..., M-1 and n = 0, 1, ..., N-1 are the row index and column index of the spatial domain, respectively. Parallel Feature Extraction Module: Receives the preliminary feature map F0 output from the gated convolutional downsampling module, and inputs F0 in parallel to extract global frequency domain information F. FFC Fast Fourier Convolution module, extracting local spatial information F Conv The conventional convolutional modules work together to extract multi-scale spatial context features; Global-Local Feature Fusion Module: This module fuses global frequency domain information F... FFC and local spatial information F Conv By adding elements one by one at the channel or spatial location, a preliminary fusion feature is obtained: S = F FFC +F Conv Subsequently, batch normalization (BN) and ReLU activation are applied sequentially to the initial fused feature S to obtain the fused feature: F1 = ReLU(BN(S)); Skip connections and feature enhancement module: This module stores shallow features F saved during the downsampling process of the gated convolution downsampling module. skip The enhanced feature F2 is obtained by directly adding the fused feature F1: F2 = F1 + F skip ; Gated convolution upsampling module: The enhanced feature F2 is restored to the original image resolution through gated deconvolution to obtain the lamp shadow restoration image.

6. The target-guided infrared non-destructive testing light shadow processing method according to claim 5, characterized in that: The fast Fourier convolution module in the parallel feature extraction module includes: Performing a two-dimensional discrete Fourier transform on X(m,n) yields its frequency domain representation. Where u = 0, 1, ..., M-1 and v = 0, 1, ..., N-1 are the frequency domain row index and frequency domain column index, respectively; After transformation, a set of learnable frequency domain convolution kernels W are used. F Multiplying (u,v) element-wise yields the convolution result Y. F (u,v)=X F (u,v)·W F (u,v); Subsequently, the inverse Fourier transform was used to transform Y F (u,v) is restored to the time domain, generating the features output by the Fast Fourier Convolution module branch, i.e., the global frequency domain information F. FFC The expression is as follows: The conventional convolution module in the parallel feature extraction module includes: A series of learnable spatial convolution kernels and biases are applied to the initial feature map F0, denoted as F Conv =Conv(F0).

7. The infrared non-destructive testing light shadow processing method based on target guidance according to claim 6, characterized in that: The gated convolutional upsampling module includes: S351: Gating weight generation: M g =σ(W g *F2+b g ) In the formula, W g and b g These are the learnable convolution kernel and the bias, respectively; σ(x) = 1 / (1+e -x ) is activated by Sigmoid, and the output gate mask M is... g ∈(0,1) C×M×N ; S352: Deconvolution upsampling: U = Deconv(F2) Where Deconv is a regular deconvolution operator or a transpose convolution operator, and its output is U; S353: Gated Fusion F3=M g ×U In the formula, × indicates element-wise multiplication to obtain the upsampled feature F3 with dynamic weights; S354: Layer-by-layer iteration: Repeat the above process, upsampling multiple times until the image size is restored to the original size, and finally output the repaired result.

8. The target-guided infrared non-destructive testing light shadow processing method according to claim 7, characterized in that: The Fourier convolutional feature extraction and shadow restoration network based on the target guidance mechanism is trained through a multi-loss function objective joint optimization mechanism, specifically including: Generator G receives the masking condition input, namely the infrared image I and the lamp shadow mask image M, and outputs the lamp shadow restoration image. Discriminator D adopts the PatchGAN structure, which determines the authenticity of local image regions and gives the probability distribution of the true / false values ​​of the input image. The training objective is to generate G. The goal is to make D true as much as possible, while preserving the similarity with the original infrared image I at the pixel and perceptual feature levels; training is performed using pixel reconstruction loss, adversarial loss, feature matching loss, and high receptive field perceptual loss.

9. The infrared non-destructive testing light shadow processing method based on target guidance according to claim 8, characterized in that: Pixel reconstruction loss L1, guarantee Consistent with the real image I in the known area without light or shadow; Combat loss L adv Including discriminator loss L D And generator loss L G , is described as L adv =L D +L G +λ GP L GP In the formula For gradient penalty, λ GP This represents the weight coefficients of the generator gradient smoothing loss term. This represents the expected value or batch mean of the sample. Indicates the discriminator's effect on the image The probability distribution of the output "is a real image". Indicates the image The gradient calculation performed on the discriminator output is used to smooth the generator loss; Discriminator loss L D ; Generator loss L G for: Feature matching loss L FM Including the L1 loss between discriminators based on real and fake samples, expressed as: In the formula, D k The output of the discriminator's intermediate layer is T, where T is the number of intermediate layers. G(I,M) represents the original, undamaged image corresponding to I, i.e., the "correct answer", and G(I,M) represents the output of the generator. High receptive field perception loss L HRF This includes calculating pixel-wise feature similarity using a pre-trained network, expressed as: In the formula φ HRF For Fourier convolutional pre-trained encoding networks; Combining the above loss functions, the final loss function is: L Final =λ1L1+λ adv L adv +λ FM L FM +λ HRF L HRF In the formula, λ1 and λ adv , λ FM , λ HRF These represent the weights of pixel reconstruction loss, adversarial loss, feature matching loss, and high receptive field perception loss, respectively.

10. The target-guided infrared non-destructive testing light shadow processing method according to claim 1, characterized in that: The defect detection network uses a deep segmentation model, including CANet or U-Net, to extract the boundaries of real defects and quantify their thermal response features. It outputs parameters, including defect location, area, and severity level, for final evaluation.

Images