Ultrasonic and infrared thermography integrated detection method and system sharing excitation source

Abstract

The application provides an ultrasonic and infrared thermal imaging integrated detection method and system sharing an excitation source, applies the shared excitation source to a to-be-detected object, synchronously collects multi-channel ultrasonic echo signals and infrared thermal imaging images, divides the to-be-detected object into multiple target regions, and calculates the frequency band ultrasonic attenuation coefficient and acoustic impedance of each region, and then obtains a densification value; after extracting a heat flow response feature map from the infrared image, according to the spatial gradient of the densification value and the heat flow response energy gathering area, the multi-modal data is adaptively partitioned to form multiple data analysis subsets. For each subset, a local joint dictionary is constructed according to the densification value and the heat flow feature; the ultrasonic and infrared features in the subset are jointly sparsely represented by using the dictionary to obtain a group of cross-modal fusion coefficients, based on the fusion coefficients and a defect recognition model, the defect information in the to-be-detected object is accurately determined, and a defect distribution map is generated.

Description

Technical Field

[0001] This application belongs to the field of defect detection, and in particular relates to an integrated detection method and system for ultrasonic and infrared thermal imaging with a shared excitation source. Background Technology

[0002] Non-destructive testing (NDT) is an important means of ensuring structural integrity, evaluating material properties, preventing catastrophic accidents, and controlling product quality. Ultrasonic testing utilizes the propagation characteristics of high-frequency sound waves within materials, analyzing reflected ultrasonic signals to detect internal defects. However, its results are easily affected by the coupling state between the probe and the surface of the workpiece, heavily relying on operator experience. It is challenging to test components with complex geometries and has limited ability to characterize certain minute defects or early material damage, especially when there are significant microstructural inhomogeneities within the material, leading to missed detections. Infrared thermal imaging is another common method. When defects exist within a material, the defect area creates an abnormal temperature distribution on the material surface. Infrared thermal imagers detect these defects by capturing infrared radiation energy. Using ultrasound as the excitation source, thermal imaging induces a localized temperature rise at the defect location, offering high detection sensitivity and efficiency. However, infrared thermal imaging has relatively limited detection depth, resulting in insufficient accuracy for deeply buried defects. Integrating both methods for collaborative detection can improve the comprehensiveness and reliability of the testing. However, existing integration methods mostly employ sequential detection followed by data comparison, or simple data-level and image-level fusion, such as image overlay and simple feature weighting. These methods often fail to fully utilize the rich information carried by the ultrasonic signals themselves. How to utilize ultrasonic signals to enhance the accuracy of infrared thermal imaging detection and more accurately distinguish between genuine defect signals and interference caused by the inherent structure of the material is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] To address the challenge of effectively distinguishing defect signals from complex background interference in integrated ultrasonic and infrared thermal imaging detection methods, this application proposes an integrated ultrasonic and infrared thermal imaging detection method with a shared excitation source, comprising:

[0004] A shared excitation source is applied to the object under test, and multi-channel ultrasonic echo signals and infrared thermal imaging image sequences are acquired simultaneously; the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target region are calculated based on the multi-channel ultrasonic echo signals; and the densification value is calculated based on the frequency band ultrasonic attenuation coefficient and the acoustic impedance.

[0005] Heat flow response feature maps are extracted from infrared thermal imaging image sequences; based on the spatial distribution gradient of the densification value and the energy accumulation region of the heat flow response feature maps, the multi-channel ultrasonic echo signals and the infrared thermal imaging images are partitioned to obtain multiple data analysis subsets;

[0006] For each data analysis subset, a local joint dictionary is obtained based on the thermal flow response characteristic energy. The ultrasonic signal features and infrared thermal image features within the corresponding data analysis subset are jointly sparsely represented using the local joint dictionary to obtain a set of cross-modal fusion coefficients. Based on the cross-modal fusion coefficients and the defect identification model, the defect information in the test object is determined.

[0007] Optionally, the calculation of the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target region based on the multi-channel ultrasonic echo signal includes:

[0008] The preprocessed ultrasonic echo signals of each channel are converted to the frequency domain through short-time Fourier transform or wavelet packet decomposition to obtain spectral information under different time windows; for at least three preset narrowband frequency bands with different center frequencies, the signal energy attenuation curve of each channel in the narrowband frequency band as a function of propagation depth is extracted; the energy attenuation curve is subjected to exponential fitting to obtain the frequency band ultrasonic attenuation coefficient of each target area in each preset narrowband frequency band.

[0009] The acoustic impedance of each target region is calculated based on the material density and sound velocity.

[0010] Optionally, the step of calculating the densification value based on the ultrasonic attenuation coefficient of the frequency band and the acoustic impedance includes:

[0011] For each target region, the ultrasonic attenuation coefficient values ​​calculated under all preset narrowband frequency bands are normalized.

[0012] The comprehensive attenuation parameter is obtained by multiplying the normalized ultrasonic attenuation coefficients of each frequency band by the weighting factors of the corresponding ultrasonic attenuation coefficients of different frequency bands and summing the results.

[0013] The integrated attenuation parameter and the acoustic impedance of the target region are input into the pre-trained SVM to obtain the densification value.

[0014] Optionally, based on the spatial distribution gradient of the densification value and the energy accumulation region of the heat flow response characteristic map, the multi-channel ultrasonic echo signal and the infrared thermal imaging image are partitioned to obtain multiple data analysis subsets, including:

[0015] Spatial interpolation is performed on the densification values ​​of all target regions to generate a densification value distribution map; the gradient magnitude of the densification value distribution map is calculated to obtain a densification gradient map;

[0016] Inter-frame difference calculation is performed on the infrared thermal image sequence to extract the heat flow response feature map and identify energy accumulation areas with energy higher than the preset energy threshold;

[0017] On the densification gradient map, the boundary regions with gradient values ​​higher than a preset gradient threshold are identified by image segmentation method, and the test object is divided into multiple dense and uniform regions.

[0018] The boundary of the energy accumulation region is projected onto the divided dense and uniform regions. If the energy accumulation region spans multiple dense and uniform regions, the boundary of the energy accumulation region is prioritized, and the division result is adjusted to obtain a data analysis subset. The data in the multi-channel ultrasonic echo signal and the infrared thermal imaging image corresponding to the spatial range of each data analysis subset are respectively assigned to the corresponding subset.

[0019] Optionally, the step of obtaining the local joint dictionary of the data analysis subset based on the thermal flux response characteristic energy includes:

[0020] Initialize a basic dictionary containing ultrasonic feature atoms and infrared thermal feature atoms, and calculate the average densification value and average thermal flux response feature energy of all target regions within the current data analysis subset;

[0021] If the average thermal flux response characteristic energy is not greater than the first thermal energy preset value, then through the iterative dictionary update process, the weight coefficient of the structural background atom is increased, and at the same time, L1 norm sparsity constraints are applied to the atoms in the basic dictionary that are related to the known defect patterns; otherwise, the weight coefficient of the defect pattern atom that matches the most likely defect pattern in the current data analysis subset is enhanced.

[0022] The optimized dictionary is used as a local joint dictionary for the data analysis subset.

[0023] Optionally, the step of using the local joint dictionary to perform joint sparse representation of the ultrasonic signal features and infrared thermal image features within the corresponding data analysis subset to obtain a set of cross-modal fusion coefficients includes:

[0024] Extract a preset ultrasonic signal feature vector from the multi-channel ultrasonic echo signals within the current data analysis subset. The ultrasonic signal feature vector includes the time-domain amplitude envelope, the frequency-domain energy spectral density, and the attenuation value of a specific frequency band.

[0025] Extract a preset infrared thermal image feature vector from the infrared thermal imaging images in the current data analysis subset. The infrared thermal image feature vector includes a sequence of pixel temperature values, temperature gradient amplitude, and average temperature change rate of a specific area.

[0026] The fused feature vector of the data analysis subset is obtained by concatenating the feature vector of the ultrasonic signal and the feature vector of the infrared thermal image.

[0027] The orthogonal matching pursuit method is used to perform sparse decomposition of the fused feature vector on the local joint dictionary of the data analysis subset, resulting in a set of cross-modal fusion coefficients projected onto the dictionary atoms.

[0028] Optionally, determining the defect information in the object under test based on the cross-modal fusion coefficients and the defect identification model includes:

[0029] The cross-modal fusion coefficients obtained from each data analysis subset are input into the defect identification model to obtain the probability of defects existing in the data analysis subset and the defect type label;

[0030] Spatial mapping is performed on the defect probabilities and type labels of all data analysis subsets to generate a defect distribution map of the entire object under test.

[0031] This application also proposes an integrated detection system for ultrasound and infrared thermal imaging with a shared excitation source, comprising:

[0032] The acquisition and feature extraction unit is used to apply a shared excitation source to the object under test and simultaneously acquire multi-channel ultrasonic echo signals and infrared thermal imaging image sequences; calculates the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target region based on the multi-channel ultrasonic echo signals; and calculates the densification value based on the frequency band ultrasonic attenuation coefficient and the acoustic impedance.

[0033] A subset partitioning unit is used to extract heat flow response feature maps from infrared thermal imaging image sequences; based on the spatial distribution gradient of the densification value and the energy accumulation region of the heat flow response feature map, the multi-channel ultrasonic echo signal and the infrared thermal imaging image are partitioned to obtain multiple data analysis subsets;

[0034] The detection unit is used to obtain a local joint dictionary for each data analysis subset based on the thermal flow response characteristic energy, and to use the local joint dictionary to perform joint sparse representation of the ultrasonic signal features and infrared thermal image features in the corresponding data analysis subset to obtain a set of cross-modal fusion coefficients; based on the cross-modal fusion coefficients and the defect identification model, the defect information in the test object is determined.

[0035] Compared with the prior art, this application has the following beneficial effects:

[0036] This application uses the densification value calculated based on the frequency band ultrasonic attenuation coefficient and acoustic impedance as a local material parameter. It utilizes the spatial distribution gradient of the densification value and the energy accumulation region of the heat flow response feature map extracted from the infrared image to adaptively partition the ultrasonic and infrared data, forming multiple data analysis subsets. For each data analysis subset, a basic cross-modal joint dictionary is optimized based on its densification value and heat flow response feature energy, so that the dictionary can more accurately express the signal characteristics of the specific region. The optimized local joint dictionary is used to perform joint sparse representation of the ultrasonic and infrared features in the corresponding subset, and the resulting cross-modal fusion coefficient can better reflect the true state of the test object. Attached Figure Description

[0037] Figure 1 This is a flowchart of Example 1;

[0038] Figure 2 This is an infrared thermal imaging image;

[0039] Figure 3 This is a schematic diagram of performing a short-time Fourier transform on an ultrasound signal.

[0040] Figure 4 This is a schematic diagram of energy changes in each frequency band. Detailed Implementation

[0041] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0042] In a specific embodiment, this application proposes an integrated detection method for ultrasound and infrared thermal imaging with a shared excitation source, such as... Figure 1 As shown, it includes:

[0043] Step 1: Apply a shared excitation source to the object under test and simultaneously acquire multi-channel ultrasonic echo signals and infrared thermal imaging image sequences; calculate the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target region based on the multi-channel ultrasonic echo signals; calculate the densification value based on the frequency band ultrasonic attenuation coefficient and the acoustic impedance.

[0044] A shared excitation source capable of simultaneously exciting ultrasonic wave propagation and thermal effects is applied to the object under test (DUT) using an excitation device. Simultaneously, a multi-channel ultrasonic probe array receives ultrasonic echo signals reflected or scattered from within the DUT. At the same time, an infrared thermal imager non-contactly captures changes in infrared radiation energy on the DUT surface during or after excitation and converts these changes into time-series infrared thermal images, such as... Figure 2 As shown, the acquired multi-channel ultrasonic echo signals are processed to calculate the frequency band ultrasonic attenuation coefficient and acoustic impedance of the test object within each pre-divided target region. The target region is the spatial region corresponding to the array elements or beamforming units of the ultrasonic probe. In one embodiment, the frequency band ultrasonic attenuation coefficient is calculated by analyzing the echo amplitude changes of reflectors at different depths and fitting it with an acoustic propagation model. The acoustic impedance is based on the material density and sound velocity, where the density can be obtained from the material of the test object, and the sound velocity is estimated from the transit time of the ultrasonic signal or indirectly calculated using the reflection and transmission coefficients of ultrasonic waves at different medium interfaces. The calculated frequency band ultrasonic attenuation coefficients and acoustic impedances of each target region are used to calculate a densification value, which is a quantitative indicator representing the density of the material in the target region. In another embodiment, the sum of the attenuation coefficients of all frequency bands is calculated, and the weighted difference between the acoustic impedance and the sum of the attenuation coefficients is used as the densification value.

[0045] Step 2: Extract heat flow response feature maps from the infrared thermal imaging image sequence; based on the spatial distribution gradient of the densification value and the energy accumulation region of the heat flow response feature map, partition the multi-channel ultrasonic echo signal and the infrared thermal imaging image to obtain multiple data analysis subsets;

[0046] The acquired infrared thermal imaging image sequence is analyzed to extract thermal flow response feature maps. In one embodiment, differential operations are performed on consecutive frames of images to highlight temperature change areas, or blind source separation methods such as principal component analysis are used to extract dynamic thermal patterns from time-series thermal images, or thermal signal reconstruction methods are used to analyze heat decay curves to invert heat source characteristics. Defects or structural differences inside the material will be abnormal areas in the thermal flow response feature map.

[0047] Regions with drastic changes in densification values ​​on the spatial distribution map typically correspond to boundaries where the internal structure or density of the material undergoes significant changes. Concentrated energy areas in the thermal flow response characteristic map indicate significant thermal activity, which may suggest defects. In one embodiment, preliminary region division is first performed based on the densification value gradient. Then, these preliminary boundaries are refined and detailed using energy-concentrated areas in the thermal flow response characteristic map. Alternatively, image segmentation methods can be used to divide the original ultrasonic and infrared data into multiple data analysis subsets with different acoustic and thermal properties. Each data analysis subset corresponds to a local region of the test object that shares commonalities in density and thermal response.

[0048] Step 3: For each data analysis subset, obtain a local joint dictionary of the data analysis subset based on the thermal flow response characteristic energy. Use the local joint dictionary to perform joint sparse representation of the ultrasonic signal features and infrared thermal image features in the corresponding data analysis subset to obtain a set of cross-modal fusion coefficients. Based on the cross-modal fusion coefficients and the defect identification model, determine the defect information in the test object.

[0049] For a specific subset of data for analysis, a basic cross-modal joint dictionary is generated based on the average densification value and average thermal flux response characteristic energy within that subset region. This basic dictionary is pre-learned from a large number of typical ultrasonic signal features and infrared thermal image features, for example, using the K-SVD algorithm or online dictionary learning methods. If the subset exhibits high densification but insignificant thermal activity, or is neither highly densified nor exhibiting significant thermal activity, it may represent background structures or non-critical regions. In this case, dictionary optimization focuses on accurately representing such structural backgrounds. For example, this can be achieved by enhancing the weights of dictionary atoms associated with known background structures, while simultaneously imposing constraints on atoms associated with defects to make them less likely to be selected in subsequent signal representations, thus obtaining the first local joint dictionary. Conversely, if a subset exhibits high density with significant thermal activity, or is not highly dense but has significant thermal activity, it may contain defects. Dictionary optimization will focus on enhancing atoms associated with the thermoacoustic coupling modes of potential defects, and will further utilize the acoustic properties and thermal flow response energy intensity within the subset to adjust these defect mode atoms, such as adjusting their shape, amplitude, or weight in the dictionary, so that they can more accurately match and capture specific defect signals that may exist within the subset, and obtain a second local joint dictionary.

[0050] Then, the corresponding ultrasonic signal features and infrared thermal image features within the data analysis subset are extracted. Ultrasonic signal features include, but are not limited to, time-domain envelope and spectral features, while infrared thermal image features include, but are not limited to, temperature change sequences and spatial gradients. These cross-modal features are then jointly sparsely represented on the local joint dictionary. Sparse representation approximates the original signal features using a linear combination of as few atoms as possible from the dictionary; the resulting combination coefficients are the cross-modal fusion coefficients. Preferably, sparse coding methods such as orthogonal matching pursuit, minimum angular regression, or basis pursuit are used. The cross-modal fusion coefficients obtained from each data analysis subset are input into a pre-trained defect identification model. The defect identification model can be a traditional machine learning classifier, such as a support vector machine or random forest, or a deep learning model, such as a convolutional neural network or a recurrent neural network. This invention does not specifically limit the defect identification model. The defect identification model determines whether a subset has defects based on the feature patterns of the input fusion coefficients, as well as the possible defect types and severity. Finally, the judgment results of all subsets are summarized to form an assessment of the overall defect status of the object under test.

[0051] A single-frequency attenuation coefficient may not be able to fully capture subtle changes in the internal state of a material, especially for complex materials with multiple potential defect types or structural inhomogeneities. Acoustic impedance, as an intrinsic parameter of a material, is closely related to its density and elastic properties and is an important basis for judging material homogeneity and interfacial characteristics. In an optional embodiment, the calculation of the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target region based on the multi-channel ultrasonic echo signal includes:

[0052] The preprocessed ultrasonic echo signals of each channel are converted to the frequency domain through short-time Fourier transform or wavelet packet decomposition to obtain spectral information under different time windows; for at least three preset narrowband frequency bands with different center frequencies, the signal energy attenuation curve of each channel in the narrowband frequency band as a function of propagation depth is extracted; the energy attenuation curve is subjected to exponential fitting to obtain the frequency band ultrasonic attenuation coefficient of each target area in each preset narrowband frequency band.

[0053] The acoustic impedance of each target region is calculated based on the material density and sound velocity.

[0054] Specifically, the raw A-scan echo signals acquired from each ultrasonic transducer channel are preprocessed, for example, by using bandpass filtering to remove noise unrelated to the excitation frequency, and by performing signal amplification or normalization. For each preprocessed A-scan signal, the Short-Time Fourier Transform (STFT) method or wavelet packet decomposition is used to decompose the signal into orthogonal wavelet packet coefficients at different scales and frequencies. Figure 3The results of performing a short-time Fourier transform on an ultrasound signal are shown. After obtaining the time-spectrum information, for three or more narrowband frequency bands with pre-defined center frequencies of f1, f2, and f3, each band having a certain bandwidth, such as f1 ± Δf1, the energy values ​​of the signal changing with time within these frequency bands are extracted. Preferably, this is done by integrating the energy of the corresponding frequency band on the spectrogram, or by calculating the energy of the coefficients of the corresponding frequency band after wavelet packet decomposition. Figure 4 The energy transformation across each frequency band is illustrated. For each channel and each preset frequency band, a curve showing energy attenuation with depth can be obtained. Sound energy attenuates approximately exponentially in a homogeneous medium, i.e. Where E(d) is the energy at depth d, E0 is the initial energy, and α is the attenuation coefficient. The frequency band ultrasonic attenuation coefficient α of the target area in the specific frequency band can be obtained by taking the logarithm of the extracted energy attenuation curve and then performing linear fitting or direct nonlinear exponential fitting.

[0055] One approach to calculating acoustic impedance is based on the fundamental physical properties of the material. The density ρ of the material under test is obtained from materials handbooks, and the sound velocity c in the material is estimated from ultrasonic testing data, for example, by measuring the transit time of ultrasound waves in a sample of known thickness, or by analyzing the time intervals between multiple bottom echoes. The acoustic impedance Z of the target region is then calculated using the formula Z = ρ. c is calculated. In another embodiment, when an ultrasonic wave is incident perpendicularly on the interface between a reference material with a known acoustic impedance Z1 and a target region with an acoustic impedance Z2, its reflection coefficient R is obtained by measuring the incident wave amplitude Ai and the reflected wave amplitude Ar. The relationship between the reflection coefficient R and the acoustic impedance of the two media is R = (Z2 - Z1) / (Z2 + Z1). If Z1 is known, the acoustic impedance Z2 of the target region can be calculated by using the measured reflection coefficient R.

[0056] A single acoustic parameter is insufficient to comprehensively and accurately assess the true level of densification. In an optional embodiment, calculating the densification value based on the ultrasonic attenuation coefficient of the frequency band and the acoustic impedance includes:

[0057] For each target region, the ultrasonic attenuation coefficient values ​​calculated under all preset narrowband frequency bands are normalized.

[0058] The comprehensive attenuation parameter is obtained by multiplying the normalized ultrasonic attenuation coefficients of each frequency band by the weighting factors of the corresponding ultrasonic attenuation coefficients of different frequency bands and summing the results.

[0059] The integrated attenuation parameter and the acoustic impedance of the target region are input into the pre-trained SVM to obtain the densification value.

[0060] Specifically, when calculating the comprehensive attenuation parameter, for each independently analyzed target region, the ultrasonic attenuation coefficient values ​​of all preset narrowband frequency bands are collected. After normalizing the attenuation coefficient values, a preset weighting factor is assigned to the normalized attenuation coefficient of each frequency band. The weighting factor is a hyperparameter; if the attenuation of a certain frequency band is more sensitive to changes in the densification degree of a specific material, then the weighting factor will be larger; of course, other methods can also be used to determine the weighting factor. The normalized attenuation coefficient of each frequency band is multiplied by its corresponding weighting factor, and the results of all frequency bands are summed to obtain the comprehensive attenuation parameter of the target region.

[0061] Before calculating the densification value using the trained SVM model, training is required. The training dataset contains a series of material samples with known densification levels. These samples undergo the same ultrasonic testing as the test object, and their combined attenuation parameters and acoustic impedance values ​​are calculated. These parameters constitute the input feature vector of the training samples, while their known densification levels serve as the labels for the training samples. When testing the test object, the calculated combined attenuation parameters and acoustic impedance values ​​for each target region are combined into a feature vector, which is then input into the trained SVM model to obtain the densification value.

[0062] The material properties and defect distribution inside the test object are often not uniform. In an optional embodiment, based on the spatial distribution gradient of the densification value and the energy accumulation region of the heat flow response characteristic map, the multi-channel ultrasonic echo signal and the infrared thermal imaging image are partitioned to obtain multiple data analysis subsets, including:

[0063] Spatial interpolation is performed on the densification values ​​of all target regions to generate a densification value distribution map; the gradient magnitude of the densification value distribution map is calculated to obtain a densification gradient map;

[0064] Inter-frame difference calculation is performed on the infrared thermal image sequence to extract the heat flow response feature map and identify energy accumulation areas with energy higher than the preset energy threshold;

[0065] On the densification gradient map, the boundary regions with gradient values ​​higher than a preset gradient threshold are identified by image segmentation method, and the test object is divided into multiple dense and uniform regions.

[0066] The boundary of the energy accumulation region is projected onto the divided dense and uniform regions. If the energy accumulation region spans multiple dense and uniform regions, the boundary of the energy accumulation region is prioritized, and the division result is adjusted to obtain a data analysis subset. The data in the multi-channel ultrasonic echo signal and the infrared thermal imaging image corresponding to the spatial range of each data analysis subset are respectively assigned to the corresponding subset.

[0067] The calculated density values ​​of each discrete target region are smoothed and meshed within the two-dimensional or three-dimensional detection space of the test object through spatial interpolation to obtain a density value distribution map. Further, the spatial gradient of the density value distribution map is calculated, and then the gradient magnitude is calculated to obtain a density gradient map. When extracting the heat flow response feature map, difference images between consecutive frames are calculated. These difference image sequences reflect the rate of temperature change over time, i.e., the dynamic response of heat flow. To obtain a comprehensive feature map, the difference images within a certain time period can be accumulated, averaged, or their maximum values ​​can be extracted and projected. Threshold segmentation is performed on the obtained heat flow response feature map. Specifically, during the initial partitioning based on the density gradient map, pixels or regions with gradient magnitudes higher than a preset gradient threshold in the gradient map are identified as potential boundaries. Combined with the boundaries, an image segmentation algorithm is used to divide the test object into multiple dense, uniform regions with relatively consistent internal density values.

[0068] If an energy accumulation region falls entirely within a dense, homogeneous region, that region remains unchanged. If an energy accumulation region spans the boundaries of multiple defined dense, homogeneous regions, the actual boundary of the energy accumulation region takes precedence, and the original boundaries of the dense, homogeneous regions are modified accordingly. In other words, the energy accumulation region itself can be considered an independent subset of the data analysis, or it can incorporate portions of intersecting dense, homogeneous regions. The original multi-channel ultrasound echo signals and infrared thermal imaging data are sliced ​​or mapped according to the spatial ranges defined by these data analysis subsets, ensuring that each subset contains ultrasound signal segments and infrared image features within its corresponding spatial region.

[0069] In an optional embodiment, obtaining the local joint dictionary of the data analysis subset based on the thermal flux response characteristic energy includes:

[0070] Initialize a basic dictionary containing ultrasonic feature atoms and infrared thermal feature atoms, and calculate the average densification value and average thermal flux response feature energy of all target regions within the current data analysis subset;

[0071] If the average thermal flux response characteristic energy is not greater than the first thermal energy preset value, then through the iterative dictionary update process, the weight coefficient of the structural background atom is increased, and at the same time, L1 norm sparsity constraints are applied to the atoms in the basic dictionary that are related to the known defect patterns; otherwise, the weight coefficient of the defect pattern atom that matches the most likely defect pattern in the current data analysis subset is enhanced.

[0072] The optimized dictionary is used as a local joint dictionary for the data analysis subset.

[0073] When generating a local joint dictionary for each data analysis subset, an initial base cross-modal joint dictionary is first required. In one embodiment, the base dictionary contains a series of atoms representing typical ultrasonic signal features and a series of atoms representing typical infrared thermal image features. The average densification value and average thermal flux response characteristic energy of all target regions within the subset are calculated. If the average thermal flux response characteristic energy of the subset is not greater than a first preset thermal energy value, the subset is more likely to represent background structure or non-critical regions. Ultrasonic and infrared atoms primarily related to the inherent structure of the material, i.e., structural background atoms, are identified from the base dictionary. Then, the weight coefficients of the structural background atoms are increased through an iterative dictionary update. Simultaneously, to suppress potential misjudgments, L1 norm sparsity constraints are imposed or the weights are reduced on atoms in the base dictionary known to be associated with typical defect modes, making them less likely to be selected or contributing less when subsequently performing sparse representation of the subset signal. Otherwise, the subset is more likely to contain defects. Further analysis of the basic dictionary identifies ultrasonic and infrared atoms corresponding to typical thermo-acoustic coupling modes that various potential defects may generate under shared excitation. These atoms are called defect mode atoms. Based on the current data analysis, more specific acoustic and thermal characteristics within the subset are analyzed to enhance the representation of defect mode atoms that best match the signal characteristics of the current subset. The defect mode atoms that most likely match the existing defect modes are determined based on attenuation spectra and thermal energy. For example, if the attenuation spectrum of the subset shows that it is sensitive to high-frequency attenuation and has high thermal energy, the weights of high-frequency ultrasonic scattering atoms and rapidly heating infrared atoms associated with microcracks in the dictionary are increased.

[0074] While raw ultrasonic signals and infrared thermal image data contain rich information about material state and potential defects, this information is often high-dimensional, redundant, and susceptible to noise interference. Direct analysis and identification on the raw data or its simple combinations is inefficient and makes it difficult to extract essential features. In an optional embodiment, the ultrasonic signal features and infrared thermal image features within the corresponding data analysis subset are jointly sparsely represented using the local joint dictionary to obtain a set of cross-modal fusion coefficients, including:

[0075] Extract a preset ultrasonic signal feature vector from the multi-channel ultrasonic echo signals within the current data analysis subset. The ultrasonic signal feature vector includes the time-domain amplitude envelope, the frequency-domain energy spectral density, and the attenuation value of a specific frequency band.

[0076] Extract a preset infrared thermal image feature vector from the infrared thermal imaging images in the current data analysis subset. The infrared thermal image feature vector includes a sequence of pixel temperature values, temperature gradient amplitude, and average temperature change rate of a specific area.

[0077] The fused feature vector of the data analysis subset is obtained by concatenating the feature vector of the ultrasonic signal and the feature vector of the infrared thermal image.

[0078] The orthogonal matching pursuit method is used to perform sparse decomposition of the fused feature vector on the local joint dictionary of the data analysis subset, resulting in a set of cross-modal fusion coefficients projected onto the dictionary atoms.

[0079] For ultrasound signals, features are extracted from the A-scan signal of each channel. For example, time-domain features include peak amplitude, root mean square value, zero-crossing rate, energy within a specific time window, and the peak position and full width at half maximum (FWHM) of the amplitude envelope curve obtained after performing a Hilbert transform on the original A-scan signal. Frequency-domain features are obtained by performing a Fourier transform on the signal segment, including the energy spectral density distribution of the entire spectrum, amplitude at a specific center frequency, dominant frequency, bandwidth, and attenuation values ​​for specific frequency bands. Similarly, for infrared thermal imaging images, features are extracted from the image frame sequence corresponding to the subset or its average image. For example, spatial-domain features may include the average temperature value, temperature standard deviation, amplitude of the temperature gradient, maximum temperature rise, temperature rise rate, and cooling time constant for each pixel within the subset. The ultrasound signal features and infrared image features are then stitched together.

[0080] The fused feature vector of the current data analysis subset was obtained. and the local union dictionary specific to this subset After that, exist Perform sparse decomposition on the top to obtain a set of sparse coefficient vectors. , making ,and The number of non-zero elements should be minimized. Preferably, orthogonal matching pursuit or basis pursuit methods are used for sparse decomposition. The coefficient vector obtained through these sparse decompositions is... The non-zero elements and their corresponding amplitudes constitute the aforementioned set of cross-modal fusion coefficients.

[0081] Compared to single-modal features or simple feature splicing, fusion coefficients include complementary information from both ultrasonic and infrared modalities, enabling the recognition model to learn more complex defect patterns and thus have a better ability to identify defects of different types, sizes, and depths.

[0082] In an optional embodiment, determining the defect information in the object under test based on the cross-modal fusion coefficients and the defect identification model includes:

[0083] The cross-modal fusion coefficients obtained from each data analysis subset are input into the defect identification model to obtain the probability of defects existing in the data analysis subset and the defect type label;

[0084] Spatial mapping is performed on the defect probabilities and type labels of all data analysis subsets to generate a defect distribution map of the entire object under test.

[0085] Defect identification models can employ support vector machines (SVMs) or multilayer perceptrons. The model's input is cross-modal fusion coefficients, and its output is a K-dimensional vector, with each element corresponding to a probability of a defect type. During actual detection, the cross-modal fusion coefficients calculated for each new data analysis subset are input into the trained defect identification model, allowing for real-time acquisition of the defect probability and predicted defect type for that subset. Each data analysis subset corresponds to a specific physical region on the surface of the object under test; therefore, the defect probability of each subset can be explicitly expressed at its corresponding spatial coordinates or the center point of the region.

[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. In addition, various different implementations of the embodiments of the present invention can be arbitrarily combined, as long as they do not violate the spirit of the embodiments of the present invention, and they should also be regarded as the content disclosed in the embodiments of the present invention.

Claims

1. An integrated detection method for ultrasound and infrared thermal imaging with a shared excitation source, characterized in that, include: A shared excitation source is applied to the object under test, and multi-channel ultrasonic echo signals and infrared thermal imaging image sequences are acquired simultaneously. The frequency band ultrasonic attenuation coefficient and acoustic impedance of each target region are calculated based on the multi-channel ultrasonic echo signal; the densification value is calculated based on the frequency band ultrasonic attenuation coefficient and the acoustic impedance. Heat flow response feature maps are extracted from infrared thermal imaging image sequences; based on the spatial distribution gradient of the densification value and the energy accumulation region of the heat flow response feature maps, the multi-channel ultrasonic echo signals and the infrared thermal imaging images are partitioned to obtain multiple data analysis subsets; For each data analysis subset, a local joint dictionary is obtained based on the thermal response characteristic energy. The ultrasonic signal features and infrared thermal imaging image features within the corresponding data analysis subset are jointly sparsely represented using the local joint dictionary to obtain a set of cross-modal fusion coefficients. Based on the cross-modal fusion coefficients and the defect identification model, the defect information in the test object is determined. Based on the spatial distribution gradient of the densification value and the energy accumulation region of the heat flow response characteristic map, the multi-channel ultrasonic echo signal and the infrared thermal imaging image are partitioned to obtain multiple data analysis subsets, including: Spatial interpolation is performed on the densification values ​​of all target regions to generate a densification value distribution map; the gradient magnitude of the densification value distribution map is calculated to obtain a densification gradient map; Inter-frame difference calculation is performed on the infrared thermal imaging image sequence to extract the heat flow response feature map and identify energy accumulation areas with energy higher than the preset energy threshold. On the densification gradient map, the boundary regions with gradient values ​​higher than a preset gradient threshold are identified by image segmentation method, and the test object is divided into multiple dense and uniform regions. The boundary of the energy accumulation region is projected onto the divided dense and uniform regions. If the energy accumulation region spans multiple dense and uniform regions, the boundary of the energy accumulation region is prioritized, and the division result is adjusted to obtain a data analysis subset. The data in the multi-channel ultrasonic echo signal and the infrared thermal imaging image corresponding to the spatial range of each data analysis subset are respectively assigned to the corresponding subset.

2. The method according to claim 1, characterized in that, The calculation of the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target region based on the multi-channel ultrasonic echo signal includes: The preprocessed ultrasonic echo signals of each channel are converted to the frequency domain through short-time Fourier transform or wavelet packet decomposition to obtain spectral information under different time windows; for at least three preset narrowband frequency bands with different center frequencies, the signal energy attenuation curve of each channel in the narrowband frequency band as a function of propagation depth is extracted; the energy attenuation curve is subjected to exponential fitting to obtain the frequency band ultrasonic attenuation coefficient of each target area in each preset narrowband frequency band. The acoustic impedance of each target region is calculated based on the material density and sound velocity.

3. The method according to claim 1, characterized in that, The process of calculating the densification value based on the ultrasonic attenuation coefficient of the frequency band and the acoustic impedance includes: For each target region, the ultrasonic attenuation coefficient values ​​calculated under all preset narrowband frequency bands are normalized. The comprehensive attenuation parameter is obtained by multiplying the normalized ultrasonic attenuation coefficients of each frequency band by the weighting factors of the corresponding ultrasonic attenuation coefficients of different frequency bands and summing the results. The integrated attenuation parameter and the acoustic impedance of the target region are input into the pre-trained SVM to obtain the densification value.

4. The method according to claim 1, characterized in that, The local joint dictionary for obtaining the data analysis subset based on the characteristic energy of the heat flow response includes: Initialize a basic dictionary containing ultrasonic feature atoms and infrared thermal feature atoms, and calculate the average densification value and average thermal flux response feature energy of all target regions within the current data analysis subset; If the average thermal flux response characteristic energy is not greater than the first thermal energy preset value, then through the iterative dictionary update process, the weight coefficient of the structural background atoms is increased, and at the same time, L1 norm sparsity constraints are applied to the atoms in the basic dictionary that are related to the known defect patterns; otherwise, the weight coefficient of the defect pattern atoms that match the most likely defect patterns in the current data analysis subset is enhanced. The optimized dictionary is used as a local joint dictionary for the data analysis subset.

5. The method according to claim 1, characterized in that, The method utilizes the local joint dictionary to perform joint sparse representation of the ultrasonic signal features and infrared thermal imaging image features within the corresponding data analysis subset to obtain a set of cross-modal fusion coefficients, including: Extract a preset ultrasonic signal feature vector from the multi-channel ultrasonic echo signals within the current data analysis subset. The ultrasonic signal feature vector includes the time-domain amplitude envelope, the frequency-domain energy spectral density, and the attenuation value of a specific frequency band. Extract a preset infrared thermal imaging image feature vector from the infrared thermal imaging images in the current data analysis subset. The infrared thermal imaging image feature vector includes a sequence of pixel temperature values, temperature gradient amplitude, and average temperature change rate of a specific area. The fused feature vector of the data analysis subset is obtained by concatenating the feature vector of the ultrasonic signal and the feature vector of the infrared thermal imaging image. The orthogonal matching pursuit method is used to perform sparse decomposition of the fused feature vector on the local joint dictionary of the data analysis subset, resulting in a set of cross-modal fusion coefficients projected onto the dictionary atoms.

6. The method according to claim 1, characterized in that, The step of determining the defect information in the object under test based on the cross-modal fusion coefficients and the defect identification model includes: The cross-modal fusion coefficients obtained from each data analysis subset are input into the defect identification model to obtain the probability of defects existing in the data analysis subset and the defect type label; Spatial mapping is performed on the defect probabilities and type labels of all data analysis subsets to generate a defect distribution map of the entire object under test.

7. An integrated ultrasonic and infrared thermal imaging detection system with a shared excitation source, characterized in that, include: The acquisition and feature extraction unit is used to apply a shared excitation source to the object under test and simultaneously acquire multi-channel ultrasonic echo signals and infrared thermal imaging image sequences; calculates the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target region based on the multi-channel ultrasonic echo signals; and calculates the densification value based on the frequency band ultrasonic attenuation coefficient and the acoustic impedance. A subset partitioning unit is used to extract heat flow response feature maps from infrared thermal imaging image sequences; based on the spatial distribution gradient of the densification value and the energy accumulation region of the heat flow response feature map, the multi-channel ultrasonic echo signal and the infrared thermal imaging image are partitioned to obtain multiple data analysis subsets; The detection unit is used to obtain a local joint dictionary for each data analysis subset based on the thermal response characteristic energy, and to use the local joint dictionary to perform joint sparse representation of the ultrasonic signal features and infrared thermal imaging image features in the corresponding data analysis subset to obtain a set of cross-modal fusion coefficients; based on the cross-modal fusion coefficients and the defect identification model, the defect information in the test object is determined. Based on the spatial distribution gradient of the densification value and the energy accumulation region of the heat flow response characteristic map, the multi-channel ultrasonic echo signal and the infrared thermal imaging image are partitioned to obtain multiple data analysis subsets, including: Spatial interpolation is performed on the densification values ​​of all target regions to generate a densification value distribution map; the gradient magnitude of the densification value distribution map is calculated to obtain a densification gradient map; Inter-frame difference calculation is performed on the infrared thermal imaging image sequence to extract the heat flow response feature map and identify energy accumulation areas with energy higher than the preset energy threshold. On the densification gradient map, the boundary regions with gradient values ​​higher than a preset gradient threshold are identified by image segmentation method, and the test object is divided into multiple dense and uniform regions. The boundary of the energy accumulation region is projected onto the divided dense and uniform regions. If the energy accumulation region spans multiple dense and uniform regions, the boundary of the energy accumulation region is prioritized, and the division result is adjusted to obtain a data analysis subset. The data in the multi-channel ultrasonic echo signal and the infrared thermal imaging image corresponding to the spatial range of each data analysis subset are respectively assigned to the corresponding subset.

8. The system according to claim 7, characterized in that, The calculation of the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target region based on the multi-channel ultrasonic echo signal includes: The preprocessed ultrasonic echo signals of each channel are converted to the frequency domain through short-time Fourier transform or wavelet packet decomposition to obtain spectral information under different time windows; for at least three preset narrowband frequency bands with different center frequencies, the signal energy attenuation curve of each channel in the narrowband frequency band as a function of propagation depth is extracted; the energy attenuation curve is subjected to exponential fitting to obtain the frequency band ultrasonic attenuation coefficient of each target area in each preset narrowband frequency band. The acoustic impedance of each target region is calculated based on the material density and sound velocity.

9. The system according to claim 7, characterized in that, The process of calculating the densification value based on the ultrasonic attenuation coefficient of the frequency band and the acoustic impedance includes: For each target region, the ultrasonic attenuation coefficient values ​​calculated under all preset narrowband frequency bands are normalized. The comprehensive attenuation parameter is obtained by multiplying the normalized ultrasonic attenuation coefficients of each frequency band by the weighting factors of the corresponding ultrasonic attenuation coefficients of different frequency bands and summing the results. The integrated attenuation parameter and the acoustic impedance of the target region are input into the pre-trained SVM to obtain the densification value.

Images