Excitation source-shared ultrasonic and infrared thermal imaging integrated detection method and system

Abstract

The invention provides a shared excitation source ultrasonic and infrared thermal imaging integrated detection method and system, and the method comprises the steps: applying a shared excitation source to a to-be-detected object, synchronously collecting a multi-channel ultrasonic echo signal and an infrared thermal imaging image, dividing the to-be-detected object into a plurality of target regions, and calculating the frequency band ultrasonic attenuation coefficient and acoustic impedance of each region, further obtaining a densification value; after a heat flow response feature map is extracted from an infrared image, adaptive partitioning is carried out on multi-modal data according to the spatial gradient of a densified value and a heat flow response energy accumulation area, and a plurality of data analysis subsets are formed. For each subset, constructing a local joint dictionary according to the densification value and the heat flow feature of the subset; and performing joint sparse representation on the ultrasonic and infrared features in the subsets by using the dictionary to obtain a group of cross-modal fusion coefficients, accurately determining defect information in the object to be detected based on the fusion coefficients and a defect identification model, and generating a defect distribution diagram.

Description

Technical Field

[0001] The present application relates to the field of defect detection, and in particular to an integrated ultrasonic and infrared thermal imaging detection method and system with a shared excitation source. Background Art

[0002] Nondestructive testing (NDT) is a crucial tool for ensuring structural integrity, evaluating material properties, preventing catastrophic accidents, and controlling product quality. Ultrasonic testing utilizes the propagation characteristics of high-frequency sound waves within materials to detect internal defects by analyzing the reflected ultrasonic signals. However, its results are susceptible to the coupling between the probe and the surface of the object being tested, relying heavily on operator experience. Components with complex geometries present significant challenges, and its ability to characterize certain minor defects or early stages of material damage is limited. This can easily lead to missed detections, especially when the material contains significant microstructural heterogeneity. Infrared thermal imaging is another common testing method. When defects are present within a material, the defective area creates an abnormal temperature distribution on the surface. Infrared thermal imagers detect defects by capturing this infrared radiation. Thermal imaging, which uses ultrasound as an excitation source, produces a localized temperature rise at the defect site, resulting in high sensitivity and efficiency. However, infrared thermal imaging has a relatively limited detection depth and lacks accuracy for detecting deeply buried defects. Integrating these two methods for collaborative testing can improve comprehensiveness and reliability. However, existing integration methods often rely on post-sequential data comparison or simple data-level or image-level fusion, such as image overlay or simple feature weighting. These methods often fail to fully utilize the rich information inherent in ultrasonic signals. Using ultrasonic signals to enhance the accuracy of infrared thermal imaging inspections and more accurately distinguish between true defect signals and interference caused by the inherent material structure remains a pressing technical challenge. Summary of the Invention

[0003] To address the difficulty in effectively distinguishing defect signals from complex background interference in integrated ultrasonic and infrared thermal imaging detection methods, this application proposes an integrated ultrasonic and infrared thermal imaging detection method with a shared excitation source, including:

[0004] A shared excitation source is applied to the object to be tested, and a multi-channel ultrasonic echo signal and an infrared thermal imaging image sequence are synchronously acquired; a frequency band ultrasonic attenuation coefficient and an acoustic impedance of each target area are calculated based on the multi-channel ultrasonic echo signal; and a densification value is calculated based on the frequency band ultrasonic attenuation coefficient and the acoustic impedance;

[0005] extracting a heat flow response characteristic map from an infrared thermal imaging image sequence; partitioning the multi-channel ultrasonic echo signal and the infrared thermal imaging image to obtain a plurality of data analysis subsets based on a spatial distribution gradient of the densification value and an energy concentration region of the heat flow response characteristic map;

[0006] For each data analysis subset, a local joint dictionary of the data analysis subset is obtained according to the characteristic energy of the thermal flow response. The ultrasonic signal features and infrared thermal image features in the corresponding data analysis subset are jointly sparsely represented using the local joint dictionary to obtain a set of cross-modal fusion coefficients. Based on the cross-modal fusion coefficients and the defect recognition model, the defect information in the object to be tested is determined.

[0007] Optionally, the calculating of the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target area based on the multi-channel ultrasonic echo signal includes:

[0008] The preprocessed ultrasonic echo signals of each channel are converted into the frequency domain by short-time Fourier transform or wavelet packet decomposition to obtain spectrum information in different time windows; for at least three preset narrowband frequency bands with different center frequencies, the curve of signal energy attenuation versus propagation depth of each channel in the narrowband frequency band is extracted; the energy attenuation curve is exponentially fitted to obtain the frequency band ultrasonic attenuation coefficient of each target area in each preset narrowband frequency band;

[0009] The acoustic impedance of each target area is calculated based on the density and sound velocity of the material.

[0010] Optionally, the densification value is calculated according to the ultrasonic attenuation coefficient of the frequency band and the acoustic impedance, including:

[0011] For each target area, the frequency band ultrasonic attenuation coefficient values ​​calculated in all preset narrowband frequency bands are normalized;

[0012] The normalized ultrasonic attenuation coefficient of each frequency band is multiplied by the corresponding weight factor of the ultrasonic attenuation coefficient of different frequency bands and then summed to obtain the comprehensive attenuation parameter;

[0013] The comprehensive attenuation parameter and the acoustic impedance of the target area are input into the pre-trained SVM to obtain a densification value.

[0014] Optionally, partitioning the multi-channel ultrasonic echo signal and the infrared thermal imaging image to obtain a plurality of data analysis subsets based on the spatial distribution gradient of the densification value and the energy concentration area of ​​the heat flow response characteristic map includes:

[0015] Performing spatial interpolation on the densification values ​​of all target areas to generate a densification value distribution map; calculating the gradient amplitude of the densification value distribution map to obtain a densification gradient map;

[0016] Perform inter-frame difference calculation on the infrared thermal image sequence to extract the heat flow response feature map and identify the energy concentration area with energy higher than the preset energy threshold;

[0017] On the densified gradient map, an image segmentation method is used to identify boundary areas where the gradient value is higher than a preset gradient threshold, and the object to be tested is divided into a plurality of dense and uniform areas;

[0018] The boundary of the energy concentration area is projected onto the divided dense and uniform area. If the energy concentration area spans multiple dense and uniform areas, the boundary of the energy concentration area is given priority, and the division result is adjusted to obtain a data analysis subset; the data corresponding to the spatial range of each data analysis subset in the multi-channel ultrasonic echo signal and the infrared thermal imaging image are respectively classified into corresponding subsets.

[0019] Optionally, obtaining a local joint dictionary of a data analysis subset according to the heat flow response characteristic energy includes:

[0020] Initialize a basic dictionary containing ultrasonic characteristic atoms and infrared thermal characteristic atoms, and calculate the average densification value and average heat flow response characteristic energy of all target areas in the current data analysis subset;

[0021] If the average heat flux response characteristic energy is not greater than the first heat energy preset value, then the weight coefficient of the structural background atom is increased through an iterative dictionary update process, and an L1 norm sparsity constraint is applied to the atoms related to the known defect mode in the basic dictionary; otherwise, the weight coefficient of the defect mode atom that matches the most likely defect mode in the current data analysis subset is increased;

[0022] The optimized dictionary is used as the local joint dictionary of the data analysis subset.

[0023] Optionally, the using of the local joint dictionary to perform joint sparse representation on the ultrasound signal features and the infrared thermal image features in the corresponding data analysis subset to obtain a set of cross-modal fusion coefficients includes:

[0024] Extracting a preset ultrasonic signal feature vector from the multi-channel ultrasonic echo signal in the current data analysis subset, wherein the ultrasonic signal feature vector includes a time domain amplitude envelope, a frequency domain energy spectrum density, and a specific frequency band attenuation value;

[0025] Extracting a preset infrared thermal image feature vector from the infrared thermal imaging image in the current data analysis subset, wherein the infrared thermal image feature vector includes a pixel temperature value sequence, a temperature gradient amplitude, and an average temperature change rate of a specific area;

[0026] splicing the ultrasonic signal feature vector and the infrared thermal image feature vector to obtain a fused feature vector of the data analysis subset;

[0027] Sparse decomposition is performed on the fusion feature vector on the local joint dictionary of the data analysis subset by using an orthogonal matching pursuit method, to obtain a group of cross-modal fusion coefficients projected on the dictionary atoms.

[0028] Optionally, the defect information in the test object is determined based on the cross-modal fusion coefficients and a defect identification model, including:

[0029] The cross-modal fusion coefficients obtained by each data analysis subset are input into a defect identification model to obtain a defect probability and a defect type label of the data analysis subset;

[0030] The defect probability and the type label of all data analysis subsets are spatially mapped to generate a defect distribution map of the test object as a whole.

[0031] The application also provides an ultrasonic and infrared thermal imaging integrated detection system sharing an excitation source, including:

[0032] A collection and feature extraction unit is configured to apply the shared excitation source to the test object and synchronously collect a multi-channel ultrasonic echo signal and an infrared thermal imaging image sequence; the frequency band ultrasonic attenuation coefficient and the acoustic impedance of each target region are calculated based on the multi-channel ultrasonic echo signal; and the densification value is calculated according to the frequency band ultrasonic attenuation coefficient and the acoustic impedance.

[0033] A subset division unit is configured to extract a heat flow response feature map from the infrared thermal imaging image sequence; and the multi-channel ultrasonic echo signal and the infrared thermal imaging image are divided into a plurality of data analysis subsets according to the spatial distribution gradient of the densification value and the energy aggregation region of the heat flow response feature map.

[0034] A detection unit is configured to, for each data analysis subset, obtain a local joint dictionary of the data analysis subset according to the heat flow response feature energy, perform joint sparse representation on the ultrasonic signal features and the infrared thermal image features in the corresponding data analysis subset by using the local joint dictionary to obtain a group of cross-modal fusion coefficients, and determine the defect information in the test object based on the cross-modal fusion coefficients and a defect identification model.

[0035] Compared with the prior art, the application has the following beneficial effects:

[0036] This application uses the densification value obtained by calculating the frequency band ultrasonic attenuation coefficient and acoustic impedance as the local material parameter, and uses the spatial distribution gradient of the densification value and the energy concentration area of ​​the heat flow response characteristic map extracted from the infrared image to adaptively partition the ultrasonic and infrared data to form multiple data analysis subsets. For each data analysis subset, a basic cross-modal joint dictionary is optimized according to its densification value and heat flow response characteristic energy, so that the dictionary can more accurately express the signal characteristics of the specific area. The optimized local joint dictionary is used to jointly sparsely represent the ultrasonic and infrared features in the corresponding subset, and the obtained cross-modal fusion coefficient can better reflect the true state of the object to be tested. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of Example 1;

[0038] Figure 2 It is an infrared thermal imaging image;

[0039] Figure 3 Schematic diagram of short-time Fourier transform of ultrasonic signals;

[0040] Figure 4 Schematic diagram of energy changes in each frequency band. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0042] In a specific embodiment, the present application proposes an integrated detection method of ultrasound and infrared thermal imaging with a shared excitation source, such as Figure 1 Shown, including:

[0043] Step 1: Apply a shared excitation source to the object to be tested and synchronously acquire a multi-channel ultrasonic echo signal and an infrared thermal imaging image sequence; calculate the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target area based on the multi-channel ultrasonic echo signal; and calculate the densification value based on the frequency band ultrasonic attenuation coefficient and the acoustic impedance;

[0044] The excitation device applies a shared excitation source that can simultaneously stimulate ultrasonic propagation and thermal effects to the object under test. While applying the excitation, a multi-channel ultrasonic probe array is used to receive ultrasonic echo signals reflected or scattered from the inside of the object under test. At the same time, an infrared thermal imager is used to capture the changes in infrared radiation energy on the surface of the object under test during or after the excitation in a non-contact manner and convert it into infrared thermal imaging images arranged in time sequence, such as Figure 2 The multi-channel ultrasonic echo signals are processed to calculate the frequency band ultrasonic attenuation coefficient and acoustic impedance of the object in each target region. The target region is a spatial region corresponding to the element or beamformed unit of the ultrasonic probe. In one embodiment, the frequency band ultrasonic attenuation coefficient is calculated by analyzing the echo amplitude variation of the reflector at different depths and fitting the sound propagation model. The acoustic impedance is based on the density and sound speed of the material, where the density can be obtained according to the material of the object, and the sound speed is estimated from the transit time of the ultrasonic signal, or indirectly calculated from the reflection and transmission coefficients of the ultrasonic wave at the interface of different media. The calculated frequency band ultrasonic attenuation coefficient and acoustic impedance of each target region are calculated by a regression model or a neural network to obtain a densification value, which is a quantitative index representing the material density of the target region. In another embodiment, the sum of the attenuation coefficients of all frequency bands is calculated, and the weighted difference between the acoustic impedance and the sum of the attenuation coefficients is taken as the densification value.

[0045] Step 2, extracting a heat flow response feature map from the infrared thermal imaging image sequence; according to the spatial distribution gradient of the densification value and the energy aggregation region of the heat flow response feature map, the multi-channel ultrasonic echo signals and the infrared thermal imaging images are partitioned to obtain a plurality of data analysis subsets;

[0046] The infrared thermal imaging image sequence is analyzed to extract a heat flow response feature map. In one embodiment, the temperature change region is highlighted by difference operation on consecutive frames, or dynamic heat patterns are extracted from time series thermal images using principal component analysis or other blind source separation methods, or heat signal reconstruction methods are used to analyze heat decay curves to invert heat source characteristics. The internal defects or structural differences in the material will be abnormal regions in the heat flow response feature map.

[0047] The region with a sharp change in the spatial distribution of the densification value usually corresponds to the boundary where the internal structure or density of the material changes significantly. The energy concentrated region in the heat flow response feature map indicates that there is obvious thermal activity, which may be a defect. In one embodiment, the gradient of the densification value is used to preliminarily divide the region, and then the energy aggregation region in the heat flow response feature map is used to modify and refine the boundaries of the preliminary division, or the original ultrasonic and infrared data are divided into multiple data analysis subsets with different acoustic and thermal characteristics using image segmentation methods. Each data analysis subset corresponds to a local region with common density and thermal response of the object.

[0048] Step 3: For each data analysis subset, a local joint dictionary of the data analysis subset is obtained according to the characteristic energy of the thermal flow response, and the ultrasonic signal features and infrared thermal image features in the corresponding data analysis subset are jointly sparsely represented using the local joint dictionary to obtain a set of cross-modal fusion coefficients; based on the cross-modal fusion coefficients and the defect recognition model, the defect information in the object to be tested is determined.

[0049] For a specific data analysis subset, a basic cross-modal joint dictionary is generated based on the average densification value and average heat flux response characteristic energy within the subset region. This basic dictionary is pre-learned from a large number of typical ultrasonic signal features and infrared thermal image features, for example using the K-SVD algorithm or online dictionary learning methods. If the subset exhibits high densification but low thermal activity, or low densification and low thermal activity, it may be background structure or a non-critical area. In this case, dictionary optimization focuses on accurately representing this structural background, for example by increasing the weights of dictionary atoms associated with known background structures while imposing constraints on atoms associated with defects to make them less likely to be selected in subsequent signal representations, thereby obtaining a first local joint dictionary. On the contrary, if the subset shows high densification and obvious thermal activity, or is not highly densified but has significant thermal activity, defects may exist. The dictionary optimization will focus on enhancing atoms related to the thermoacoustic coupling modes of potential defects, and will further utilize the acoustic properties and the intensity of the thermal flow response energy in the subset to adjust these defect mode atoms, such as adjusting their shape, amplitude or weight in the dictionary, so that it can more accurately match and capture specific defect signals that may exist in the subset, and obtain the second local joint dictionary.

[0050] Then, the corresponding ultrasonic signal features and infrared thermal image features within the data analysis subset are extracted. Ultrasonic signal features include, but are not limited to, time-domain envelopes and spectral features, and infrared thermal imaging features include, but are not limited to, temperature change sequences and spatial gradients. These cross-modal features are then jointly sparsely represented on the local joint dictionary. Sparse representation approximates the original signal features using linear combinations of as few atoms as possible from the dictionary. The resulting combination coefficients are the cross-modal fusion coefficients. Preferably, sparse coding methods such as orthogonal matching pursuit, minimum angle regression, or basis pursuit are employed. The cross-modal fusion coefficients obtained from each data analysis subset are input into a pre-trained defect recognition model. The defect recognition model can be a traditional machine learning classifier, such as a support vector machine or random forest, or a deep learning model, such as a convolutional neural network or a recurrent neural network. The present invention does not specifically limit the defect recognition model. The defect recognition model determines whether a defect exists in the subset, as well as the possible defect type and severity, based on the characteristic patterns of the input fusion coefficients. Ultimately, the judgment results of all subsets are aggregated to form an assessment of the overall defect status of the object under test.

[0051] The attenuation coefficient of a single frequency may not be able to fully capture subtle changes in the internal state of the material, especially for complex materials with multiple potential defect types or structural inhomogeneities. Acoustic impedance, as an inherent parameter of the material, is closely related to the density and elastic properties of the material and is an important basis for judging the uniformity of the material and the interface characteristics. In an optional embodiment, the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target area are calculated based on the multi-channel ultrasonic echo signal, including:

[0052] The preprocessed ultrasonic echo signals of each channel are converted into the frequency domain by short-time Fourier transform or wavelet packet decomposition to obtain spectrum information in different time windows; for at least three preset narrowband frequency bands with different center frequencies, the curve of signal energy attenuation versus propagation depth of each channel in the narrowband frequency band is extracted; the energy attenuation curve is exponentially fitted to obtain the frequency band ultrasonic attenuation coefficient of each target area in each preset narrowband frequency band;

[0053] The acoustic impedance of each target area is calculated based on the density and sound velocity of the material.

[0054] Specifically, the original A-scan echo signals collected from each ultrasonic transducer channel are pre-processed, such as using bandpass filtering to remove noise unrelated to the excitation frequency, and performing signal amplification or normalization. For each pre-processed A-scan signal, the short-time Fourier transform (STFT) method or wavelet packet decomposition is used to decompose the signal into orthogonal wavelet packet coefficients at different scales and frequencies. Figure 3The results of a short-time Fourier transform of an ultrasonic signal are shown. After obtaining the time-frequency spectrum information, the energy values ​​of the signal varying over time within three or more pre-set narrowband frequency bands, such as those with center frequencies f1, f2, and f3, each with a certain bandwidth, such as f1±Δf1, are extracted. Preferably, this is obtained by integrating the energy of the corresponding frequency band on the spectrum graph or calculating the coefficient energy of the corresponding frequency band after wavelet packet decomposition. Figure 4 The energy conversion of each frequency band is shown. For each channel and each preset frequency band, a curve of energy attenuation with depth can be obtained. The acoustic energy in a homogeneous medium decays approximately exponentially, that is, , where E(d) is the energy at depth d, E0 is the initial energy, and α is the attenuation coefficient. The extracted energy attenuation curve is logarithmically fitted with a linear fit or directly subjected to a nonlinear exponential fit to obtain the frequency band ultrasonic attenuation coefficient α of the target area in the specific frequency band.

[0055] When calculating acoustic impedance, one method is based on the basic physical properties of the material. The density ρ of the material to be tested is obtained from the material manual, and the sound velocity c in the material is estimated from the ultrasonic test data, for example, by measuring the transit time of the ultrasonic wave in a sample of known thickness, or by analyzing the time interval between multiple bottom surface echoes. The acoustic impedance Z of the target area is then calculated by the formula Z=ρ c is calculated. In yet another embodiment, when an ultrasonic wave is perpendicularly incident on the interface between a reference material of known acoustic impedance Z1 and a target area of ​​known acoustic impedance Z2, its reflection coefficient R is determined by measuring the incident wave amplitude Ai and the reflected wave amplitude Ar. The relationship between the reflection coefficient R and the acoustic impedances of the two media is R = (Z2 - Z1) / (Z2 + Z1). If Z1 is known, the acoustic impedance Z2 of the target area can be inversely calculated from the measured reflection coefficient R.

[0056] A single acoustic parameter is difficult to fully and accurately evaluate the true densification level. In an optional embodiment, the densification value is calculated based on the ultrasonic attenuation coefficient of the frequency band and the acoustic impedance, including:

[0057] For each target area, the frequency band ultrasonic attenuation coefficient values ​​calculated in all preset narrowband frequency bands are normalized;

[0058] The normalized ultrasonic attenuation coefficient of each frequency band is multiplied by the corresponding weight factor of the ultrasonic attenuation coefficient of different frequency bands and then summed to obtain the comprehensive attenuation parameter;

[0059] The comprehensive attenuation parameter and the acoustic impedance of the target area are input into the pre-trained SVM to obtain a densification value.

[0060] Specifically, when calculating the comprehensive attenuation parameters, for each independently analyzed target area, the frequency band ultrasonic attenuation coefficient values ​​in all preset narrowband frequency bands are collected. After the attenuation coefficient values ​​are normalized, a preset weight factor is assigned to the normalized attenuation coefficient of each frequency band. The weight factor is a hyperparameter. If the attenuation of a certain frequency band is more sensitive to changes in the degree of densification of a specific material, the weight factor will be larger; of course, other methods can also be used to determine the weight factor. Multiply the normalized attenuation coefficient of each frequency band by its corresponding weight factor, and add up the results of all frequency bands to obtain the comprehensive attenuation parameter of the target area.

[0061] Before using the trained support vector machine model to calculate densification values, a training dataset is required. The training dataset consists of a series of material samples with known densification levels. These samples undergo the same ultrasonic testing as the object under test, and their comprehensive attenuation parameters and acoustic impedance values ​​are calculated. These parameters form the input feature vectors for the training samples, while the known densification levels serve as labels for the training samples. When testing the object under test, the calculated comprehensive attenuation parameters and acoustic impedance values ​​for each target area are combined into a feature vector and input into the trained SVM model to obtain the densification value.

[0062] The material properties and defect distribution inside the object to be tested are often not uniform. In an optional embodiment, the multi-channel ultrasonic echo signal and the infrared thermal imaging image are partitioned according to the spatial distribution gradient of the densification value and the energy concentration area of ​​the heat flow response characteristic diagram to obtain multiple data analysis subsets, including:

[0063] Performing spatial interpolation on the densification values ​​of all target areas to generate a densification value distribution map; calculating the gradient amplitude of the densification value distribution map to obtain a densification gradient map;

[0064] Perform inter-frame difference calculation on the infrared thermal image sequence to extract the heat flow response feature map and identify the energy concentration area with energy higher than the preset energy threshold;

[0065] On the densified gradient map, an image segmentation method is used to identify boundary areas where the gradient value is higher than a preset gradient threshold, and the object to be tested is divided into a plurality of dense and uniform areas;

[0066] The boundary of the energy concentration area is projected onto the divided dense and uniform area. If the energy concentration area spans multiple dense and uniform areas, the boundary of the energy concentration area is given priority, and the division result is adjusted to obtain a data analysis subset; the data corresponding to the spatial range of each data analysis subset in the multi-channel ultrasonic echo signal and the infrared thermal imaging image are respectively classified into corresponding subsets.

[0067] The calculated densification values of each target region of the discrete distribution are smoothed and gridded in the two-dimensional or three-dimensional detection space of the object to be detected by spatial interpolation, to obtain a densification value distribution map. Further, the spatial gradient of the densification value distribution map is calculated, and then the gradient amplitude is calculated, to obtain a densification gradient map. When extracting the heat flow response feature map, difference images between consecutive frames are calculated, and these difference image sequences can reflect the rate of change of temperature over time, i.e. the dynamic response of heat flow. In order to obtain a comprehensive feature map, the difference images within a certain time period can be accumulated, averaged or have their maximum values projected. The obtained heat flow response feature map is subjected to threshold segmentation. Specifically, when preliminary partitioning is performed based on the densification gradient map, the pixel points or regions in the gradient map whose gradient amplitudes are higher than a preset gradient threshold value are identified as potential boundaries. The object to be detected is divided into multiple densification uniform regions with relatively consistent internal densification values by using an image segmentation algorithm in combination with the boundaries.

[0068] If a certain energy aggregation region falls completely within a densification uniform region, the densification uniform region remains unchanged; if an energy aggregation region crosses the boundaries of multiple divided densification uniform regions, the actual boundary of the energy aggregation region is prioritized, and the original densification uniform region boundary is corrected, i.e. the energy aggregation region itself can be regarded as an independent data analysis subset, or the partial regions of the densification uniform regions intersected by the energy aggregation region are merged. The original multi-channel ultrasonic echo signal and infrared thermal imaging image data are sliced or mapped according to the spatial range defined by the data analysis subsets, so that each subset contains the ultrasonic signal segment and infrared image feature within its corresponding spatial region.

[0069] In an optional embodiment, the local joint dictionary of the data analysis subset according to the heat flow response feature energy comprises:

[0070] A basic dictionary containing ultrasonic feature atoms and infrared thermal feature atoms is initialized, and the average densification value and the average heat flow response feature energy of all target regions in the current data analysis subset are calculated;

[0071] If the average heat flow response feature energy is not greater than a first thermal energy preset value, the weight coefficient of the structural background atom is increased by an iterative dictionary updating process, and an LI norm sparsity constraint is applied to the atoms in the basic dictionary that are related to known defect modes; otherwise, the weight coefficient of the defect mode atom that matches the defect mode most likely to exist in the current data analysis subset is enhanced;

[0072] The optimized dictionary is used as the local joint dictionary of the data analysis subset.

[0073] In generating the local joint dictionary for each data analysis subset, an initial base cross-modal joint dictionary is needed first. In one embodiment, the base dictionary contains a series of atoms capable of representing typical ultrasonic signal features and a series of atoms capable of representing typical infrared thermal image features. The average densification value and the average thermal flow response feature energy of all target regions in the subset are calculated. If the average thermal flow response feature energy of the subset is not greater than a first thermal energy preset value, the subset is more likely to represent background structure or non-critical regions, and the ultrasonic atoms and infrared atoms mainly related to material inherent structure, i.e. structure background atoms, are identified from the base dictionary. Then, the weight coefficients of the structure background atoms are increased by iterative dictionary updating. At the same time, in order to suppress potential misjudgment, L1 norm sparsity constraints are imposed on or the weight of the atoms in the base dictionary known to be related to typical defect modes is reduced, so that they are more difficult to be selected or contribute less in subsequent sparse representation of the signals in the subset. Otherwise, the subset is more likely to contain defects, and further, ultrasonic atoms and infrared atoms corresponding to typical thermal-acoustic coupling modes that potential defects may produce under shared excitation are identified from the base dictionary, which are referred to as defect mode atoms. The expression ability of the defect mode atoms that best match the current subset signal features is enhanced according to the more specific acoustic and thermal characteristics within the current data analysis subset. Among them, the defect mode atoms that best match the most likely existing defect modes are determined according to the attenuation spectrum and thermal energy, for example, if the attenuation spectrum of the subset shows that it is sensitive to high-frequency attenuation and the thermal energy is high, the weight of the high-frequency ultrasonic scattering atom and the fast temperature rise infrared atom related to micro-cracks in the dictionary is increased.

[0074] Although the original ultrasonic signals and infrared thermal image data contain rich information about the material state and potential defects, this information is often high-dimensional, redundant, and may be disturbed by noise. Direct analysis and identification on the original data or their simple combinations are inefficient and difficult to extract essential features. In an optional embodiment, the local joint dictionary is used to jointly sparsely represent the ultrasonic signal features and the infrared thermal image features in the corresponding data analysis subset to obtain a set of cross-modal fusion coefficients, including:

[0075] A preset ultrasonic signal feature vector is extracted from the multi-channel ultrasonic echo signal in the current data analysis subset, and the ultrasonic signal feature vector includes a time-domain amplitude envelope, a frequency energy spectrum density, and a specific frequency band attenuation value.

[0076] A preset infrared thermal image feature vector is extracted from the infrared thermal imaging image in the current data analysis subset, and the infrared thermal image feature vector includes a pixel temperature value sequence, a temperature gradient amplitude, and a specific region average temperature change rate.

[0077] splicing the ultrasonic signal feature vector and the infrared thermal image feature vector to obtain a fused feature vector of the data analysis subset;

[0078] The orthogonal matching pursuit method is used to perform sparse decomposition on the fused feature vector on the local joint dictionary of the data analysis subset to obtain a set of cross-modal fusion coefficients projected on dictionary atoms.

[0079] For ultrasonic signals, features are extracted for the A-scan signal of each channel. For example, time domain features include the peak amplitude, root mean square value, zero-crossing rate, energy within a specific time window, etc. of the signal, as well as the peak position and half-height width of the amplitude envelope curve obtained after the Hilbert transform of the original A-scan signal. Frequency domain features are obtained by Fourier transforming the signal segment, including the energy spectral density distribution of the entire spectrum, the amplitude of a specific center frequency, the main frequency, the bandwidth, etc., as well as the attenuation value of a specific frequency band. Similarly, for infrared thermal imaging images, features are extracted from the image frame sequence corresponding to the subset or its average image. For example, spatial domain features can include the average temperature value of each pixel in the subset, the temperature standard deviation, the amplitude of the temperature gradient, the maximum temperature rise, the temperature rise rate, the cooling time constant, etc. The ultrasonic signal features and the infrared image features are spliced.

[0080] Obtained the fusion feature vector of the current data analysis subset And the local union dictionary specific to that subset Afterwards, exist Perform sparse decomposition on it to obtain a set of sparse coefficient vectors , making ,and The number of non-zero elements in is as small as possible. Preferably, orthogonal matching pursuit or basis pursuit is used for sparse decomposition. The coefficient vectors obtained by these sparse decompositions are , its non-zero elements and their corresponding amplitudes constitute the set of cross-modal fusion coefficients.

[0081] Compared with the features of a single modality or simple feature splicing, the fusion coefficient includes the complementary information of the ultrasonic and infrared modalities, enabling the recognition model to learn more complex defect patterns, thereby having better recognition capabilities for defects of different types, sizes, and depths.

[0082] In an optional embodiment, determining defect information in the object to be tested based on the cross-modal fusion coefficient and the defect recognition model includes:

[0083] The cross-modal fusion coefficients obtained from each data analysis subset are input into the defect recognition model to obtain the probability of defects in the data analysis subset and the defect type label;

[0084] The defect probability and type label of each data analysis subset are mapped in space to generate a defect distribution map of the whole object under test.

[0085] The defect recognition model can be a support vector machine or a multilayer perceptron, etc. The input of the model is the cross-modal fusion coefficient, and the output of the model is a K-dimensional vector, each element corresponding to the probability of a type. In actual detection, the cross-modal fusion coefficient calculated by each new data analysis subset is input into the trained defect recognition model, and the defect probability and predicted defect type of the subset can be obtained in real time. Each data analysis subset corresponds to a specific physical region on the surface of the object under test, so the defect probability of each subset can be explicitly displayed at the corresponding spatial coordinates or region center point position.

[0086] The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. In addition, the various different embodiments of the embodiments of the present application can also be combined arbitrarily, as long as it does not deviate from the idea of the embodiments of the present application, it should also be considered as the disclosed content of the embodiments of the present application.

Claims

1. An integrated detection method of ultrasound and infrared thermal imaging with a shared excitation source, characterized in that: include: Apply a shared excitation source to the object to be tested, and synchronously acquire multi-channel ultrasonic echo signals and infrared thermal imaging image sequences; Calculating the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target area based on the multi-channel ultrasonic echo signal; and calculating the densification value based on the frequency band ultrasonic attenuation coefficient and the acoustic impedance; extracting a heat flow response characteristic map from an infrared thermal imaging image sequence; partitioning the multi-channel ultrasonic echo signal and the infrared thermal imaging image to obtain a plurality of data analysis subsets based on a spatial distribution gradient of the densification value and an energy concentration region of the heat flow response characteristic map; For each data analysis subset, a local joint dictionary of the data analysis subset is obtained according to the characteristic energy of the thermal flow response. The ultrasonic signal features and infrared thermal image features in the corresponding data analysis subset are jointly sparsely represented using the local joint dictionary to obtain a set of cross-modal fusion coefficients. Based on the cross-modal fusion coefficients and the defect recognition model, the defect information in the object to be tested is determined.

2. The method according to claim 1, characterized in that The calculating of the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target area based on the multi-channel ultrasonic echo signal includes: The preprocessed ultrasonic echo signals of each channel are converted into the frequency domain by short-time Fourier transform or wavelet packet decomposition to obtain spectrum information in different time windows; for at least three preset narrowband frequency bands with different center frequencies, the curve of signal energy attenuation versus propagation depth of each channel in the narrowband frequency band is extracted; the energy attenuation curve is exponentially fitted to obtain the frequency band ultrasonic attenuation coefficient of each target area in each preset narrowband frequency band; The acoustic impedance of each target area is calculated based on the density and sound velocity of the material.

3. The method according to claim 1, characterized in that The densification value is calculated based on the ultrasonic attenuation coefficient of the frequency band and the acoustic impedance, including: For each target area, the frequency band ultrasonic attenuation coefficient values ​​calculated in all preset narrowband frequency bands are normalized; The normalized ultrasonic attenuation coefficient of each frequency band is multiplied by the corresponding weight factor of the ultrasonic attenuation coefficient of different frequency bands and then summed to obtain the comprehensive attenuation parameter; The comprehensive attenuation parameter and the acoustic impedance of the target area are input into the pre-trained SVM to obtain a densification value.

4. The method according to claim 1, wherein Partitioning the multi-channel ultrasonic echo signal and the infrared thermal imaging image to obtain a plurality of data analysis subsets based on the spatial distribution gradient of the densification value and the energy concentration area of ​​the heat flow response characteristic map includes: Performing spatial interpolation on the densification values ​​of all target areas to generate a densification value distribution map; calculating the gradient amplitude of the densification value distribution map to obtain a densification gradient map; Perform inter-frame difference calculation on the infrared thermal image sequence to extract the heat flow response feature map and identify the energy concentration area with energy higher than the preset energy threshold; On the densified gradient map, an image segmentation method is used to identify boundary areas where the gradient value is higher than a preset gradient threshold, and the object to be tested is divided into a plurality of dense and uniform areas; The boundary of the energy concentration area is projected onto the divided dense and uniform area. If the energy concentration area spans multiple dense and uniform areas, the boundary of the energy concentration area is given priority, and the division result is adjusted to obtain a data analysis subset; the data corresponding to the spatial range of each data analysis subset in the multi-channel ultrasonic echo signal and the infrared thermal imaging image are respectively classified into corresponding subsets.

5. The method according to claim 1, wherein The local joint dictionary of the data analysis subset obtained according to the heat flow response characteristic energy includes: Initialize a basic dictionary containing ultrasonic characteristic atoms and infrared thermal characteristic atoms, and calculate the average densification value and average heat flow response characteristic energy of all target areas in the current data analysis subset; If the average heat flux response characteristic energy is not greater than the first thermal energy preset value, then the weight coefficients of the structural background atoms are increased through an iterative dictionary update process, and an L1 norm sparsity constraint is applied to the atoms related to the known defect modes in the basic dictionary; otherwise, the weight coefficients of the defect mode atoms that match the most likely defect mode in the current data analysis subset are increased; The optimized dictionary is used as the local joint dictionary of the data analysis subset.

6. The method according to claim 1, characterized in that The method of using the local joint dictionary to perform a joint sparse representation on the ultrasonic signal features and the infrared thermal image features in the corresponding data analysis subset to obtain a set of cross-modal fusion coefficients includes: Extracting a preset ultrasonic signal feature vector from the multi-channel ultrasonic echo signal in the current data analysis subset, wherein the ultrasonic signal feature vector includes a time domain amplitude envelope, a frequency domain energy spectrum density, and a specific frequency band attenuation value; Extracting a preset infrared thermal image feature vector from the infrared thermal imaging image in the current data analysis subset, wherein the infrared thermal image feature vector includes a pixel temperature value sequence, a temperature gradient amplitude, and an average temperature change rate of a specific area; splicing the ultrasonic signal feature vector and the infrared thermal image feature vector to obtain a fused feature vector of the data analysis subset; The orthogonal matching pursuit method is used to perform sparse decomposition on the fused feature vector on the local joint dictionary of the data analysis subset to obtain a set of cross-modal fusion coefficients projected on dictionary atoms.

7. The method according to claim 1, characterized in that The determining of defect information in the object to be tested based on the cross-modal fusion coefficient and the defect recognition model includes: The cross-modal fusion coefficients obtained from each data analysis subset are input into the defect recognition model to obtain the probability of defects in the data analysis subset and the defect type label; The defect probabilities and type labels of all data analysis subsets are spatially mapped to generate a defect distribution map of the entire DUT.

8. An integrated ultrasonic and infrared thermal imaging detection system with a shared excitation source, characterized in that: include: An acquisition and feature extraction unit is configured to apply a shared excitation source to the object to be tested and synchronously acquire a multi-channel ultrasonic echo signal and an infrared thermal imaging image sequence; calculate the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target area based on the multi-channel ultrasonic echo signal; and calculate the densification value based on the frequency band ultrasonic attenuation coefficient and the acoustic impedance; a subset partitioning unit configured to extract a heat flow response characteristic map from an infrared thermal imaging image sequence; and partition the multi-channel ultrasonic echo signal and the infrared thermal imaging image to obtain a plurality of data analysis subsets based on a spatial distribution gradient of the densification value and an energy concentration region of the heat flow response characteristic map; A detection unit is configured to obtain, for each data analysis subset, a local joint dictionary of the data analysis subset based on the characteristic energy of the heat flow response, and use the local joint dictionary to perform a joint sparse representation of the ultrasonic signal features and the infrared thermal image features in the corresponding data analysis subset to obtain a set of cross-modal fusion coefficients; and determine defect information in the object to be tested based on the cross-modal fusion coefficients and a defect recognition model.

9. The system according to claim 8, characterized in that The calculating of the frequency band ultrasonic attenuation coefficient and acoustic impedance of each target area based on the multi-channel ultrasonic echo signal includes: The preprocessed ultrasonic echo signals of each channel are converted into the frequency domain by short-time Fourier transform or wavelet packet decomposition to obtain spectrum information in different time windows; for at least three preset narrowband frequency bands with different center frequencies, the curve of signal energy attenuation versus propagation depth of each channel in the narrowband frequency band is extracted; the energy attenuation curve is exponentially fitted to obtain the frequency band ultrasonic attenuation coefficient of each target area in each preset narrowband frequency band; The acoustic impedance of each target area is calculated based on the density and sound velocity of the material.

10. The system according to claim 8, wherein: The densification value is calculated based on the ultrasonic attenuation coefficient of the frequency band and the acoustic impedance, including: For each target area, the frequency band ultrasonic attenuation coefficient values ​​calculated in all preset narrowband frequency bands are normalized; The normalized ultrasonic attenuation coefficient of each frequency band is multiplied by the corresponding weight factor of the ultrasonic attenuation coefficient of different frequency bands and then summed to obtain the comprehensive attenuation parameter; The comprehensive attenuation parameter and the acoustic impedance of the target area are input into the pre-trained SVM to obtain a densification value.

Images