A subsurface defect multi-mode laser ultrasonic C-scan image reconstruction method

Abstract

The application discloses a sub-surface defect multi-mode laser ultrasonic C-scan image reconstruction method. The method comprises the following steps: a laser ultrasonic detection device is used to perform two-dimensional scanning on a measured object to obtain a multi-mode ultrasonic A-scan time domain signal data set; the data set is converted into a frequency domain signal, and position coding is performed according to the signal acquisition position; the frequency domain signal and the position coding are spliced, and the similarity between signals is calculated; according to the calculated signal similarity, a certain number of signals are selected to construct a graph data set; the graph data set is input into a trained graph neural network model to obtain the classification result of the defect signal at each position, to judge whether it is from a defect area, and to reconstruct a C-scan image according to the original acquisition position of each A-scan signal and the classification result. The application can realize high-resolution scanning imaging of sub-surface defects, and significantly improve the accuracy and reliability of sub-surface defect detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent manufacturing equipment, more particularly, to a sub-surface defect multi-mode laser ultrasonic C-scan image reconstruction method. BACKGROUND

[0002] Metal Additive Manufacturing (MAM) is an advanced manufacturing technology that can directly process complex-shaped parts according to three-dimensional model data. Compared with traditional subtractive manufacturing technology, MAM has the advantages of unlimited part structure and high material utilization, and is widely used in aerospace and other industrial fields. Commonly used MAM technology uses high-energy sources such as laser and electron beam to generate high temperature to melt metal powder or metal wire, and realizes rapid manufacturing of parts in a layer-by-layer accumulation manner. However, due to the inherent properties of the material, and the unreasonable setting of energy beam parameters (such as power, spot size, etc.) and processing parameters, cracks, holes, un-melted and residual stress defects are often generated, which will have a significant negative impact on the internal microstructure, mechanical properties and overall performance of the MAM product. Therefore, it is urgent to develop an accurate non-destructive testing technology to detect defects in MAM workpieces, evaluate and further optimize the MAM process, so as to ensure the quality and performance of the final product.

[0003] In the prior art, the methods commonly used for metal additive defect detection mainly include X-ray and ultrasonic detection. X-ray detection is based on the difference in X-ray absorption and transmission rate of different parts after X-ray passes through the object, and by analyzing the change in intensity of X-ray after penetrating the object, the potential irregularities and defects in the material are identified and evaluated. However, in materials with large thickness or density, the effect of X-ray detection is not good due to rapid energy attenuation, and radiation exposure poses a potential safety risk to the operator. The principle of ultrasonic detection relies on the scattering, reflection and attenuation phenomena of ultrasonic waves in the material. However, the ultrasonic detection method based on piezoelectric transducers is a contact type detection, which requires coupling agent or other medium to successfully couple the sound waves emitted by the transducer into the measured object, and is not suitable for harsh environments such as high temperature, high pressure or complex-shaped measured parts.

[0004] Laser ultrasonic testing (LUT) is a completely non-contact non-destructive testing technology. By using a pulsed laser to irradiate the surface of an object to cause thermal expansion effects to generate ultrasonic waves inside the material, and using optical methods such as laser interference to receive ultrasonic wave signals. Compared with X-ray and traditional ultrasonic testing, LUT has the advantages of high sensitivity, high spatial resolution, strong real-time detection capability, etc. In laser ultrasonic non-destructive testing based on thermoelastic mechanism, pulsed laser will excite complex coupling acoustic field of multiple modes on the surface and inside the material, which contains longitudinal wave (L), transverse wave (T) and Rayleigh wave (R) and other modes of ultrasonic waves. These wave modes have different propagation directions, speeds and energy distributions. When the defect is located in the subsurface, the Rayleigh wave, longitudinal wave and transverse wave will all be reflected on the defect surface with different time delays, resulting in a mixed signal of multi-mode ultrasonic waves at the receiving point, as shown in Figure 1 When using the amplitude of these A-scan signals for C-scan imaging, the C-scan images at different times are different, which cannot accurately present the morphology of the defect, affecting the accuracy of the detection, thereby hindering the high-resolution imaging of subsurface defects by the traditional C-scan imaging method. C-scan is a surface scan, which represents the projection surface condition of the object being tested, and draws the horizontal projection position of the defect on the projection surface.

[0005] To solve the above problems, some researches have proposed improved methods. For example, patent application CN109269986A discloses a phased array laser ultrasonic testing system, which detects defects by laser-induced phased array. However, this method requires strict analysis of signals captured from different angles and delay intervals. Patent application CN113406010A discloses a laser ultrasonic testing method based on synthetic aperture focusing imaging. However, the synthetic aperture focusing imaging method requires a deep understanding of the propagation rules of ultrasonic wave propagation in materials, the complex interaction between multi-modal ultrasonic waves and defects, and the use of complex signal processing techniques.

[0006] With the rapid development of artificial intelligence technology and its wide application in various fields, its application in laser ultrasonic defect detection has also achieved certain results, significantly improving the efficiency and accuracy of defect detection. For example, patent application CN116773679A discloses a laser ultrasonic imaging method based on neural networks. This method first performs empirical mode decomposition and reconstruction on each A-scan signal individually, then inputs the decomposed signal into a neural network to extract features. The extracted feature values are then converted to pixel values, and the state of each detection point is evaluated by setting a threshold, and finally a corresponding pixel image is generated. Although this method is suitable for low signal-to-noise ratio signals, its detection accuracy depends on the selection of intrinsic mode functions (IMF) in the empirical mode decomposition process and the setting of thresholds in the neural network model. In addition, existing artificial intelligence-based detection methods still require accurate labeling of each signal, which often consumes a lot of manpower and time in practical applications. Moreover, when faced with multi-modal ultrasonic signal aliasing, accurate labeling becomes extremely difficult, which greatly limits the application effect of this method in subsurface defect detection. Another challenge is that current deep learning models usually only extract features from a single signal, without fully considering the relationship between signals, which makes feature extraction completely dependent on the autonomous learning ability of the model. The lack of consideration of the relationship between signals may lead to insufficient performance of the model when dealing with complex signals, especially when dealing with multi-modal and aliasing signals. These problems indicate that although artificial intelligence technology has potential in laser ultrasonic defect detection, further optimization and development are still needed to address the complexity and challenges in practical applications.

[0007] Graph Neural Network (GNN) has been widely used in classification tasks due to its efficiency in handling non-Euclidean structured graph data. Graph data is composed of nodes and edges between nodes, which can be understood as a network structure rather than a picture. Among them, nodes represent entities, and edges represent the relationship between them. GNN uses the relationship between nodes to perform aggregation operations such as averaging and summing, updating node feature vectors, and extracting key features needed for decision-making. GNN updates the feature vectors of nodes through aggregation (such as averaging and summing) and convolution operations on the relationships between nodes, thereby extracting key features needed for decision-making. Through this process, GNN can capture information in the graph structure and reduce the impact of outliers, thereby enhancing the robustness of the model. However, when dealing with newly emerging nodes (i.e. nodes that have not been seen during training), the general aggregation or convolution performance is limited, so further strategies are needed to improve its generalization ability. Adding an attention mechanism to the graph neural network can enhance the generalization ability of unknown nodes by replacing the static normalization convolution attention mechanism.

[0008] In summary, the current defect detection methods mainly have the following problems:

[0009] 1) Traditional laser ultrasonic imaging methods rely on complex signal processing techniques to mitigate the interference caused by signal aliasing, but the final imaging results still have a large gap with the actual situation.

[0010] 2) Existing deep learning methods require preprocessing of A-scan signals to improve the accuracy of subsequent deep learning models in feature extraction, reducing detection efficiency.

[0011] 3) Existing deep learning models only extract defect features from a single signal, failing to fully utilize the spatial structure information between signals, increasing the difficulty of feature extraction. Moreover, each signal needs to be assigned an accurate label to support the supervised learning process, which is very time-consuming and difficult to implement in practical applications.

[0012] 4) When the defect area only accounts for a small part of the entire detection area, the number of detection points in the defect area is significantly less than that in the non-defect area. Non-defect signals dominate in terms of quantity, leading to data set skew, which affects the accuracy and reliability of subsequent analysis. This imbalance hinders the algorithm's ability to effectively extract significant features, causing the algorithm to favor majority class signals while ignoring minority class signals.

[0013] 5) When using non-focused or diffuse sound fields for imaging, the complexity of signals in the defect edge region makes it extremely challenging to accurately extract features. These A-scan signals contain both defect and non-defect features, and relying solely on obviously non-defect or defect signals is insufficient to accurately classify these edge signals. Furthermore, when relying on prior knowledge for signal labeling, these edge signals may exhibit different characteristics at different times, making accurate labeling difficult.

[0014] Therefore, there is an urgent need for a new imaging method to effectively solve the signal aliasing problem of multi-mode ultrasonic waves in laser ultrasonic detection, thereby accurately characterizing subsurface defects to improve detection accuracy. SUMMARY

[0015] The purpose of the present application is to overcome the above-mentioned defects of the prior art and provide a subsurface defect multi-mode laser ultrasonic C-scan image reconstruction method based on graph neural networks. The method comprises the following steps:

[0016] (1) Two-dimensional scanning of the object to be measured to obtain a multi-modal ultrasonic A-scan signal data set;

[0017] (2) Convert the data set to a frequency domain signal and encode the position according to the original acquisition position of the signal;

[0018] (3) Splicing the frequency domain signals with the position encodings and calculating the similarity between the signals;

[0019] (4) Treating each data acquisition point as an independent node, taking the frequency domain signal as the node feature, and selecting a set number of nodes with high similarity from each other, and connecting them with edges in this way to construct a graph data set;

[0020] (5) Inputting the graph data set into a trained graph neural network model to determine whether each A-scan signal is from a defect area, obtaining a classification result, and reconstructing a C-scan image according to the original acquisition position of each A-scan signal and the classification result.

[0021] Compared with the prior art, the advantages of the present application are that the provided sub-surface defect multi-mode laser ultrasonic C-scan image reconstruction method determines whether the laser ultrasonic A-scan signal is from a defect area, and reconstructs a C-scan image in combination with the acquisition position of the signal, and uses a deep learning model to extract defect features in the signal, thereby replacing the traditional complex signal processing process, such as full focus technology, empirical mode decomposition, synthetic aperture focusing technology, etc. In addition, considering that in laser ultrasonic detection, the defect area only accounts for a small part of the overall detection area, the number of detection points in the defect area is significantly less than that in the non-defect area, resulting in a dominant position of non-defect signals in the data set, thereby causing data set imbalance and affecting the accuracy and reliability of subsequent analysis. Moreover, when using non-focused or diffuse acoustic field imaging, the signals of the defect edge area are difficult to accurately extract features due to their complexity, these signals contain both defect and non-defect features, and when relying on prior knowledge to label the signals, the signals located at the defect edge may exhibit different features at different times, making it difficult to accurately assign labels, and relying only on obvious non-defect or defect signals is not enough to accurately judge these edge signals. In view of these problems, the present application proposes a semi-supervised graph neural network model combining graph attention mechanism and adversarial framework, by introducing an unbalanced and negative sampling graph attention adversarial module, the defect area signal misjudgment problem caused by data set imbalance is solved, and the accuracy of defect boundary signal classification is improved. In summary, the present application can accurately represent sub-surface defects and significantly improve the accuracy and reliability of detection.

[0022] Other features of the present application and its advantages will become apparent from the following detailed description of exemplary embodiments thereof, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings incorporated in and forming a part of the specification, illustrate embodiments of the present application and, together with the description, serve to explain the principles of the application.

[0024] Figure 1is a schematic diagram of a multi-mode ultrasonic aliasing principle of the prior art;

[0025] Figure 2 is a flow chart of a subsurface defect multi-mode laser-ultrasound C-scan image reconstruction method according to an embodiment of the present application;

[0026] Figure 3 is a schematic diagram of the overall process of a subsurface defect multi-mode laser-ultrasound C-scan image reconstruction method according to an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of a non-contact laser-ultrasound inspection system according to an embodiment of the present application;

[0028] Figure 5 is a flow chart of an unbalanced and negative sampling map attention GAN according to an embodiment of the present application;

[0029] Figure 6 is a schematic diagram of a comparison of experimental results of different methods according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] Various illustrative embodiments of the present application will now be described in detail with reference to the accompanying figures. It should be noted that the relative arrangements of the components and steps illustrated in these embodiments, numerical expressions, and numerical values set forth herein are not limiting of the scope of the present application.

[0031] The following description of at least one example embodiment is merely illustrative in nature and is in no way limiting of the scope of the application or its applications or uses.

[0032] Techniques, methods, and apparatus known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered as part of the specification, where appropriate.

[0033] In all of the examples shown and discussed herein, any specific values should be interpreted as merely illustrative and not limiting. Other examples of the illustrative embodiments can have different values.

[0034] It should be noted that like references and characters herein relate to like items throughout the figures, and once an item is defined in one figure, it need not be discussed further in subsequent figures.

[0035] The present application is applicable to a non-contact laser ultrasonic detection system. Overall, the provided method comprises: excitation and reception of laser ultrasonic signals; fast Fourier transform of the collected multimodal ultrasonic mixed signals to convert them into frequency domain signals; generation of corresponding position information for each signal collection position, combination of the position information with the frequency domain signals, calculation of the similarity between the signals, and construction of a sample data set; input of the data set into a graph neural network model to determine whether each A-scan signal originates from a defect area; and C-scan image reconstruction based on the original collection position of each A-scan signal and the model result.

[0036] Specifically, as shown in Figure 2 and Figure 3 The provided sub-surface defect multimodal laser ultrasonic C-scan image reconstruction method comprises the following steps:

[0037] Step S1, a non-contact laser ultrasonic detection system is built.

[0038] Figure 4 The non-contact laser ultrasonic detection system is built. The system includes a data acquisition card, a computer, a pulsed laser, a dual-wave mixing interferometer, a control unit, and a two-dimensional moving scanning platform. First, the pulsed laser generates pulsed laser, which is reflected by a mirror, focused into a linear laser by a cylindrical lens, and then irradiated on the surface of the object, thereby exciting multimodal ultrasonic waves. The ultrasonic wave signal is received by the dual-wave mixing interferometer, then digitized and recorded by the data acquisition card, thereby obtaining the multimodal ultrasonic A-scan time domain signal carrying the internal characteristics of the measured sample.

[0039] Step S2, using the non-contact laser ultrasonic detection system to collect multimodal ultrasonic A-scan time domain signals and construct a sample image data set.

[0040] Using the above non-contact laser ultrasonic detection system, a two-dimensional scan is performed on the metal additive sample to be detected to obtain a multimodal mixed data set A(x, y, t). Then, the time domain signal is converted to a frequency domain signal using the fast Fourier transform formula (1), and then normalized according to formula (2) to reduce the influence of high-frequency noise and highlight the key frequency characteristics. These normalized signals are used as inputs to the deep learning model.

[0041]

[0042] Where F[k] represents the kth frequency component of the signal s[n] in the frequency domain, s[n] represents the nth sampling point in the time domain signal; N is the total number of signal sampling points; k is the frequency index, and j is the imaginary unit, i.e. is a complex exponential function.

[0043]

[0044] where S norm denotes the normalized result, s n is each frequency component of the frequency domain signal, s min is the minimum value of the frequency component, s max is the maximum value.

[0045] In order to ensure that the graph data set can accurately capture the spatial proximity relationship between nodes, X and Y coordinate absolute position encoding is introduced for each signal respectively in the construction stage. The specific encoding formula is shown in formula (3) and formula (4). By position encoding according to the collection position of the signal, it is effectively prevented that the ultrasonic signals with different internal structure characteristics due to far away from each other are incorrectly identified as similar, so as to avoid false or irrelevant node connection.

[0046]

[0047] where i represents the collection position (X and Y coordinates) of the signal, j represents the position in the position encoding, for example, when j is even, the generated encoding value is placed in the 2j position of the position encoding, when j is odd, the encoding value is placed in the 2j+1 position. D represents the total dimension size of the position encoding, that is, the length of the encoding.

[0048] Finally, the frequency domain signal is spliced with the position encoding, and the similarity between the signals is calculated using formula (5), and a certain number of signals with the highest similarity are taken as the connected nodes.

[0049]

[0050] where X k and Y k respectively represent the eigenvalues of the k-th dimension of the signals v i and v j , and N represents the dimension of the A-scan signal frequency component and the position encoding. In an embodiment, the dimension of the position encoding is set to half of the dimension of the frequency domain signal to prevent the position encoding from excessively affecting the selection of adjacent nodes. Due to the variability of the relative position of different defects in the scanning area, the position encoding may interfere with the algorithm to make accurate judgments during the training process, so the position encoding is only used in the data set construction stage

[0051] Step S3, training the graph neural network model using the sample graph data set until the set loss target is met.

[0052] The graph neural network model can adopt various types, such as graph convolution network GCN, graph autoencoder GAE, etc.

[0053] In one embodiment, the proposed graph neural network model is an inductive semi-supervised model for multi-modal ultrasound signal classification, or an unbalanced and negative sampling graph attention adversarial network. The process of using the graph neural network model for C-scan image reconstruction is shown in FIG. 3. Figure 5 The model generally mainly includes an unbalanced graph attention adversarial network (or an unbalanced graph attention adversarial module) and a negative sampling graph attention adversarial network (or a negative sampling graph attention adversarial module). The framework of these two modules is composed of a generator and a discriminator, both of which use a graph attention mechanism as a feature extractor. The key role of the graph attention mechanism is to extract features by calculating the attention coefficient of the target node and its adjacent nodes, as shown in equations (6) and (7). At the same time, the generator generates synthetic data to challenge the discriminator, and the discriminator must accurately distinguish between real data and false data. In the unbalanced graph attention adversarial network, the generator first generates false nodes to balance the number of nodes of different categories, forming a uniformly distributed graph structure. The graph attention mechanism of the discriminator extracts features from the uniformly distributed graph structure. In the negative sampling graph attention adversarial network, the generator combines the graph attention adversarial network and the probability negative sampling strategy to generate high-quality negative samples. These negative samples help the graph attention mechanism of the discriminator to effectively extract significant defect features from unlabelled signals. The parameters of the algorithm are optimized by backpropagation using label information, thereby improving classification accuracy. After the model converges, the C-scan image can be reconstructed according to the classification results and the original position of the signal.

[0054]

[0055] where v i represents the source node, v j represents the neighbor node connected to the source node v i , e ij represents the attention coefficient between v i and v j . is the set of first-order neighbors of v i , which includes v i itself, the purpose of which is to add a self-loop in the calculation to avoid losing its main features.

[0056] In one embodiment, by analyzing the similarity of frequency domain signals, a graph data set with a non-Euclidean topological structure is constructed, denoted as G = (V, E, A), where v e V represents the node set, each node corresponds to an ultrasound signal received at different positions. E represents the connection relationship between these signals, and the adjacency matrix A is a two-dimensional array representing the connectivity between nodes, with a value of 1 indicating a connection and a value of 0 indicating no connection, as shown in equation (8):

[0057]

[0058] (1) Imbalanced graph attention adversarial network

[0059] In unbalanced subgraphs In this context, V contains a smaller number of nodes v. less and a large number of nodes v more The relationship between the two is |v less |<<|v more First, through a fully connected layer... Generate fake nodes Where |v im |Indicates the number of fake nodes generated by the generator|v im |=|v less |-|v more |,d o The dimension of the data is represented. Then, the adjacency matrix A between spurious nodes and a minority of nodes is calculated using formula (9). im .

[0060]

[0061] By calculating v im With v less The similarity between nodes is used to determine the link weights between generated and real nodes, and a normalization operation is performed to ensure that the sum of each row is 1. Finally, a GAT (Graph Attention Network) is used as the discriminator. The input of the GAT is a uniformly distributed new graph. Where V′ is composed of v im The new set of nodes v′∈V′ is formed by combining the original node v with the original node v. E′ is... The new set consists of V and the links generated by the generator. A′ is the new adjacency matrix obtained from E′. For example, the generator's loss function is expressed as:

[0062]

[0063] in, and The negative log-likelihood loss is calculated based on the discriminator's evaluation of fake nodes, with the aim of making fake nodes more similar to real nodes. The predicted probability. It calculates the Euclidean distance between the features of the fake nodes and the embedded features of the few real nodes, with the aim of making the generated node features close to the features of the real few nodes. Let θ be the L2 regularizer, θ be the training weight set of the generator, and α be the regularization coefficient. The loss function of the discriminator can be expressed as follows:

[0064]

[0065] wherein, and are Binary Cross-Entropy. is used to distinguish the false node from the real node. is used to distinguish the minority node from the majority node. The purpose of is to increase the embedding distance of real nodes of different categories. is the L2 regularizer, and is the set of training weights. The regularization coefficient is β. In one embodiment, the training objective function of the unbalanced graph attention adversarial network is shown in equation (12):

[0066]

[0067] wherein, represents the adversarial loss function between the discriminator D and the generator G, is the expected symbol, used to calculate the average loss, v ~ P data(v) represents the sampling node v from all data P data(v) , P data(v) includes real nodes and false nodes generated by the generator, v im ~ P im(vim) represents the sampling node v im(vim) from the false nodes P im generated by the generator; is the loss function of the discriminator; is the loss function of the generator.

[0068] (2) Negative sampling graph attention adversarial network

[0069] The present application proposes a probability-based negative sampling graph adversarial module, which generates a certain number of high-quality negative samples (nodes of different categories) for the model by distinguishing the difference between labeled signals and unlabeled signals. This strategy aims to encourage embedding similarity among nodes of the same category, while ensuring that negative samples are far away from the target node in the embedding space, solving the problem of gradient disappearance that may occur in the uniform negative sampling method, and improving the ability of the model to identify complex signals in the edge area of defects. First, the generator calculates the similarity between the labeled signals and the unlabeled signals, and based on this similarity, further calculates the probability of each unlabeled signal as a negative sample, and selects nodes with higher probability and appropriate number as negative samples. For example, the function of the generator is represented as:

[0070]

[0071] wherein, v iv′ j is the candidate node. NSG represents the union of all node embeddings in the generator. The summation of all nodes using the Softmax function will face the problem of extremely low efficiency, so the mini-batch method is used to optimize the calculation of Softmax. For the discriminator, its function is represented as:

[0072]

[0073] where v j represents the negative sampling node with higher probability, H NSD represents the union of all node embeddings in the discriminator. The loss function of the generator is represented as:

[0074]

[0075] where, represents the similarity of the positive sample (i.e. the actual connected node pair), v p represents the neighbor node connected with the node v i by the edge. The loss expectation value of the negative sample pair is calculated, H NSG and H NSD respectively represent the node embedding vectors from the generator and the discriminator, aiming to minimize the similarity of the negative sample node pair while maximizing the similarity of the actually connected node pair. is the binary cross-entropy calculated according to its true label, aiming to learn the intrinsic features of the data from the labeled data, where N is the number of labeled nodes. In short, the negative sampling generator is to sample high-quality negative nodes using the probability distribution method. However, the embedding output of the generator is a discrete index. Therefore, the stochastic gradient descent method (SGD) cannot be directly used for optimization. Reinforcement learning methods based on policy gradient can be used to optimize the generator loss, represented as:

[0076]

[0077] where W NSG represents the parameter matrix in the generator, the gradient of is calculated by the discriminator D NS (·) weighted calculation. In the field of reinforcement learning, D NS (·) in the above formula can be regarded as a reward function. If the discriminator judges that the data generated by the generator is real data, the generator can be provided with a "reward". The generator is trained to generate as realistic data as possible to obtain a higher reward. For each negative sample pair (v i ,v j), the strategy adopted by the generator will punish some low-quality negative sample pairs by reducing their corresponding probabilities, and encourage the discriminator to assign high-quality negative vertices. The loss function of the discriminator is represented as:

[0078]

[0079] where, denotes a batch in the training process, v p denotes the nodes directly connected to v i , v n is a negative node sampled using the generator, is a regularization term, ||·|| F is the Frobenius norm of the node features, and λ is the regularization harmonic factor (for example, set to 1e-5). The discriminator D NS can be optimized using gradient descent techniques. The training objective function of the negative sampling graph attention adversarial network is shown in equation (18):

[0080]

[0081] where, denotes the adversarial objective function between the discriminator D and the generator G, is the expectation symbol used to calculate the average loss; v j ~ D (· | v i ; H NSD ) denotes that node vj is a negative sample of target node vi, and its node embedding vector in the discriminator D is H NSD ; Similarly, v j ~ G (· | v i ; H NSG ) denotes that node v j is a negative sample of node v i generated by the generator G, and its node embedding vector in the generator G is H NSG ; is the loss function of the discriminator; is the loss function of the generator.

[0082] The above basic process is iterated continuously until the model converges, that is, a trained graph neural network model is obtained.

[0083] Step S4, for the actually collected multi-mode ultrasonic aliasing signal, the trained graph neural network model is used to obtain the classification result, and the C-scan image is reconstructed combined with the original position of the signal.

[0084] After the graph neural network model is trained, it can be used for actual C-scan image reconstruction. For example, the specific application steps of the model include: performing two-dimensional scanning on the measured object to obtain a multi-mode ultrasonic A-scan signal dataset; converting the dataset into a frequency domain signal, and performing position coding according to the original acquisition position of the signal; splicing the frequency domain signal and the position coding, and calculating the similarity between the signals; selecting a set number of signals to construct a graph dataset according to the calculated similarity between the signals; inputting the graph dataset into the trained graph neural network model to determine whether each A-scan signal is from a defect area, obtaining a classification result, and reconstructing a C-scan image according to the original acquisition position of each A-scan signal and the classification result.

[0085] To further verify the effect of the present application, an experimental verification was carried out. Overall, the experimental process includes: performing A-scan signal two-dimensional scanning acquisition on metal additive samples, converting the A-scan signal from time domain to frequency domain through fast Fourier transform, and establishing a graph dataset according to the similarity between the frequency domain signals. Using a graph neural network model to determine whether each A-scan signal is from a defect area; according to the classification result and the original position of the signal, the C-scan image of the subsurface defect is accurately reconstructed.

[0086] For example, a plurality of AlSi10Mg and TC4 metal additive samples made by powder bed fusion technology and containing artificial subsurface defects of different shapes and sizes are two-dimensionally scanned. A multi-mode ultrasonic aliasing dataset A(x, y, t) is obtained, and the dataset information is shown in Table 1.

[0087] Table 1 Specific information of metal additive sample of different materials

[0088]

[0089] After the image reconstruction experiment is carried out by using the present application, the final result is compared with X-ray (X-Ray) and scanning acoustic microscope (Scanning Acoustic Microscope, SAM), Figure 6 is a comparison chart of experimental results. The experimental results show that, compared with scanning acoustic microscope imaging, the method based on graph neural network can more clearly outline the defect boundary, and the relative error (Relative Error, RE) can be calculated by formula (19) to quantify the size deviation between the reconstructed image of the algorithm and the standard X-ray imaging, and the results are shown in Table 2.

[0090]

[0091] wherein, L T represents the actual size of the defect. L PDefect size calculated by the model, which is obtained by calculating the sum of the areas of all points marked as defects.

[0092] Table 2 Size deviation between reconstructed image and X-ray image

[0093] Dataset C1 C2 C3 C4 C5 RE 5.08％±0.15％ 7.21％±0.41％ 6.22％±0.14％ 7.17％±0.43％ 9.53％±0.35％

[0094] In summary, compared with the prior art, the present application has the following advantages:

[0095] 1) The present application uses laser ultrasonic non-contact non-destructive testing technology, under the mechanism of thermal elasticity, uses pulsed laser to excite multi-mode ultrasonic waves, and uses a dual-wave mixing laser interferometer to receive ultrasonic wave signals to obtain A-scan signals. Further, through deep learning, the defect features in the A-scan signals are extracted, replacing the traditional complex signal processing technology, to determine whether the A-scan signals come from a defect area, and reconstruct a C-scan image according to the original position of the signals, thereby realizing high-resolution and accurate imaging of subsurface defects.

[0096] 2) Considering the technologies such as full-focus technology and synthetic aperture focusing technology that require complex signal processing methods, the present application uses a non-full-focus non-contact laser ultrasonic system combined with a deep learning model, which has higher application flexibility, can quickly scan, and thus provide results faster without waiting for the full-focus system to complete the complex focusing processing process.

[0097] 3) Usually only defect features are extracted from a single signal, the present application uses a graph neural network, which replaces the traditional preprocessing process through its special feature extraction method, captures the spatial relationship and topological structure information between signals, and fully extracts defect features, thereby improving the accuracy of feature extraction.

[0098] 4) Considering that in practical applications it is impossible to accurately assign labels to each signal, the present application uses a semi-supervised deep learning model, which reduces the dependence on accurate labels and enhances the generalization ability of the model when processing unlabeled data. Moreover, considering that current deep learning models usually only extract features from a single signal, excessively rely on the autonomous learning ability of the model, and require accurate labels, the present application uses a semi-supervised graph neural network, which can capture the relationship between signals, improve the accuracy of feature extraction, and reduce the dependence on accurate labels.

[0099] 5) In order to solve the problem of data imbalance, the present application constructs an imbalance graph adversarial module to balance the training data set by generating false nodes to extract the potential defect features of the signals.

[0100] 6) The application extracts defect features from unlabeled signals by relying on labeled signals through the way of probabilistic negative sampling, effectively dealing with the problem of mixed defect and non-defect features in edge signals, and improving the accuracy of edge area signal classification.

[0101] 7) At the level of data processing and analysis, the application innovatively uses the graph attention mechanism as a feature extractor, effectively solving the problem of insufficient feature extraction capability of traditional methods.

[0102] It should be noted that the above-mentioned embodiments can be appropriately changed or modified by those skilled in the art without departing from the spirit and scope of the present application. For example, the present application has universality and flexibility, and its detection object category significantly exceeds the limitation of subsurface defects, and can be effectively applied to defect identification in deep and complex structures. The present application is not limited to the detection of metal additive manufacturing samples, but shows wide compatibility for various manufacturing methods and material types, such as traditional subtractive manufacturing, casting forming, emerging 3D printing, composite material preparation, etc. The present application can be flexibly adapted to achieve efficient and accurate non-destructive testing. Although the core of the present application focuses on laser ultrasonic non-destructive testing technology, the design concept and implementation path of the present application leave sufficient space for the integration of diversified technical means, such as piezoelectric ultrasonic.

[0103] The present application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0104] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a holographic memory, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0105] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0106] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0107] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0108] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, or digital content stored thereon or therein for a short time or not at all. The computer readable storage medium can also have instructions stored thereon or therein which may

[0109] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0110] The flow diagrams and block diagrams in the attached Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams and block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logic functions (i.e., acts). In some alternative implementations, the functions noted in the block can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0111] Having described various embodiments of the application, it is to be understood that the above description is meant to be illustrative only and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art, without departing from the scope and spirit of the described embodiments. The selection of terms to be used in the description is intended to best explain the principles of the embodiments, the practical application, or technical improvement over the prior art, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the application is defined by the claims appended hereto.

Claims

1. A method for reconstructing a sub-surface defect multi-modal laser-ultrasound C-scan image, comprising the following steps: performing two-dimensional scanning on a measured object to obtain a multi-modal ultrasound A-scan time-domain signal dataset; converting the dataset into a frequency domain signal and performing position encoding according to the signal acquisition position; splicing the frequency domain signal and the position encoding and calculating the similarity between signals; constructing a graph dataset, wherein each data acquisition point is regarded as an independent node, the frequency domain signal is regarded as a node feature, and a set number of nodes with a similarity exceeding a set threshold are selected and connected to each other through edges; inputting the graph dataset into a trained graph neural network model to determine whether each A-scan signal is from a defect area, obtaining a classification result, and reconstructing a C-scan image according to the original acquisition position of each A-scan signal and the classification result; wherein the graph neural network model comprises a first graph attention adversarial network and a second graph attention adversarial network, the first graph attention adversarial network comprises a first generator and a first discriminator, the second graph attention adversarial network comprises a second generator and a second discriminator, the first generator is used to generate false sample nodes, the first discriminator is used to extract features through a first graph neural network and determine the category and authenticity of the nodes, and the second generator is used to generate a negative sample set, and the second discriminator extracts features through a second graph neural network and determines the category of different nodes.

2. The method of claim 1, wherein, In the process of training the graph neural network model, the loss function of the first graph attention adversarial network is set as: wherein: wherein, represents the adversarial loss function between the first discriminator D and the first generator G, is the expected symbol, is the loss function of the first generator, is the loss function of the first discriminator, and is the negative log-likelihood loss calculated according to the evaluation of the discriminator on the fake nodes, is the predicted probability, is the Euclidean distance of the node features after embedding the fake nodes and the few real nodes, is the L2 regularizer, θ is the training weight set of the generator, and the regularization coefficient is α, and are both binary cross-entropy, for discriminating whether the node is a fake node or a real node, for distinguishing nodes of different categories, Ω is the training weight set, and the regularization coefficient is β, less represents the nodes with a smaller number, v more represents the nodes with a larger number, |v im | less |v more | represents the number of fake nodes generated by the generator, v ~ P data(v) represents the nodes v sampled from all data P data(v) , P data(v) includes real nodes and fake nodes generated by the first generator, represents the fake nodes v generated by the first generator, im v' is a candidate node, v i is the current node, v j represents the negative sample nodes of the current node v i .

3. The method of claim 1, wherein, In the process of training the graph neural network model, the loss function of the second graph attention adversarial network is set as: where G NS is a function of the second generator, denoted as: where v i is the current node, v′ j is the candidate node, H NSG denotes the union of all node embeddings in the second generator; D NS is a function of the second discriminator, denoted as: wherein v j represents the negative sample nodes of the current node v i , H NSD represents the union of all node embeddings in the second discriminator, is the loss function of the second generator, is the loss function of the second discriminator, represents the adversarial objective function between the second discriminator D and the second generator G, is the expected symbol; v j ~ G (· |v i ; H NSG ) represents the node v j is the negative sample of the node v i generated by the second generator G, and the node embedding vector of the second generator G is H NSG , represents a batch of data in the training process.

4. The method of claim 3, wherein, Loss function of the second generator is set to: wherein, denotes the similarity degree of positive samples, v p denotes the similarity degree of node v i denotes the similarity degree of node v is the loss expectation value of negative sample pairs, is the binary cross-entropy calculated according to the true label, and N is the number of nodes with labels.

5. The method of claim 3, wherein, Loss function of the second discriminator is set to: where, denotes the loss expectation value of negative sample pairs, H NSG denotes the node embedding vector from the second generator, H NSD denotes the node embedding vector from the second discriminator, is the binary cross-entropy calculated according to the real label, N is the number of nodes with labels; denotes a batch of data in the training process, v p denotes the nodes directly connected to v i , v n is a negative vertex sampled using the generator, is a regularization term, ||·|| F is the Frobenius norm of the node feature, and λ is the harmonic factor of regularization.

6. The method of claim 1, wherein, The acquired signal is position-encoded according to the following formula: wherein i represents the acquisition position of the signal, j represents the position in the position encoding, and d represents the total dimension of the position encoding.

7. The method of claim 1, wherein, The similarity between signals is calculated according to the following formula: where X k and Y k represent the signal v i and v j respectively, and N represents the dimension of the A-scan signal frequency components and the position encoding.

8. A computer readable storage medium having stored thereon a computer program, wherein, The computer program is executed by a processor to realize the steps of the method according to any one of claims 1 to 7.

9. A computer device comprising a memory and a processor, having stored on the memory a computer program capable of running on the processor, characterized in that, The processor executes the computer program to realize the steps of the method according to any one of claims 1 to 7.

Images