Laser additive pore defect monitoring method based on multi-sensor fusion network and related device

Abstract

The invention discloses a laser additive pore defect monitoring method based on a multi-sensor fusion network and a related device. The method comprises the following steps: acquiring multi-source airborne acoustic emission signals which are acquired by a plurality of airborne acoustic emission sensors with different resonant frequencies and are generated along with the melting process of a laser powder bed; inputting the multi-source airborne acoustic emission signal into a pre-trained pore defect monitoring model, and outputting a pore defect monitoring result; wherein the pore defect monitoring model is obtained by training a multi-source sensing adaptive excitation fusion lightweight convolutional neural network by using a multi-source data set; the multi-source sensing adaptive excitation fusion lightweight convolutional neural network comprises a parallel dynamic weight multi-scale residual convolution module, a multi-sensor information adaptive fusion module, an asymmetric lightweight depth feature extraction module and a defect identification module. According to the invention, monitoring of in-situ pore defects in the melting process of the laser powder bed under complex working conditions of multi-sensor information interpretability intelligent fusion driving is realized, and the accuracy and stability of acoustic monitoring in the melting process of the laser powder bed are enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of metal additive manufacturing, and particularly relates to a method and related device for monitoring laser additive pore defects based on a multi-sensor fusion network. Background Art

[0002] Additive manufacturing technology has become one of the most promising metal manufacturing technologies in various metal manufacturing fields, providing a new perspective for the design and processing of parts and materials and revolutionizing the manufacturing industry. Compared with traditional metal manufacturing technologies, metal additive manufacturing technology has many advantages, such as no need for tools or dies, high material utilization rate, short product manufacturing cycle, etc., providing an opportunity for the transformation and upgrading of traditional manufacturing industries and gradually becoming a national strategy of major manufacturing countries in the world. In particular, Laser Powder Bed Fusion (L-PBF) technology is one of the most widely used metal laser additive manufacturing technologies and is widely used in fields such as aerospace and biomedicine. However, this process is extremely prone to unpredictable pore defects. The generation of such pore defects is related to various factors, and the most common ones are unfused pores and keyhole defects caused by the intense and complex interaction between the laser and the powder. On the one hand, insufficient energy input from the laser to the powder during the laser powder bed fusion process will cause the metal powder not to be completely melted during the laser processing, resulting in long and narrow unfused pores inside the part. On the other hand, excessive energy input from the laser to the powder will lead to the appearance of keyholes. The above two most common pore defects will have a great impact on the mechanical properties of the part, easily induce fatigue cracking, and are difficult to eliminate through post-treatment methods such as heat treatment and surface treatment. In order to monitor the formation of pore defects during the L-PBF process to achieve the type or quantitative evaluation of pore defects, on-site real-time monitoring is a promising solution and also the key to ensuring the quality of additive manufacturing parts.

[0003] In-situ acoustic emission monitoring has received extensive attention due to its advantages such as high sensitivity and low cost. In particular, the booming development of artificial intelligence has provided new insights for data-driven acoustic monitoring technology in laser additive manufacturing. This technology can sensitively monitor the generation of internal defects such as pores, which is beneficial to better monitor the instability of the laser powder bed fusion process caused by internal pore defects. Currently, there are still the following problems in the acoustic monitoring method during the laser powder bed fusion process: on the one hand, most of the existing acoustic monitoring studies monitor the L-PBF process through a single acoustic sensor with a determined model, and single-sensor monitoring may have limitations and incompleteness in information acquisition, which will affect the accuracy and stability of the final defect monitoring or quality assessment. On the other hand, there is a lack of effective multi-sensor fusion methods to integrate the data of sensors with different monitoring performances to achieve in-situ precise monitoring of pore defects during the laser additive manufacturing process. Summary of the Invention

[0004] In view of the problems existing in the prior art, the present invention provides a method and related device for monitoring laser additive pore defects based on a multi-sensor fusion network, aiming to develop an effective deep learning model for multi-sensor fusion based on a more comprehensive multi-source acoustic monitoring technology for information monitoring, so as to realize the in-situ pore defect monitoring of the laser powder bed melting process under complex working conditions driven by the interpretable intelligent fusion of multi-sensor information, and enhance the accuracy and stability of the acoustic monitoring of the laser powder bed melting process.

[0005] To solve the above technical problems, the present invention is realized through the following technical solutions:

[0006] According to a first aspect of the present invention, there is provided a method for monitoring laser additive pore defects based on a multi-sensor fusion network, characterized by comprising:

[0007] Obtaining multi-source airborne acoustic emission signals generated during the laser powder bed melting process collected by a plurality of airborne acoustic emission sensors with different resonant frequencies;

[0008] Inputting the multi-source airborne acoustic emission signals into a pre-trained pore defect monitoring model to output a pore defect monitoring result; wherein, the pore defect monitoring model is obtained by training a multi-source sensing adaptive excitation fusion lightweight convolutional neural network using a multi-source data set, the multi-source data set includes airborne acoustic emission signal samples collected by airborne acoustic emission sensors with different resonant frequencies corresponding to different laser powder bed melting pore defect working conditions, and the multi-source sensing adaptive excitation fusion lightweight convolutional neural network includes a parallel dynamic weight multi-scale residual convolutional module, a multi-sensor information adaptive fusion module, an asymmetric lightweight deep feature extraction module, and a defect recognition module.

[0009] In a possible implementation manner of the first aspect, the parallel dynamic weight multi-scale residual convolutional module drives the constructed multi-scale convolutional layer and residual structure through a dynamic weight mechanism of multi-source information parallel calculation to synchronously obtain different receptive field features with weight information input by multiple sensors o i , and adopts a channel feature concatenation method to obtain a multi-resolution depth information set of complementary integration of multi-sensor information o f , and the mathematical expression is as follows:

[0010]

[0011] o f = concate(o 1 ,···,o i )

[0012] In the formula, X iRepresents the input of the i-th airborne acoustic emission sensor, * represents the convolution operation, α k is the dynamic weight of the convolutional layer with convolution kernel k, b k represents the bias term, and concate() is the concatenation operator.

[0013] In a possible implementation of the first aspect, the multi-sensor information adaptive fusion module performs a squeezing operation through a global feature evaluation operator and a local feature evaluation operator for parallel computing to obtain the channel weight information of multiple sensors, and then performs an excitation operation of non-linear mapping through a weight-sharing perceptron MLP and a Sigmoid activation function to obtain the attention weights measuring the importance of multi-resolution information of different sensors. Finally, the multi-sensor fusion output of weighted fusion is obtained through a multiplication operation, and the mathematical expression is as follows:

[0014] Z c = GFEO + LFEO

[0015] S c = σ(MLP(Z c ))

[0016]

[0017] In the formula, X c represents the feature tensor, c is the channel index, GFEO represents the global feature evaluation operator, LFEO represents the local feature evaluation operator, Z c is the sensor channel importance value, S c is the channel weight, σ() represents the Sigmoid activation function, MLP(·) represents the MLP operation, is the multi-sensor fusion output of weighted fusion.

[0018] In a possible implementation of the first aspect, the global feature evaluation operator is a squeezing operation based on average pooling with adaptive weights, and the local feature evaluation operator is a squeezing operation based on max pooling with adaptive weights. The squeezing operation dynamically takes into account the prior knowledge of the physical and reasonable global-local defect feature expression of the defect signal, and guides the reliable fusion of multi-sensor information based on the optimal weight combination of defect feature description. The mathematical expression is as follows:

[0019]

[0020] In the formula, represents the squeezing operation based on average pooling, represents the squeezing operation based on max pooling, w G is the global feature evaluation weight, w L is the local feature evaluation weight, and satisfies w G + wL = 1.

[0021] In a possible implementation of the first aspect, the asymmetric lightweight depth feature extraction module uses an asymmetric combination of 1×k and k×1 convolutions to replace the k×k two-dimensional square convolution kernel, and a pooling layer is introduced after each group of asymmetric convolutions. The mathematical expression is as follows:

[0022]

[0023] In the formula, and are the weights and bias terms of the 1×k convolution kernel of the l-th layer respectively, and are the weights and bias terms of the k×1 convolution kernel of the l-th layer respectively, x represents the input information, F(·) is the activation function, is the output feature value.

[0024] In a possible implementation of the first aspect, the defect recognition module realizes the final defect classification task through three fully connected layers with a dropout layer.

[0025] In a possible implementation of the first aspect, the process of obtaining the multi-source data set includes:

[0026] Using air-borne acoustic emission sensors with different resonant frequencies to synchronously obtain air-borne acoustic emission signals generated during the laser powder bed melting process corresponding to different laser powder bed melting pore defect conditions;

[0027] Performing a sliding window operation on the air-borne acoustic emission signals generated during the laser powder bed melting process corresponding to different laser powder bed melting pore defect conditions to generate a series of air-borne acoustic emission signal samples;

[0028] Based on the air-borne acoustic emission signal samples of different sensors corresponding to different laser powder bed melting pore defect conditions, a multi-source data set is constructed.

[0029] According to the second aspect of the present invention, there is provided a laser additive manufacturing pore defect monitoring device based on a multi-sensor fusion network, including:

[0030] An acquisition module for acquiring multi-source air-borne acoustic emission signals generated during the laser powder bed melting process collected by a plurality of air-borne acoustic emission sensors with different resonant frequencies;

[0031] A monitoring module, configured to input the multi-source airborne acoustic emission signals into a pre-trained pore defect monitoring model, and output a pore defect monitoring result; wherein, the pore defect monitoring model is obtained by training a multi-source sensing adaptive excitation fusion lightweight convolutional neural network using a multi-source dataset, the multi-source dataset includes airborne acoustic emission signal samples collected by airborne acoustic emission sensors with different resonant frequencies corresponding to different laser powder bed fusion pore defect working conditions, and the multi-source sensing adaptive excitation fusion lightweight convolutional neural network includes a parallel dynamic weight multi-scale residual convolution module, a multi-sensor information adaptive fusion module, an asymmetric lightweight deep feature extraction module, and a defect recognition module.

[0032] According to a third aspect of the present invention, there is provided a device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method for monitoring laser additive pore defects based on a multi-sensing fusion network are implemented.

[0033] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the method for monitoring laser additive pore defects based on a multi-sensing fusion network are implemented.

[0034] Compared with the prior art, the present invention has at least the following beneficial effects:

[0035] A laser additive manufacturing pore defect monitoring method based on a multi-sensor fusion network provided by the present invention obtains multi-source airborne acoustic emission signals generated during the laser powder bed melting process collected by airborne acoustic emission sensors with multiple different resonant frequencies, and uses a pore defect monitoring model to process the multi-source airborne acoustic emission signals and output pore defect monitoring results. The pore defect monitoring model is obtained by training a multi-source sensing adaptive excitation fusion lightweight convolutional neural network using a multi-source dataset. The multi-source dataset includes airborne acoustic emission signal samples collected by airborne acoustic emission sensors with different resonant frequencies corresponding to different laser powder bed melting pore defect conditions. The multi-source sensing adaptive excitation fusion lightweight convolutional neural network includes a parallel dynamic weight multi-scale residual convolution module, a multi-sensor information adaptive fusion module, an asymmetric lightweight deep feature extraction module, and a defect recognition module. The present invention constructs an off-axis multi-source acoustic monitoring using a combination of airborne acoustic emission sensors with different resonant frequencies that conform to the physical characteristics of the acoustic signals during the laser powder bed melting process for monitoring the laser powder bed melting process. It can integrate the information monitoring advantages of sensors with different physical characteristics to obtain more comprehensive and rich sound field information during the laser powder bed melting manufacturing process, which is beneficial to improving the accuracy and stability of defect monitoring or quality assessment. The multi-source sensing adaptive excitation fusion lightweight convolutional neural network of the present invention establishes a strong correlation between multi-sensor signals and pore defects. While taking into account the synchronous perception of multi-sensor information and the importance of different scales of channels and spaces, it introduces a physically knowledge-driven multi-sensor information adaptive fusion strategy to achieve effective complementary fusion of multi-sensor information. An asymmetric lightweight deep feature extraction module that takes into account both lightweight and feature extraction capabilities is designed to improve the speed and accuracy of model defect assessment.

[0036] In summary, the present invention develops an effective deep learning model for multi-sensor fusion based on a multi-source acoustic monitoring technology with more comprehensive information monitoring to achieve in-situ pore defect monitoring of the laser powder bed melting process under complex conditions driven by interpretable intelligent fusion of multi-sensor information, and enhances the accuracy and stability of acoustic monitoring during the laser powder bed melting process.

[0037] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. Description of the Drawings

[0038] In order to more clearly illustrate the technical solutions in the specific embodiments of the present invention, the following will briefly introduce the drawings required for the description of the specific embodiments. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0039] Figure 1 Flow chart of a method for monitoring laser additive pore defects based on a multi-sensor fusion network according to an embodiment of the present invention;

[0040] Figure 2 Airborne acoustic emission signals and their spectrograms during the laser powder bed melting process collected by the AM2I sensor in an embodiment of the present invention;

[0041] Figure 3 Airborne acoustic emission signals and their spectrograms during the laser powder bed melting process collected by the AM4I sensor in an embodiment of the present invention;

[0042] Figure 4 Time-frequency spectrogram of the airborne acoustic emission signal during the laser powder bed melting process collected by the AM2I sensor in an embodiment of the present invention;

[0043] Figure 5 Time-frequency spectrogram of the airborne acoustic emission signal during the laser powder bed melting process collected by the AM4I sensor in an embodiment of the present invention;

[0044] Figure 6 Schematic diagram of the multi-source sensor adaptive excitation fusion lightweight convolutional neural network architecture in an embodiment of the present invention, where a is a schematic diagram of the parallel dynamic weight multi-scale residual convolution module, b is a schematic diagram of the multi-sensor information adaptive fusion module, c is a schematic diagram of the asymmetric lightweight depth feature extraction module, d is a schematic diagram of the defect recognition module, and e is a schematic diagram of the asymmetric convolution;

[0045] Figure 7 Confusion matrix of the model test results in an embodiment of the present invention. Detailed implementation manners

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0047] As Figure 1 shown, an embodiment of the present invention provides a method for monitoring laser additive pore defects based on a multi-sensor fusion network, which specifically includes the following steps:

[0048] S1. Obtain multi-source airborne acoustic emission signals generated during the laser powder bed melting process collected by multiple airborne acoustic emission sensors with different resonance frequencies.

[0049] Specifically, install multiple air-borne acoustic emission sensors with different resonant frequencies at appropriate positions in the laser powder bed fusion equipment. It should be noted that these sensors should be evenly distributed or arranged specifically according to the expected monitoring area to ensure that multi-source air-borne acoustic emission signals generated during the L-PBF process can be comprehensively captured.

[0050] Exemplarily, the air-borne acoustic emission sensors with different resonant frequencies adopt the AM2I and AM4I resonant air-coupled acoustic emission sensors of the American PAC company, and their resonant frequencies are 20 kHz and 40 kHz respectively.

[0051] S2. Input the multi-source air-borne acoustic emission signals into a pre-trained pore defect monitoring model, and output the pore defect monitoring results. Among them, the pore defect monitoring model is obtained by training a multi-source sensing adaptive excitation fusion lightweight convolutional neural network using a multi-source data set. The multi-source data set includes air-borne acoustic emission signal samples collected by air-borne acoustic emission sensors with different resonant frequencies corresponding to different laser powder bed fusion pore defect conditions. The multi-source sensing adaptive excitation fusion lightweight convolutional neural network includes a parallel dynamic weight multi-scale residual convolution module, a multi-sensor information adaptive fusion module, an asymmetric lightweight deep feature extraction module, and a defect recognition module.

[0052] Specifically, regarding the multi-source data set, collect air-borne acoustic emission signal samples collected by air-borne acoustic emission sensors with different resonant frequencies under different laser powder bed fusion pore defect conditions, including normal conditions, unfused hole conditions, keyhole conditions, etc.; label the samples to clarify the defect types to which each sample belongs. According to the collected samples, construct a multi-source data set containing data of sensors with multiple resonant frequencies. The data set should comprehensively cover various possible pore defect situations to ensure the generalization ability of the model.

[0053] Exemplarily, the acquisition process of the multi-source data set can include the following process:

[0054] a. Install air-borne acoustic emission sensors with different resonant frequencies near the forming substrate of the laser powder bed fusion equipment, and use the air-borne acoustic emission sensors with different resonant frequencies to synchronously obtain the air-borne acoustic emission signals generated during the laser powder bed fusion process corresponding to different laser powder bed fusion pore defect conditions;

[0055] b. Perform a sliding window operation on the air-borne acoustic emission signals generated during the laser powder bed fusion process corresponding to different laser powder bed fusion pore defect conditions to generate a series of air-borne acoustic emission signal samples;

[0056] c. Based on the airborne acoustic emission signal samples of different sensors corresponding to different working conditions of laser powder bed fusion porosity defects, a multi-source data set is constructed.

[0057] Use the constructed multi-source data set to train the multi-source sensing adaptive excitation fusion lightweight convolutional neural network, and optimize the network parameters through the backpropagation algorithm until the model reaches the predetermined performance index.

[0058] Regarding the multi-source sensing adaptive excitation fusion lightweight convolutional neural network, the network includes a parallel dynamic weight multi-scale residual convolution module, a multi-sensor information adaptive fusion module, an asymmetric lightweight depth feature extraction module, and a defect recognition module.

[0059] Specifically, the parallel dynamic weight multi-scale residual convolution module drives the constructed multi-scale convolution layer and residual structure through a dynamic weight mechanism of multi-source information parallel computing, and synchronously obtains different receptive field features with weight information input by multiple sensors. i , and adopts the method of channel feature concatenation to obtain a multi-resolution depth information set of complementary integration of multi-sensor information. f , and the mathematical expression is as follows:

[0060]

[0061] o f = concate(o 1 ,···,o i )

[0062] In the formula, X i represents the input of the i-th airborne acoustic emission sensor, * represents the convolution operation, α k is the dynamic weight of the convolution layer with a convolution kernel of k, b k represents the bias term, and concate() is the concatenation operator.

[0063] That is to say, for the input signal of each sensor, the parallel dynamic weight multi-scale residual convolution module will process it in parallel. During the processing, the parallel dynamic weight multi-scale residual convolution module will dynamically adjust the weights of different convolution layers according to the characteristics of the input signal; the parallel dynamic weight multi-scale residual convolution module uses a multi-scale convolution layer to capture feature information of different scales, and at the same time uses the residual structure to maintain the integrity of the information and avoid the problem of gradient disappearance or gradient explosion in the deep network; finally, the parallel dynamic weight multi-scale residual convolution module fuses the features from different sensors and different convolution layers together through the method of channel feature concatenation to form a multi-resolution depth information set of complementary integration of multi-sensor information.

[0064] Specifically, the multi-sensor information adaptive fusion module performs a squeezing operation through a global feature evaluation operator and a local feature evaluation operator for parallel computing to obtain the channel weight information of multi-sensors. Then, through a weight-sharing perceptron MLP and a Sigmoid activation function, an excitation operation of non-linear mapping is performed to obtain the attention weights measuring the importance of multi-resolution information of different sensors. Finally, through a multiplication operation, the multi-sensor fusion output of weighted fusion is obtained, and the mathematical expression is as follows:

[0065] Z c = GFEO + LFEO

[0066] S c = σ(MLP(Z c ))

[0067]

[0068] In the formula, X c represents the feature tensor, c is the channel index, GFEO represents the global feature evaluation operator, LFEO represents the local feature evaluation operator, Z c is the sensor channel importance value, S c is the channel weight, σ() represents the Sigmoid activation function, MLP(·) represents the MLP operation, is the multi-sensor fusion output of weighted fusion.

[0069] Specifically, the global feature evaluation operator is a squeezing operation based on average pooling with adaptive weights, and the local feature evaluation operator is a squeezing operation based on max pooling with adaptive weights. The squeezing operation dynamically takes into account the prior knowledge of the physically reasonable global-local defect feature expression of the defect signal, and guides the reliable fusion of multi-sensor information based on the optimal weight combination of defect feature description. The mathematical expression is as follows:

[0070]

[0071]

[0072] In the formula, represents the squeezing operation based on average pooling, represents the squeezing operation based on max pooling, w G is the global feature evaluation weight, w L is the local feature evaluation weight, and satisfies w G + w L = 1.

[0073] Specifically, based on the additivity of the convolutional kernels and the flexibility of the network structure, the asymmetric lightweight depth feature extraction module uses an asymmetric combination of 1×k and k×1 convolutions to replace the traditional k×k two-dimensional square convolutional kernel, and introduces a pooling layer after each group of asymmetric convolutions to significantly reduce the dimensionality of the feature matrix and accelerate the network training process, improving the model efficiency while ensuring the feature extraction ability. The mathematical expression is as follows:

[0074]

[0075] In the formula, and are the weights and bias terms of the 1×k convolutional kernel of the l-th layer respectively, and are the weights and bias terms of the k×1 convolutional kernel of the l-th layer respectively, x represents the input information, F(·) is the activation function, is the output feature value.

[0076] Specifically, the defect recognition module realizes the final defect classification task through three fully connected layers with a dropout layer.

[0077] In a specific embodiment, two air-borne acoustic emission sensors with different resonant frequencies are used to synchronously collect the air-borne acoustic emission signals generated during the laser powder bed fusion process. In the experiment, the sampling frequency of the sensors is set to 100 kHz. The experimental material uses atomized 316L austenitic stainless steel powder with a powder particle diameter ranging from 15 to 53 μm. In this embodiment, 8 groups of 10×10×10 mm cube samples with different laser powder bed fusion processes are set. They are achieved by adjusting the laser power P, the scanning speed V, and the scanning strategy, and have different porosity defects quantified by density. As shown in Table 1, the laser powder bed fusion processing parameters and defect characterization results are presented.

[0078] Table 1 Laser powder bed fusion processing parameters and defect characterization results in this embodiment

[0079]

[0080] In this embodiment, AM2I and AM4I resonant air-coupled acoustic emission sensors produced by the American PAC company with resonant frequencies of 20 kHz and 40 kHz respectively are used to synchronously and real-time collect the air-borne acoustic emission signals generated during the laser powder bed fusion additive manufacturing process. The air-borne acoustic emission signals and their spectrograms of the laser powder bed fusion process collected by the AM2I and AM4I air-borne acoustic emission sensors are shown in Figure 2 and Figure 3As shown, it can be found that the combination of air - borne acoustic emission sensors with different resonant frequencies adopted in the embodiments of the present invention is helpful for effectively perceiving defect signals whose spectra are mainly distributed around 20 kHz and 40 kHz. Set the sampling sliding window length to 5000, and there is no overlap between adjacent windows. To ensure the fairness of data comparison, the number of air - borne acoustic emission sample signals corresponding to 8 different pore defect states is 1250 each.

[0081] Convert the air - borne acoustic emission sample signals from AM2I and AM4I air - borne acoustic emission sensors into two - dimensional time - frequency spectrograms with a size of 224*224 and 3 channels through short - time Fourier transform, so as to construct a multi - source data set containing air - borne acoustic emission signal samples of different sensors, and divide the data set into a training data set and a test data set according to 7:3. As Figure 4 and Figure 5 are the time - frequency spectrograms of the air - borne acoustic emission signals collected by AM2I and AM4I sensors during the laser powder bed fusion process respectively.

[0082] Build a multi - source sensing adaptive excitation fusion lightweight convolutional neural network including a parallel dynamic weight multi - scale residual convolution module, a multi - sensor information adaptive fusion module, an asymmetric lightweight depth feature extraction module, and a defect recognition module, as Figure 6 shown.

[0083] The parallel dynamic weight multi - scale residual convolution module drives convolutional layers and residual structures with multi - scale convolution kernels of 1×1, 3×3, 5×5, and 7×7 through a dynamic weight mechanism for parallel computing of multi - source information, so as to synchronously obtain different receptive field features with important information of multi - sensor inputs, and uses the Concatenate channel feature concatenation method to obtain a multi - resolution depth information set with complementary integration of multi - sensor information. Among them, the number of channels of the multi - scale convolutional layers is set to 8, and the number of channels of the multi - resolution depth information set with complementary integration of multi - sensor information obtained by the channel feature concatenation method is 64, as Figure 6 (a) shown. This module allows the model to adaptively adjust the convolution kernel weights according to the importance of different scale features of the input information of a certain sensor through setting a dynamic weight mechanism, so as to obtain multi - resolution receptive field features with important information suitable for different targets, and configures a residual structure to skip different spatial feature layers to enhance information flow, alleviate the network degradation effect of the model, and fully retain the original information of multi - source sensors. Further, this module can efficiently perform dynamic weight multi - scale residual feature extraction through parallel computing, and can effectively perceive multi - scale information of different sensors in space - time synchronization without significantly increasing the computational complexity.

[0084] The multi - sensor information adaptive fusion module has an adaptive global feature evaluation weight w through parallel computingG Average pooling-based squeezing operation Global Feature Evaluation Operator GFEO with adaptive local feature evaluation weight w L Max pooling-based squeezing operation Local Feature Evaluation Operator LFEO performs a squeezing operation to obtain multi-sensor different-channel importance information Z that takes into account the prior knowledge of the physically reasonable global-local defect feature expression of the defect signal c , and then performs an excitation operation of non-linear mapping through a weight-sharing perceptron MLP and a Sigmoid activation function to obtain the attention weight value S that measures the importance of multi-resolution information of different sensors c , and finally obtains the weighted fusion multi-sensor fusion output through a multiplication operation As Figure 6 (b) shown. This module guides the fusion of multi-resolution information of different sensors through the prior knowledge of the physically reasonable global-local defect feature expression of the L-PBF acoustic signal. On the one hand, the fused information comes from the multi-scale global or local information of different sensors, and this rich multi-resolution information representation helps to retain and accurately extract the detailed information of the global-local defect features of the target signal spectrum. On the other hand, the multi-level importance evaluation of sensors based on the prior of the global-local defect information of the acoustic signal provides an interpretable information source and fusion basis for the adaptive fusion of multi-sensor information. The complementary fusion of multi-sensor information driven by this physical mechanism will make the transmission and recognition of defect feature information more efficient and accurate

[0085] The asymmetric lightweight depth feature extraction module is based on the additivity of convolutional kernels and the flexibility of network structures. It uses asymmetric 1×11 and 11×1, 1×5 and 5×1, 1×3 and 3×1 convolutional combinations with channel numbers of 96, 256, and 256 respectively to replace the traditional 11×11, 5×5, 3×3 two-dimensional square convolutional kernels, and introduces a max pooling layer after each group of asymmetric convolutions to significantly reduce the dimension of the feature matrix and accelerate the network training process, improving the model efficiency while ensuring the feature extraction ability, as Figure 6 (c) and (e) shown

[0086] The defect recognition module realizes the final defect classification task through a 3-layer fully connected layer equipped with a dropout layer, where the random dropout parameter of the dropout layer is set to 0.5, and the outputs of the fully connected layers are 4096, 4096, and 8 in sequence, as Figure 6 (d) shown

[0087] The other parameters of the model are set as follows: the batch size is 16, and the learning rate is 10 -4, the Adam optimizer is used, and the number of training epochs is 100. In addition, the training and testing of the network model are both completed on a device with an Intel(R) Core(TM) i5-13600KF CPU and an NVIDIA RTX 4070Ti GPU, and the PyTorch deep learning framework and the Python 3.8 language are used.

[0088] A multi-source dataset containing air-borne acoustic emission signal samples collected by different sensors is input into the constructed multi-source sensing adaptive excitation fusion lightweight convolutional neural network for training, and the trained model is used to online identify the air-borne acoustic emission signals of pore defects in the laser powder bed fusion process.

[0089] As shown in Table 2, the recognition accuracy results of the multi-sensor monitoring model and the single-sensor monitoring model in this embodiment are presented. Among them, the multi-source monitoring model is the constructed multi-source sensing adaptive excitation fusion lightweight convolutional neural network, and the single-source monitoring model adopts a multi-scale convolutional model, and its multi-scale architecture settings are similar to those of the multi-source monitoring model. As Figure 7 shown, this is the confusion matrix of the multi-source sensing adaptive excitation fusion lightweight convolutional neural network model test in this embodiment. It can be seen that the final test recognition accuracy of the model reaches 98.70%, which can well distinguish the laser additive manufacturing processes of pore defect parts with 8 different densities, and has a better pore defect monitoring effect than the acoustic monitoring method of a single sensor.

[0090] Table 2 Recognition accuracy results of the multi-sensor monitoring model and the single-sensor monitoring model

[0091]

[0092] The multi-source sensing adaptive excitation fusion lightweight convolutional neural network architecture proposed by the present invention, through actual verification in the embodiment, can assign the multi-scale feature channel weights of different sensors by the global-local key information features of the physically mechanism-driven signals, so as to achieve the credible feature-level adaptive weighted fusion of multi-source acoustic sensor information, and establish an effective association between the multi-source acoustic defect signals and the pore defects in the laser additive manufacturing, and further realize the real-time and accurate monitoring of the pore defects in the laser additive manufacturing process under complex working conditions of different scanning strategies.

[0093] In another embodiment of the present invention, a laser additive pore defect monitoring device based on a multi-sensor fusion network is provided, including:

[0094] An acquisition module, configured to acquire multi-source air-borne acoustic emission signals generated during the laser powder bed fusion process collected by multiple air-borne acoustic emission sensors with different resonant frequencies.

[0095] A monitoring module, configured to input the multi-source airborne acoustic emission signals into a pre-trained pore defect monitoring model, and output a pore defect monitoring result; wherein, the pore defect monitoring model is obtained by training a multi-source sensing adaptive excitation fusion lightweight convolutional neural network using a multi-source data set, the multi-source data set includes airborne acoustic emission signal samples collected by airborne acoustic emission sensors with different resonant frequencies corresponding to different laser powder bed melting pore defect conditions, and the multi-source sensing adaptive excitation fusion lightweight convolutional neural network includes a parallel dynamic weight multi-scale residual convolutional module, a multi-sensor information adaptive fusion module, an asymmetric lightweight deep feature extraction module, and a defect recognition module.

[0096] All relevant contents of each step involved in the embodiment of the foregoing method for monitoring laser additive pore defects based on a multi-sensing fusion network can be cited in the function description of the corresponding functional modules of a device for monitoring laser additive pore defects based on a multi-sensing fusion network in the embodiments of the present invention, and will not be elaborated here. The division of modules in the embodiments of the present invention is illustrative, merely a logical function division, and there may be other division methods in actual implementation. Additionally, in each embodiment of the present invention, the functional modules may be integrated in one processor, or may exist independently physically, or two or more modules may be integrated in one module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0097] In another embodiment of the present invention, a computer device is provided. The computer device includes a processor and a memory. The memory is used to store a computer program, and the computer program includes program instructions. The processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function. The processor described in the embodiments of the present invention may be used for the operation of a method for monitoring laser additive pore defects based on a multi-sensing fusion network.

[0098] In another embodiment of the present invention, the present invention further provides a storage medium, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in a computer device and is used to store programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and, of course, the extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space, and the operating system of the terminal is stored in this storage space. Moreover, one or more instructions suitable for being loaded and executed by the processor are also stored in this storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. One or more instructions stored in the computer-readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the method for monitoring laser additive pore defects based on a multi-sensor fusion network in the above embodiment.

[0099] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0100] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0101] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions in the process Figure 1one or more processes and / or blocks Figure 1 functions specified in one or more blocks.

[0102] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one Figure 1 one or more processes and / or blocks Figure 1 or more processes and / or in one or more blocks.

[0103] In the present invention, the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0104] Finally, it should be noted that the above-described embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, and are not intended to limit them. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any person skilled in the art within the technical scope disclosed by the present invention can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for monitoring pore defects in laser additive manufacturing based on a multi-sensor fusion network, characterized in that: include: Acquire multi-source airborne acoustic emission signals generated during the laser powder bed melting process collected by multiple airborne acoustic emission sensors with different resonant frequencies; The multi-source air-propagated acoustic emission signal is input into a pre-trained pore defect monitoring model, and the pore defect monitoring result is output; wherein, the pore defect monitoring model is obtained by training a multi-source sensing adaptive excitation fusion lightweight convolutional neural network using a multi-source data set, and the multi-source data set includes air-propagated acoustic emission signal samples collected by air-propagated acoustic emission sensors of different resonant frequencies corresponding to different laser powder bed melting pore defect conditions, and the multi-source sensing adaptive excitation fusion lightweight convolutional neural network includes a parallel dynamic weight multi-scale residual convolution module, a multi-sensor information adaptive fusion module, an asymmetric lightweight deep feature extraction module and a defect recognition module.

2. According to claim 1, a method for monitoring pore defects in laser additive manufacturing based on a multi-sensor fusion network is characterized in that: The parallel dynamic weight multi-scale residual convolution module adopts a dynamic weight mechanism to drive the multi-scale convolution layer and residual structure constructed by parallel calculation of multi-source information, and synchronously obtains different receptive field features with weight information of multi-sensor input. i , and the channel feature cascade method is used to obtain a multi-resolution depth information set of multi-sensor information complementary integration. f , the mathematical expression is as follows: o f =concate(o1,···,o i ) Where, X i represents the input of the i-th airborne acoustic emission sensor, * represents the convolution operation, α k is the dynamic weight of the convolution layer with kernel k, b k Represents the bias term, and concate() is the cascade operator.

3. The method for monitoring pore defects in laser additive manufacturing based on a multi-sensor fusion network according to claim 1, characterized in that: The multi-sensor information adaptive fusion module obtains the channel weight information of multiple sensors by squeezing the global feature evaluation operator and the local feature evaluation operator calculated in parallel, and then obtains the attention weights that measure the importance of multi-resolution information of different sensors through the weight sharing perceptron MLP and the Sigmoid activation function. Finally, the weighted fusion multi-sensor fusion output is obtained by multiplication operation. The mathematical expression is as follows: WITH c =GFEO+LFEO S c =σ(MLP(Z c )) Where, X c represents the feature tensor, c is the channel index, GFEO represents the global feature evaluation operator, LFEO represents the local feature evaluation operator, and Z c is the sensor channel importance value, S c is the channel weight, σ() represents the Sigmoid activation function, MLP(·) represents the MLP operation, It is the weighted fusion multi-sensor fusion output.

4. The method for monitoring pore defects in laser additive manufacturing based on a multi-sensor fusion network according to claim 3, characterized in that: The global feature evaluation operator is a squeeze operation based on average pooling with adaptive weights, and the local feature evaluation operator is a squeeze operation based on maximum pooling with adaptive weights. The squeeze operation dynamically takes into account the prior knowledge of the physically reasonable global-local defect feature expression of the defect signal, and guides the credible fusion of multi-sensor information based on the optimal weight combination of the defect feature description. The mathematical expression is as follows: In the formula, represents the squeeze operation based on average pooling, represents the squeeze operation based on maximum pooling, w G is the global feature evaluation weight, w L is the local feature evaluation weight, and satisfies w G +w L =1.

5. The method for monitoring pore defects in laser additive manufacturing based on a multi-sensor fusion network according to claim 1, characterized in that: The asymmetric lightweight deep feature extraction module uses an asymmetric 1×k and k×1 convolution combination to replace the k×k two-dimensional square convolution kernel, and introduces a pooling layer after each set of asymmetric convolution. The mathematical expression is as follows: In the formula, and are the weights and bias terms of the 1×k convolution kernel of the lth layer, and are the weights and bias terms of the k×1 convolution kernel of the lth layer, x represents the input information, F(·) is the activation function, is the output eigenvalue.

6. The method for monitoring pore defects in laser additive manufacturing based on a multi-sensor fusion network according to claim 1, characterized in that: The defect recognition module achieves the final defect classification task through three fully connected layers with a matching discard layer.

7. The method for monitoring pore defects in laser additive manufacturing based on a multi-sensor fusion network according to claim 1, characterized in that: The process of acquiring the multi-source data set includes: Airborne acoustic emission sensors with different resonant frequencies are used to synchronously obtain airborne acoustic emission signals generated during the laser powder bed melting process corresponding to different laser powder bed melting porosity defect conditions; A sliding window operation is used to generate a series of airborne acoustic emission signal samples for the airborne acoustic emission signal generated during the laser powder bed melting process corresponding to different laser powder bed melting porosity defect conditions. A multi-source dataset is constructed based on the airborne acoustic emission signal samples of different sensors corresponding to different laser powder bed melting porosity defect conditions.

8. A laser additive pore defect monitoring device based on a multi-sensor fusion network, characterized in that: include: An acquisition module, used to acquire multi-source air-borne acoustic emission signals generated during the laser powder bed melting process collected by multiple air-borne acoustic emission sensors with different resonant frequencies; A monitoring module is used to input the multi-source air-borne acoustic emission signal into a pre-trained pore defect monitoring model and output a pore defect monitoring result; wherein the pore defect monitoring model is obtained by training a multi-source sensing adaptive excitation fusion lightweight convolutional neural network using a multi-source data set, and the multi-source data set includes air-borne acoustic emission signal samples collected by air-borne acoustic emission sensors of different resonant frequencies corresponding to different laser powder bed melting pore defect conditions, and the multi-source sensing adaptive excitation fusion lightweight convolutional neural network includes a parallel dynamic weight multi-scale residual convolution module, a multi-sensor information adaptive fusion module, an asymmetric lightweight deep feature extraction module and a defect recognition module.

9. A device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of a laser additive pore defect monitoring method based on a multi-sensor fusion network as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of a laser additive pore defect monitoring method based on a multi-sensor fusion network as described in any one of claims 1 to 7 are implemented.

Images