A laser powder bed melting defect monitoring method and system based on data fusion

Abstract

The application discloses a kind of laser powder bed melting defect monitoring method and system based on data fusion, which comprises the following steps: obtaining laser powder fusion image and laser powder fusion acoustic signal and carrying out data preprocessing, to obtain the laser powder fusion image after preprocessing and the laser powder fusion acoustic signal after preprocessing;Based on cross attention fusion mechanism, a double-branch cross-modal fusion model is constructed;Based on the double-branch cross-modal fusion model, the laser powder bed melting defect monitoring of the laser powder fusion image after preprocessing and the laser powder fusion acoustic signal after preprocessing is carried out, to obtain the laser powder bed melting defect monitoring result.By using the application, the accuracy of intelligent ultra-high monitoring of laser powder bed melting process can be improved through cross-modal fusion between image and acoustic signal.The application can be widely applied to the field of artificial intelligence interaction as a kind of laser powder bed melting defect monitoring method and system based on data fusion.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence interaction, and particularly relates to a laser powder bed fusion defect monitoring method and system based on data fusion. BACKGROUND

[0002] Laser powder bed fusion is a kind of additive manufacturing technology, which precisely lays metal powder on the substrate in a tilted or rolling manner, and then selectively melts the powder layer by layer using a pulsed laser until the product meets the specified requirements. During the manufacturing process, due to the accumulation of thermal stress generated by periodic laser heating, the surface of the part is prone to super-high defects. Super-high defects refer to the part that exceeds the powder layer due to warping or curling upward, resulting in the inability of the powder to completely cover these overhanging parts during powder laying. However, these exposed metal parts may collide with the powder laying strip, and in severe cases, even with the powder laying car. At the same time, these exposed parts will be re-melted and solidified in the next laser scanning cycle, thereby causing more severe structural deformation. As the printing proceeds layer by layer, the accumulation of super-high may exacerbate quality problems such as deformation and mechanical property degradation, ultimately leading to production failures, machine downtime, material waste, and increased production risks.

[0003] With the rapid development of intelligent manufacturing, deep learning algorithms have been used for laser powder bed fusion process monitoring, using optical, acoustic, and other sensors and signals to detect part defects. Industrial cameras are commonly used in super-high monitoring, which can capture the entire building platform, focusing on the powder bed and partial surface anomalies, including super-high defects. Researchers have also explored other sensor technologies, such as accelerometers and low-coherence interferometry. Currently, research on super-high monitoring is still limited to single-signal monitoring, mainly relying on camera systems for visual observation. However, the ability of cameras to identify super-high height conditions from top to bottom is limited, which limits the research on precise monitoring and enhanced control of super-high severity, so the existing method cannot meet the detection accuracy requirements under super-high height conditions. SUMMARY

[0004] In order to solve the above technical problems, the purpose of the present application is to provide a laser powder bed fusion defect monitoring method and system based on data fusion, which can improve the precision of intelligent super-high monitoring of the laser powder bed fusion process through cross-modal fusion between images and acoustic signals.

[0005] The first technical solution adopted by the present application is: a laser powder bed fusion defect monitoring method based on data fusion, comprising the following steps:

[0006] acquire laser powder fusion images and laser powder fusion acoustic signals and perform data preprocessing to obtain preprocessed laser powder fusion images and preprocessed laser powder fusion acoustic signals;

[0007] A dual-branch cross-modal fusion model is constructed based on a cross-attention fusion mechanism.

[0008] The laser powder bed fusion defect monitoring result is obtained by monitoring the laser powder bed fusion defects based on the dual-branch cross-modal fusion model for the preprocessed laser powder fusion image and the preprocessed laser powder fusion acoustic signal.

[0009] Further, the step of obtaining the laser powder fusion image and the laser powder fusion acoustic signal and performing data preprocessing to obtain the preprocessed laser powder fusion image and the preprocessed laser powder fusion acoustic signal specifically includes:

[0010] The laser powder fusion image includes a first type of laser powder fusion image and a second type of laser powder fusion image, and the laser powder fusion acoustic signal includes a first type of laser powder fusion acoustic signal and a second type of laser powder fusion acoustic signal.

[0011] The laser powder fusion image and the laser powder fusion acoustic signal are respectively subjected to signal windowing and alignment processing to obtain windowed and aligned laser powder fusion images and windowed and aligned laser powder fusion acoustic signals.

[0012] The windowed and aligned laser powder fusion acoustic signal is subjected to noise reduction and two-dimensional time-frequency graph conversion processing to obtain a two-dimensional time-frequency graph of the laser powder fusion acoustic signal.

[0013] The windowed and aligned laser powder fusion image and the two-dimensional time-frequency graph of the laser powder fusion acoustic signal are subjected to grayscale conversion and size standardization processing to obtain the preprocessed laser powder fusion image and the preprocessed laser powder fusion acoustic signal.

[0014] Further, the dual-branch cross-modal fusion model includes a first branch feature extraction module, a second branch feature extraction module, a residual-based linear fusion module, and a fully connected layer. The output end of the first branch feature extraction module is connected to the first input end of the residual-based linear fusion module, the output end of the second branch feature extraction module is connected to the second input end of the residual-based linear fusion module, and the output end of the residual-based linear fusion module is connected to the input end of the fully connected layer. Wherein:

[0015] The first branch feature extraction module includes a first feature extractor, a second feature extractor, and a first cross-attention network module.

[0016] The second branch feature extraction module includes a third feature extractor, a fourth feature extractor, and a second cross-attention network module.

[0017] Further, the laser powder bed fusion defect monitoring result is obtained by inputting the preprocessed laser powder fusion image and the preprocessed laser powder fusion acoustic signal into the dual-branch cross-modal fusion model.

[0018] The preprocessed laser powder fusion image and the preprocessed laser powder fusion acoustic signal are input into the dual-branch cross-modal fusion model respectively.

[0019] The laser powder fusion image features are obtained by performing feature extraction processing on the preprocessed laser powder fusion image based on the first branch feature extraction module of the dual-branch cross-modal fusion model.

[0020] The laser powder fusion acoustic signal features are obtained by performing feature extraction processing on the preprocessed laser powder fusion acoustic signal based on the second branch feature extraction module of the dual-branch cross-modal fusion model.

[0021] The laser powder fusion features are obtained by performing feature linear fusion processing on the laser powder fusion image features and the laser powder fusion acoustic signal features based on the residual-based linear fusion module of the dual-branch cross-modal fusion model.

[0022] The laser powder bed fusion defect monitoring result is obtained by performing mapping and classification processing on the laser powder fusion features based on the full connection layer of the dual-branch cross-modal fusion model.

[0023] Further, the laser powder fusion image features are obtained by performing feature extraction processing on the preprocessed laser powder fusion image based on the first branch feature extraction module of the dual-branch cross-modal fusion model.

[0024] The preprocessed laser powder fusion image is input into the first branch feature extraction module of the dual-branch cross-modal fusion model.

[0025] The first type of laser powder fusion image features are obtained by performing feature extraction processing on the first type of laser powder fusion image based on the first feature extractor of the first branch feature extraction module.

[0026] The second type of laser powder fusion image features are obtained by performing feature extraction processing on the second type of laser powder fusion image based on the second feature extractor of the first branch feature extraction module.

[0027] The laser powder fusion image features are obtained by performing cross-attention fusion processing on the first type of laser powder fusion image features and the second type of laser powder fusion image features based on the first cross-attention network module of the first branch feature extraction module.

[0028] Further, the second branch feature extraction module based on the dual-branch cross-modal fusion model performs feature extraction processing on the preprocessed laser powder fusion acoustic signal to obtain laser powder fusion acoustic signal features, which specifically includes:

[0029] The preprocessed laser powder fusion acoustic signal is input into the second branch feature extraction module of the dual-branch cross-modal fusion model;

[0030] The third feature extractor based on the second branch feature extraction module performs feature extraction processing on the first type of laser powder fusion acoustic signal to obtain first type of laser powder fusion acoustic signal features;

[0031] The fourth feature extractor based on the second branch feature extraction module performs feature extraction processing on the first type of laser powder fusion acoustic signal to obtain second type of laser powder fusion acoustic signal features;

[0032] The second cross-attention network module based on the second branch feature extraction module performs cross-attention fusion processing on the first type of laser powder fusion acoustic signal features and the second type of laser powder fusion acoustic signal features to obtain laser powder fusion acoustic signal features.

[0033] Further, the loss function of the dual-branch cross-modal fusion model is:

[0034]

[0035] In the above formula, Loss represents the loss function of the dual-branch cross-modal fusion model, y i represents the probability distribution of the basic true value label, y l is the probability of the i-th class, is the model prediction probability of the i-th class, and alpha represents the learning rate, and N represents the number of prediction samples.

[0036] The second technical solution adopted by the present application is: a laser powder bed fusion defect monitoring system based on data fusion, comprising:

[0037] The first module is used for acquiring laser powder fusion images and laser powder fusion acoustic signals and performing data preprocessing to obtain preprocessed laser powder fusion images and preprocessed laser powder fusion acoustic signals;

[0038] The second module is used for constructing a dual-branch cross-modal fusion model based on a cross-attention fusion mechanism;

[0039] The third module is used for laser powder bed fusion defect monitoring based on the dual-branch cross-modal fusion model on the preprocessed laser powder fusion images and the preprocessed laser powder fusion acoustic signals to obtain laser powder bed fusion defect monitoring results.

[0040] The method and system have the advantages that the laser powder fusion image and the laser powder fusion acoustic signal are acquired and data preprocessing is performed, the image and acoustic characteristics closely related to the ultra-high defect of the part are accurately extracted and combined, a double-branch cross-modal fusion model is constructed based on a cross-attention fusion mechanism, and laser powder bed fusion defect monitoring is performed on the preprocessed laser powder fusion image and the preprocessed laser powder fusion acoustic signal based on the double-branch cross-modal fusion model, so that the classification limitation of the traditional monitoring method is solved, the cross-modal fusion between the image and the acoustic signal is used to improve the accuracy of intelligent ultra-high monitoring of the laser powder bed fusion process. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is a step flow chart of a laser powder bed fusion defect monitoring method based on data fusion provided by the present application;

[0042] Figure 2 is a structural block diagram of a laser powder bed fusion defect monitoring system based on data fusion provided by the present application;

[0043] Figure 3 is an implementation flow diagram of laser powder bed fusion defect monitoring provided by an embodiment of the present application;

[0044] Figure 4 is an equipment schematic diagram of laser powder bed fusion provided by an embodiment of the present application;

[0045] Figure 5 is a schematic diagram of data preprocessing provided by an embodiment of the present application;

[0046] Figure 6 is a prediction schematic diagram based on a double-branch cross-modal fusion model provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. For the step numbers in the following embodiments, only the setting for facilitating the description is provided, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0048] Referring to Figure 1 , the present application provides a laser powder bed fusion defect monitoring method based on data fusion, which comprises the following steps:

[0049] S100, acquiring a laser powder fusion image and a laser powder fusion acoustic signal and performing data preprocessing to obtain a preprocessed laser powder fusion image and a preprocessed laser powder fusion acoustic signal;

[0050] Specifically, a laser powder fusion image and a laser powder fusion acoustic signal are obtained, wherein the laser powder fusion image includes a first type of laser powder fusion image and a second type of laser powder fusion image, and the laser powder fusion acoustic signal includes a first type of laser powder fusion acoustic signal and a second type of laser powder fusion acoustic signal; the laser powder fusion image and the laser powder fusion acoustic signal are respectively subjected to signal windowing and alignment processing to obtain a windowed laser powder fusion image and a windowed laser powder fusion acoustic signal; the windowed laser powder fusion acoustic signal is subjected to noise reduction and two-dimensional time-frequency graph conversion processing to obtain a two-dimensional time-frequency graph of the laser powder fusion acoustic signal; the windowed laser powder fusion image and the two-dimensional time-frequency graph of the laser powder fusion acoustic signal are subjected to grayscale conversion and size standardization processing to obtain a preprocessed laser powder fusion image and a preprocessed laser powder fusion acoustic signal.

[0051] In this embodiment, if Figure 4 As shown, an industrial camera and microphone sensor were first used to capture images of the powder bed after spreading (Type I laser powder fusion images), images of the powder after melting (Type II laser powder fusion images), acoustic signals of the powder spreading process (Type I laser powder fusion acoustic signals), and acoustic signals of the melting process (Type II laser powder fusion acoustic signals) from several sets of laser powder bed fusion experiments under different process parameters. The data was also recorded for the part's superelevation state. The industrial camera and microphone sensor used the same trigger signal to initiate data acquisition. The industrial camera was mounted externally above the build chamber and monitored the powder bed through an observation window. The microphone sensor was mounted on the left wall of the enclosed build chamber, with its probe positioned approximately 25-30 cm above the build platform.

[0052] Further, such as Figure 5 As shown, the images after powder spreading and melting, as well as the acoustic signals during powder spreading and melting, are windowed and aligned. The coordinates of each part on the powder bed are recorded, and part images are located and segmented layer by layer in real time. The acoustic signals are then time-window aligned and segmented. Noise reduction and two-dimensional time-frequency conversion are then performed on the acoustic signals during powder spreading and melting. A 2kHz high-pass filter is used to remove inherent low-frequency noise from the equipment. Short-time Fourier transforms are used to convert the one-dimensional signals into two-dimensional time-frequency maps suitable for the subsequent convolutional neural network. Finally, the two-dimensional time-frequency maps of the images after powder spreading and melting, as well as the acoustic signals during powder spreading and melting, are converted to grayscale and resized, resulting in four feature maps with dimensions of 1*224*224. Grayscale images inherently reduce dimensionality, improve computational efficiency, enhance contrast, and facilitate feature extraction. Converting to a uniform size of 224*224 ensures that feature extraction uses the same parameters, facilitating subsequent feature fusion operations.

[0053] It should be noted that, considering the practicability and cost-effectiveness of the sensor device, the system is equipped with an industrial camera and a microphone sensor. The industrial camera can capture the entire building platform, and as the height of the part increases, it is difficult for the powder to cover the entire part surface when laying the powder, and exposed metal parts will appear on the image after laying the powder, and the exposure intensity of the metal surface gradually increases on the image after laser melting, providing an intuitive visual clue for identifying defects of the super-high part. In laser powder bed melting monitoring, the microphone sensor is usually used to analyze the sound signals generated by the interaction of the laser and the powder to reveal detailed information about potential defects during melting and solidification; at the same time, it can also be used to monitor the deformation or collision sound signals generated by the material or component under stress. When the super-high part of the part collides with the powder laying strip or the powder laying vehicle, the energy generated by the collision is captured by the microphone. At the same time, due to the lack of powder caused by the super-high part, the laser acts on the metal part exposed to the powder, which causes a significant fluctuation in the intensity of the sound signal. Since the original signal of the acoustic sensor is in the form of a one-dimensional time series, in order to effectively extract defect-related features from complex sound signals, researchers often use its two-dimensional time-frequency representation. Among them, the short-time Fourier transform is a widely used conversion method that can quickly generate intuitive two-dimensional time-frequency representations. This method is particularly effective for analyzing non-stationary signals, and its high computational efficiency is beneficial for subsequent data fusion processing.

[0054] S200, constructing a double-branch cross-modal fusion model based on a cross-attention fusion mechanism;

[0055] Specifically, the double-branch cross-modal fusion model includes a first branch feature extraction module, a second branch feature extraction module, a residual-based linear fusion module, and a fully connected layer. The output end of the first branch feature extraction module is connected to the first input end of the residual-based linear fusion module, the output end of the second branch feature extraction module is connected to the second input end of the residual-based linear fusion module, and the output end of the residual-based linear fusion module is connected to the input end of the fully connected layer. The first branch feature extraction module includes a first feature extractor, a second feature extractor, and a first cross-attention network module, and the second branch feature extraction module includes a third feature extractor, a fourth feature extractor, and a second cross-attention network module.

[0056] S300, laser powder bed melting defect monitoring based on the double-branch cross-modal fusion model on the preprocessed laser powder fusion image and the preprocessed laser powder fusion sound signal, to obtain a laser powder bed melting defect monitoring result.

[0057] Specifically, the preprocessed laser powder fusion image and the preprocessed laser powder fusion sound signal are input into the double-branch cross-modal fusion model;

[0058] The first branch feature extraction module of the dual-branch cross-modal fusion model is used for feature extraction processing on the preprocessed laser powder fusion image to obtain laser powder fusion image features.

[0059] In this embodiment, the preprocessed laser powder fusion image is input into the first branch feature extraction module of the dual-branch cross-modal fusion model; the first feature extractor of the first branch feature extraction module is used for feature extraction processing on the first type of laser powder fusion image to obtain first type of laser powder fusion image features; the second feature extractor of the first branch feature extraction module is used for feature extraction processing on the second type of laser powder fusion image to obtain second type of laser powder fusion image features; and the first cross-attention network module of the first branch feature extraction module is used for cross-attention fusion processing on the first type of laser powder fusion image features and the second type of laser powder fusion image features to obtain laser powder fusion image features.

[0060] The second branch feature extraction module of the dual-branch cross-modal fusion model is used for feature extraction processing on the preprocessed laser powder fusion sound signal to obtain laser powder fusion sound signal features.

[0061] In this embodiment, the preprocessed laser powder fusion sound signal is input into the second branch feature extraction module of the dual-branch cross-modal fusion model; the third feature extractor of the second branch feature extraction module is used for feature extraction processing on the first type of laser powder fusion sound signal to obtain first type of laser powder fusion sound signal features; the fourth feature extractor of the second branch feature extraction module is used for feature extraction processing on the first type of laser powder fusion sound signal to obtain second type of laser powder fusion sound signal features; and the second cross-attention network module of the second branch feature extraction module is used for cross-attention fusion processing on the first type of laser powder fusion sound signal features and the second type of laser powder fusion sound signal features to obtain laser powder fusion sound signal features.

[0062] It should be noted that the dual-branch cross-modal fusion model is established, and the signal is divided into a powder laying branch, i.e., the first branch feature extraction module, and a melting branch and the second branch feature extraction module. The signal is divided into two branches for fusion, which is conducive to extracting the working signal characteristics of the powder laying vehicle in the super-high state and the working signal characteristics of laser melting, and realizing cross-modal interaction within the branch. Parallel computing of the two branches improves the fusion efficiency.

[0063] Further, as Figure 6As shown, each signal in the branch is respectively subjected to feature extraction to obtain a feature tensor of each signal with a dimension of 64*28*28; the feature extraction is 3 times of convolution, batch normalization and maximum pooling. Among them, the convolution kernel is 3*3, the convolution channel is 16, 32 and 64 respectively, the step is 1, the padding is 1, and the maximum pooling is 2*2. The feature tensors in the branch are fused based on the cross-attention mechanism to obtain a cross-modal feature tensor with a dimension of 128*28*28; in the branch, the cross-attention mechanism uses linear projection to map the two I1 and I2 inputs to obtain V1, K1, Q1 and V2, K2, Q2 vectors. Subsequently, the attention matrix W1 and W2 are calculated based on the exchange of Q1 and Q2, and W1 is multiplied by V1 and W2 is multiplied by V2, and then added to the original input to obtain output features P1 and P2. These output features effectively integrate the cross-modal information from various sensor inputs. Subsequently, P1 and P2 are spliced to the branch fusion feature S1. Similarly, the other branch can also obtain a branch fusion feature S2. The specific formula is as follows:

[0064]

[0065] P1 = W1 V1 + I1

[0066] P2 = W2 V2 + I2

[0067]

[0068] In the above formula, I1 and I2 represent two input features; and represents the linear projection of I1 and I2, i = V, K, Q; W1 and W2 represent the attention matrix, Softmax represents the softmax function, the dimension of the matrix, dim(·) represents the output dimension function. P1 and P2 represent the output features of I1 and I2 with interaction information; S1 represents the cross-modal fusion feature of the first branch, and represents the matrix dot product, represents the matrix cross product, represents feature splicing.

[0069] The residual-based linear fusion module based on the double-branch cross-modal fusion model performs linear fusion processing on the laser powder fusion image feature and the laser powder fusion sound signal feature to obtain a laser powder fusion feature;

[0070] The full connection layer based on the double-branch cross-modal fusion model maps and classifies the laser powder fusion feature to obtain a laser powder bed melting defect monitoring result.

[0071] In this embodiment, the cross-modal feature tensors of the two branches are linearly fused with residuals to obtain a total fusion feature tensor with a dimension of 128*28*28. The linearized fusion features S1 and S2 are obtained L1 L2 , which are input into residual-based linear fusion to obtain a total fusion feature S f , and the specific formula is as follows:

[0072] S f =S L1 ⊙S L2 +S L1 +S L2

[0073] , S L1 and S L2 represent the feature vectors after linear transformation of S1 and S2, S f represents the total fusion feature, and represents the dot product.

[0074] The total fusion feature tensor S f with a dimension of 128*28*28 is input into a first fully connected layer for feature mapping and conversion (with a dimension of 1*1024), and then input into a second fully connected layer for feature mapping and conversion (with a dimension of 1*3) to obtain a prediction result.

[0075] Therefore, by analyzing the abnormal features of the powder laying and melting images, the powder laying process and the melting process sound signals, the differences of different degrees of ultra-high defects in the two types of images and sound signal features are revealed, and the correlation mechanism between the ultra-high defects, the images and the sound signals is established, which provides a theoretical basis for realizing reliable data fusion monitoring. A dual-branch cross-modal fusion network is proposed for ultra-high monitoring. The signals are divided into powder laying branches and melting branches, and the cross-modal feature interaction is performed through convolutional neural networks and cross-attention mechanisms, and then the feature fusion is realized through residual-based linear fusion. This method can effectively predict the ultra-high state of the whole process, has high precision, speed and robustness, is suitable for real-time online monitoring scenes, and can be used for closed-loop feedback control of ultra-high.

[0076] In addition, it should be noted that the loss function of the dual-branch cross-modal fusion model is:

[0077]

[0078] In the above formula, Loss represents the loss function of the dual-branch cross-modal fusion model, y i represents the probability distribution of the basic true value label, y ι is the probability of the i-th class, ​The model prediction probability of the i-th class, a represents the learning rate, and N represents the number of prediction samples.

[0079] It should be noted that the model is trained, and the trained model and parameters are saved. The training process uses a cross-entropy loss function and an Adam optimizer, and the initial learning rate a0 is set to 0.0001. The cosine decay method is used to dynamically adjust the learning rate during training to adapt to complex tasks. The momentum is set to 0.9 to speed up optimization and convergence. The batch size is 4, and the model is trained for 50 epochs.

[0080] The cosine decay method is a method of dynamically adjusting the learning rate during training, and its formula is as follows:

[0081]

[0082] In the above formula, a t is the learning rate at time step t, a initial is the initial learning rate, T cur is the completed training period, T max is the total training period.

[0083] In summary, as Figure 3As shown, the embodiment of the present application first acquires, by using an industrial camera and a microphone sensor, a powder laying image after laser powder bed melting experiment, a melting image after laser powder bed melting experiment, a powder laying process sound signal, a melting process sound signal under different process parameters, and records the actual overheight state of the part in the experiment; the powder laying image and the melting image after laser powder bed melting experiment, the powder laying process sound signal and the melting process sound signal are windowed and aligned; the powder laying process sound signal and the melting process sound signal are denoised and converted into two-dimensional time-frequency diagrams; the two-dimensional time-frequency diagrams of the powder laying image and the melting image after laser powder bed melting experiment, the powder laying process sound signal and the melting process sound signal are converted into grayscale diagrams and unified in size, to obtain four feature maps with a dimension of 1*224*224; a double-branch cross-modal fusion model is established, and the signals are divided into a powder laying branch and a melting branch; feature extraction is performed on each signal in the branch to obtain a feature tensor with a dimension of 64*28*28; the obtained feature tensors in the branch are fused based on a cross-attention mechanism to obtain a cross-modal feature tensor with a dimension of 128*28*28; the cross-modal feature tensors of the two branches are linearly fused with a residual to obtain a total fusion feature tensor with a dimension of 128*28*28; the total fusion feature tensor is input into a fully connected layer; the model is trained, and the trained model and parameters are saved; the powder laying image and the melting image after laser powder bed melting experiment are obtained on site by using an industrial camera, and the powder laying process sound signal and the melting process sound signal of the laser powder bed melting process are obtained on site by using a microphone sensor; the powder laying image, the melting image after laser powder bed melting experiment, the powder laying process sound signal and the melting process sound signal on site are automatically identified by using the trained double-branch cross-modal fusion model and parameters, and the overheight degree of the part on site is predicted.

[0084] Referring to Figure 2 A laser powder bed melting defect monitoring system based on data fusion, comprising:

[0085] A first module 201 is configured to acquire laser powder fusion images and laser powder fusion sound signals and perform data preprocessing to obtain preprocessed laser powder fusion images and preprocessed laser powder fusion sound signals.

[0086] A second module 202 is configured to construct a double-branch cross-modal fusion model based on a cross-attention fusion mechanism.

[0087] A third module 203 is configured to perform laser powder bed melting defect monitoring on the preprocessed laser powder fusion images and the preprocessed laser powder fusion sound signals based on the double-branch cross-modal fusion model to obtain a laser powder bed melting defect monitoring result.

[0088] The contents in the method embodiments are applicable to the system embodiments, the system embodiments specifically implement the same functions as the method embodiments, and achieve the same beneficial effects as the method embodiments.

[0089] The above is a specific description of the preferred embodiments of the application, but the application is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the application. These equivalent modifications or replacements are all included in the scope defined by the claims of the application.

Claims

1. A laser powder bed melting defect monitoring method based on data fusion, characterized in that: The following steps are involved: Acquiring a laser powder fusion image and a laser powder fusion acoustic signal, wherein the laser powder fusion image includes a first type of laser powder fusion image and a second type of laser powder fusion image, and the laser powder fusion acoustic signal includes a first type of laser powder fusion acoustic signal and a second type of laser powder fusion acoustic signal; The laser powder fusion image and the laser powder fusion acoustic signal are respectively subjected to signal windowing and alignment processing to obtain the laser powder fusion image after windowing and alignment and the laser powder fusion acoustic signal after windowing and alignment; The laser powder fusion acoustic signal after window alignment is subjected to noise reduction and two-dimensional time-frequency conversion processing to obtain a two-dimensional time-frequency graph of the laser powder fusion acoustic signal; The two-dimensional time-frequency images of the laser powder fusion image and the laser powder fusion acoustic signal after window alignment are converted into grayscale and size-normalized to obtain the pre-processed laser powder fusion image and pre-processed laser powder fusion acoustic signal. Based on the cross-attention fusion mechanism, a dual-branch cross-modal fusion model is constructed; The dual-branch cross-modal fusion model includes a first branch feature extraction module, a second branch feature extraction module, a residual-based linear fusion module and a fully connected layer, wherein the output end of the first branch feature extraction module is connected to the first input end of the residual-based linear fusion module, the output end of the second branch feature extraction module is connected to the second input end of the residual-based linear fusion module, and the output end of the residual-based linear fusion module is connected to the input end of the fully connected layer, wherein: The first branch feature extraction module includes a first feature extractor, a second feature extractor and a first cross attention network module; The second branch feature extraction module includes a third feature extractor, a fourth feature extractor and a second cross attention network module; The pre-processed laser powder fusion image and the pre-processed laser powder fusion acoustic signal are input into the dual-branch cross-modal fusion model respectively; Based on the first branch feature extraction module of the dual-branch cross-modal fusion model, feature extraction processing is performed on the pre-processed laser powder fusion image to obtain the laser powder fusion image features; Based on the second branch feature extraction module of the dual-branch cross-modal fusion model, feature extraction processing is performed on the pre-processed laser powder fusion acoustic signal to obtain the laser powder fusion acoustic signal features; Based on the residual linear fusion module of the dual-branch cross-modal fusion model, the laser powder fusion image features and the laser powder fusion acoustic signal features are linearly fused to obtain the laser powder fusion features. Based on the fully connected layer of the dual-branch cross-modal fusion model, the laser powder fusion features are mapped and classified to obtain the laser powder bed melting defect monitoring results.

2. The laser powder bed melting defect monitoring method based on data fusion according to claim 1, characterized in that: The first branch feature extraction module based on the dual-branch cross-modal fusion model performs feature extraction processing on the pre-processed laser powder fusion image to obtain the laser powder fusion image features. This step specifically includes: The pre-processed laser powder fusion image is input into the first branch feature extraction module of the dual-branch cross-modal fusion model; Based on the first feature extractor of the first branch feature extraction module, feature extraction processing is performed on the first type of laser powder fusion image to obtain the first type of laser powder fusion image features; Based on the second feature extractor of the first branch feature extraction module, feature extraction processing is performed on the second type of laser powder fusion image to obtain the second type of laser powder fusion image features; Based on the first cross-attention network module of the first branch feature extraction module, the first type of laser powder fusion image features and the second type of laser powder fusion image features are cross-attention fused to obtain the laser powder fusion image features.

3. The laser powder bed melting defect monitoring method based on data fusion according to claim 2, characterized in that: The second branch feature extraction module based on the dual-branch cross-modal fusion model performs feature extraction processing on the pre-processed laser powder fusion acoustic signal to obtain the features of the laser powder fusion acoustic signal. This step specifically includes: The pre-processed laser powder fusion acoustic signal is input into the second branch feature extraction module of the dual-branch cross-modal fusion model; The third feature extractor based on the second branch feature extraction module performs feature extraction processing on the first type of laser powder fusion acoustic signal to obtain the first type of laser powder fusion acoustic signal features; A fourth feature extractor based on the second branch feature extraction module performs feature extraction processing on the first type of laser powder fusion acoustic signal to obtain a second type of laser powder fusion acoustic signal feature; The second cross-attention network module based on the second branch feature extraction module performs cross-attention fusion processing on the first type of laser powder fusion acoustic signal features and the second type of laser powder fusion acoustic signal features to obtain the laser powder fusion acoustic signal features.

4. The laser powder bed melting defect monitoring method based on data fusion according to claim 3, characterized in that: The loss function of the dual-branch cross-modal fusion model is: ； In the above formula, represents the loss function of the dual-branch cross-modal fusion model, represents the probability distribution of the ground truth labels, For the The probability of the class, For the The model predicts the probability of the class, represents the learning rate, Indicates the number of prediction samples.

5. A laser powder bed melting defect monitoring system based on data fusion, characterized in that: Includes the following modules: The first module is used to obtain a laser powder fusion image and a laser powder fusion acoustic signal, wherein the laser powder fusion image includes a first type of laser powder fusion image and a second type of laser powder fusion image, and the laser powder fusion acoustic signal includes a first type of laser powder fusion acoustic signal and a second type of laser powder fusion acoustic signal; The laser powder fusion image and the laser powder fusion acoustic signal are respectively subjected to signal windowing and alignment processing to obtain the laser powder fusion image after windowing and alignment and the laser powder fusion acoustic signal after windowing and alignment; The laser powder fusion acoustic signal after window alignment is subjected to noise reduction and two-dimensional time-frequency conversion processing to obtain a two-dimensional time-frequency graph of the laser powder fusion acoustic signal; The two-dimensional time-frequency images of the laser powder fusion image and the laser powder fusion acoustic signal after window alignment are converted into grayscale and size-normalized to obtain the pre-processed laser powder fusion image and pre-processed laser powder fusion acoustic signal. The second module is used to build a dual-branch cross-modal fusion model based on the cross-attention fusion mechanism; The dual-branch cross-modal fusion model includes a first branch feature extraction module, a second branch feature extraction module, a residual-based linear fusion module and a fully connected layer, wherein the output end of the first branch feature extraction module is connected to the first input end of the residual-based linear fusion module, the output end of the second branch feature extraction module is connected to the second input end of the residual-based linear fusion module, and the output end of the residual-based linear fusion module is connected to the input end of the fully connected layer, wherein: The first branch feature extraction module includes a first feature extractor, a second feature extractor and a first cross attention network module; The second branch feature extraction module includes a third feature extractor, a fourth feature extractor and a second cross attention network module; The third module is used to input the pre-processed laser powder fusion image and the pre-processed laser powder fusion acoustic signal into the dual-branch cross-modal fusion model respectively; Based on the first branch feature extraction module of the dual-branch cross-modal fusion model, feature extraction processing is performed on the pre-processed laser powder fusion image to obtain the laser powder fusion image features; Based on the second branch feature extraction module of the dual-branch cross-modal fusion model, feature extraction processing is performed on the pre-processed laser powder fusion acoustic signal to obtain the laser powder fusion acoustic signal features; Based on the residual linear fusion module of the dual-branch cross-modal fusion model, the laser powder fusion image features and the laser powder fusion acoustic signal features are linearly fused to obtain the laser powder fusion features. Based on the fully connected layer of the dual-branch cross-modal fusion model, the laser powder fusion features are mapped and classified to obtain the laser powder bed melting defect monitoring results.

Images