Laser additive manufacturing defect monitoring method, system, medium, equipment and device

Abstract

The invention provides a laser additive manufacturing defect monitoring method, system, medium, equipment and device, and belongs to the technical field of laser additive manufacturing defect recognition. A laser additive manufacturing defect monitoring method comprises the steps that X-ray image information and sound signal information in the melting process of a laser powder bed are obtained; preprocessing the X-ray image information and the sound signal information to obtain preprocessed X-ray image information and sound signal information; marking the preprocessed X-ray image information and sound signal information to construct a data set; inputting the preprocessed X-ray image information and sound signal information into a multi-modal teacher model for training so as to obtain defect identification and positioning capability based on multi-modal information; inputting the preprocessed sound signal information into a single-mode student model to obtain a trained single-mode student model; and migrating the defect identification and positioning capability based on the multi-modal information to the trained single-modal student model through a knowledge distillation strategy.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of laser additive manufacturing defect identification, and more particularly to a laser additive manufacturing defect monitoring method, system, medium, equipment and device. BACKGROUND

[0002] In the laser additive manufacturing process, the interaction of laser and powder particles exists, and various changes such as powder melting and solidification forming exist at the same time. During this period, various different types of signals such as optical signals, acoustic signals and thermal signals are generated. Rich information can be obtained from these signals, so as to obtain the defect condition and processing state, which is crucial for improving the quality of the final product. The current defect identification technology path in the industry is diverse. X-ray high-speed imaging technology can accurately monitor the molten pool dynamics, keyhole and other microstructures, and clearly capture the pore defect formation process and residual defects after printing. Acoustic signal acquisition technology can sense the internal information of the molten region, and the equipment is cheap and easy to install, with good time resolution.

[0003] The defect identification technology in the prior art has the following defects: the X-ray high-speed imaging technology is limited by radiation characteristics and needs to be used in a specific experimental environment with protection, and cannot be directly used for real-time accurate monitoring of part manufacturing in a conventional printer, so the application range is limited; although the acoustic signal acquisition technology is flexible and low in cost, it is easily disturbed by environmental background noise, and the defect characteristics contained in the acoustic signal are indirect and single, so it is difficult to accurately distinguish between defects such as incomplete fusion, spheroidization and porosity, and the conventional acoustic signal monitoring has the problems of poor recognition quality and insufficient precision. SUMMARY

[0004] The purpose of the present application is to provide a laser additive manufacturing defect monitoring method, which aims to solve the problems of limited application range of X-ray high-speed imaging technology, poor recognition quality and insufficient precision of single-mode acoustic signal detection in the prior art.

[0005] To achieve the above-mentioned purpose, the technical solution adopted by the present application is: In a first aspect, a laser additive manufacturing defect monitoring method is provided, comprising: obtaining X-ray image information and acoustic signal information in a laser powder bed melting process; preprocessing the X-ray image information and the acoustic signal information to obtain preprocessed X-ray image information and acoustic signal information; labeling the preprocessed X-ray image information and acoustic signal information to construct a data set; inputting the preprocessed X-ray image information and acoustic signal information into a multi-modal teacher model for training to obtain defect recognition and positioning ability based on multi-modal information; The preprocessed acoustic signal information is input into the single-modal student model to obtain the trained single-modal student model. The defect identification and localization capabilities based on multimodal information are transferred to the trained unimodal student model through a knowledge distillation strategy.

[0006] In one possible implementation, the preprocessing of the X-ray image information and the acoustic signal information to obtain preprocessed X-ray image information and acoustic signal information includes: Align the X-ray image information and the acoustic signal information during each laser powder bed melting process; Extract local regions from the X-ray image information, including the laser-powder interaction area, and perform noise reduction and image resolution unification; The acoustic signal information is subjected to noise reduction processing to eliminate background noise.

[0007] In one possible implementation, aligning the X-ray image information and the acoustic signal information during each laser powder bed melting process includes: The X-ray image information and the acoustic signal information of each laser powder bed melting process are aligned according to the time sequence; The X-ray image information and the acoustic signal information over a certain period of time are grouped together.

[0008] In one possible implementation, the step of labeling the preprocessed X-ray image information and acoustic signal information to construct a dataset includes: The preprocessed X-ray image information and acoustic signal information are labeled to obtain labeled data; The labeled data is divided into a training set and a validation set.

[0009] In one possible implementation, the labeling of the preprocessed X-ray image information and acoustic signal information to obtain labeled data includes: Label the preprocessed X-ray image information; Acoustic signal information with the same time sequence as the preprocessed X-ray image information is labeled with the same tag.

[0010] In one possible implementation, the label includes unfused, spherical, porous, and good.

[0011] In one possible implementation, the step of inputting the preprocessed X-ray image information and acoustic signal information into a multimodal teacher model for training to obtain defect identification and localization capabilities based on multimodal information includes: Based on the preprocessed X-ray image information and acoustic signal information, image feature vectors and acoustic feature vectors are extracted respectively; The image feature vector and the acoustic feature vector are fused to obtain a multimodal information feature vector; Based on the multimodal information feature vector, a defect category probability distribution based on multimodal information is obtained; The output, based on the defect category probability distribution of the multimodal information, includes the information number and the defect probability.

[0012] In one possible implementation, inputting the preprocessed acoustic signal information into a single-modal student model to obtain a trained single-modal student model includes: Based on the preprocessed acoustic signal information, acoustic feature vectors are extracted; Based on the acoustic feature vectors, a probability distribution of defect categories based on single-modal information is obtained; The output, based on the defect category probability distribution of the single-modal information, includes the information number and the defect probability.

[0013] In one possible implementation, the transfer of the defect identification and localization capabilities based on multimodal information to the trained unimodal student model via a knowledge distillation strategy includes: Extract the predicted values ​​calculated by the fully connected layer obtained from training the multimodal teacher model; The cross-entropy loss function of the true labels is calculated based on the trained unimodal student model. The soft tag distillation loss function is calculated based on the predicted values ​​calculated by the fully connected layer and the probability distribution of defect categories based on single-modal information. The single-modal distillation loss function is calculated based on the cross-entropy loss function and the soft-label distillation loss function. The parameters of the student model are updated based on the total loss function of the single-modal distillation.

[0014] Secondly, a data analysis-based construction safety monitoring system is provided, applied to the laser additive manufacturing defect monitoring method described in the first aspect, including: The information acquisition unit is used to acquire X-ray image information and acoustic signal information during the laser powder bed melting process; The information preprocessing unit preprocesses the X-ray image information and the acoustic signal information to obtain preprocessed X-ray image information and acoustic signal information. The information labeling and dataset construction unit is used to obtain and label the preprocessed X-ray image information and acoustic signal information in order to construct a dataset. The multimodal teacher model training unit is used to input the preprocessed X-ray image information and acoustic signal information into the multimodal teacher model for training, so as to obtain the defect identification and localization capability based on multimodal information; A single-modal student model training unit is used to input the preprocessed acoustic signal information into a single-modal student model to obtain a trained single-modal student model. The knowledge distillation unit is used to transfer the defect identification and localization capabilities based on multimodal information to the trained unimodal student model through a knowledge distillation strategy.

[0015] Thirdly, a storage medium is provided storing a computer-readable program for implementing the laser additive manufacturing defect monitoring method described in the first aspect.

[0016] Fourthly, an electronic device is provided, including a memory and a processor, the memory being communicatively connected to the processor, the memory storing a computer-readable program for the processor to implement the laser additive manufacturing defect monitoring method as described in the first aspect.

[0017] Fifthly, an apparatus is provided, comprising the electronic device as described in the fourth aspect.

[0018] The beneficial effects of the laser additive manufacturing defect monitoring method provided by this invention are as follows: Compared with the prior art, this laser additive manufacturing defect monitoring method can simultaneously acquire X-ray image information and acoustic signal information during the laser powder bed melting process. These two types of information reflect the state of the manufacturing process from different dimensions, providing more comprehensive raw data support for subsequent defect analysis. X-ray image information can present details related to the microstructure such as molten pool dynamics and keyholes, while acoustic signal information can sense the internal conditions of the molten area. The combination of the two breaks the limitations of a single information source, making the perception of the manufacturing process more three-dimensional.

[0019] The preprocessed multimodal information is input into the multimodal teacher model for training, enabling the model to integrate the advantages of both types of information and learn richer and more accurate features related to defect identification and localization. By leveraging the complementarity of the two types of information, the multimodal teacher model can more accurately grasp the characteristic patterns of different defect types, thus possessing stronger defect identification and localization capabilities based on multimodal information, providing a high-quality "knowledge source" for subsequent knowledge transfer.

[0020] By inputting preprocessed acoustic signal information into a single-modal student model for training, the model can first acquire the basic ability to perform defect analysis based on acoustic signal information. Acoustic signal acquisition equipment is characterized by its low cost, ease of installation, good temporal resolution, and flexible deployment. Initial training of the single-modal student model based on this information can initially form a model foundation that meets the needs of convenient deployment in practical application scenarios, preparing for subsequent performance improvements by incorporating multimodal knowledge.

[0021] By employing a knowledge distillation strategy, the defect identification and localization capabilities of the multimodal teacher model are transferred to the trained unimodal student model. This allows the unimodal student model to retain the advantage of convenient acoustic signal information acquisition while achieving higher defect identification and localization accuracy. This transfer process enables the unimodal student model to draw upon the microstructural features learned by the multimodal teacher model from X-ray image information, compensating for the indirectness and singularity of defect feature representation in single acoustic signal information, and significantly improving the monitoring performance of the unimodal student model in practical applications.

[0022] Through this process, the resulting single-modal student model combines the flexibility and cost advantages of acoustic signal monitoring in deployment with the defect identification and positioning accuracy approaching that of a multimodal model. This allows it to better meet the real-time monitoring needs of parts manufacturing in conventional printers, effectively improving the practicality and reliability of defect monitoring in laser additive manufacturing and providing strong technical support for improving the quality of the final product. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A schematic diagram illustrating the main steps of the laser additive manufacturing defect monitoring method provided in this embodiment of the invention; Figure 2 A schematic flowchart of the laser additive manufacturing defect monitoring method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the multimodal teacher model provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a single-modal student model provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the student model feature extraction module provided in an embodiment of the present invention; Figure 6This is a schematic diagram of the device for acquiring X-ray image information and acoustic signal information during the melting process of laser powder bed, as provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0026] Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0027] It should be further noted that the accompanying drawings and embodiments of the present invention mainly describe the concept of the present invention. Based on this concept, some specific forms and arrangements of connection relationships, positional relationships, power mechanisms, power supply systems, hydraulic systems and control systems may not be fully described. However, under the premise that those skilled in the art understand the concept of the present invention, they can implement the above-mentioned specific forms and arrangements in a well-known manner.

[0028] When a component is referred to as being "fixed to" or "set on" another component, it can be directly on or indirectly on that other component. When a component is referred to as being "connected to" another component, it can be directly connected to or indirectly connected to that other component.

[0029] In the description of this invention, "a plurality of" means two or more, and "several" means one or more, unless otherwise explicitly specified.

[0030] The directional terms "inner" and "outer" refer to the inner and outer contours of each component itself. The terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention.

[0031] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," and "above" are used here to describe the spatial positional relationship between a device or feature and other devices or features, as shown in the figure. It should be understood that spatial relative terms are intended to... The invention includes different orientations of the device in use or operation, in addition to those described in the figures. For example, if a device in the figures is inverted, a device described as "above" or "on top of" other devices or structures will be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below". The device may also be positioned in other different ways, and the spatial relative descriptions used herein are interpreted accordingly. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the invention, "a plurality of" means two or more, and "a number" means one or more, unless otherwise explicitly specified.

[0032] Reference Figures 1 to 6 The present invention will now describe the laser additive manufacturing defect monitoring method, system, medium, equipment and apparatus provided by the present invention.

[0033] Firstly, a method for detecting defects in laser additive manufacturing is provided, including: S100. Acquire X-ray image information and acoustic signal information during the laser powder bed melting process.

[0034] During the laser additive manufacturing process, an in-situ high-speed X-ray imaging system and an acoustic signal monitoring system are used simultaneously to acquire X-ray images and acoustic signals of the laser powder bed melting process and obtain raw experimental data.

[0035] S200. Preprocess the X-ray image information and the acoustic signal information to obtain preprocessed X-ray image information and acoustic signal information.

[0036] In one possible implementation, step S200 involves preprocessing the X-ray image information and the acoustic signal information to obtain preprocessed X-ray image information and acoustic signal information, including: S210. Align the X-ray image information and acoustic signal information for each laser powder bed melting process.

[0037] In one possible implementation, step S210, aligning the X-ray image information and acoustic signal information during each laser powder bed melting process, includes: S211. Align the X-ray image information and acoustic signal information of each laser powder bed melting process according to the time sequence.

[0038] When acquiring X-ray image and acoustic signal information during each laser powder bed melting process, timestamps are embedded in the X-ray image and acoustic signal information. The X-ray image and acoustic signal information are then aligned according to time sequence using the timestamps included in the X-ray image and acoustic signal information.

[0039] S222. The X-ray image information and acoustic signal information over a certain period of time are grouped together.

[0040] S220. Extract local areas from X-ray image information, including the laser-powder interaction area, and perform noise reduction and image resolution unification.

[0041] First, calibration is completed based on the equipment coordinates. A mapping relationship between "image pixels and physical coordinates" is established through metal markers. Then, the initial working area is delineated in conjunction with the laser scanning path. Subsequently, the Canny edge detection algorithm is used to identify the outline of the molten pool. If the molten pool is detected to exceed the initial area, the cropping range is expanded by 50 pixels horizontally and vertically, and finally cropped into a 256×256 pixel image to ensure complete coverage of the molten pool and the powder area within a 1mm radius around it.

[0042] A background template was generated by acquiring 100 frames of laser-free powder bed images. This template was then compared with the original X-ray image to eliminate fixed scattered line interference. Next, a 5×5 Gaussian filter was used to smooth the image, reducing "haze" noise. Subsequently, a 3×3 median filter was used to remove random electron bright spots and dark spots from the image. Finally, a local mean thresholding method was used to replace pixels with abnormal grayscale values, further improving image clarity.

[0043] The image resolution is checked. If the resolution is higher than 256×256, a bilinear interpolation algorithm is used to reduce it to the target size. If the resolution is lower than 256×256, it is first enlarged to near the target size using an interpolation algorithm, and then padded to 256×256 using edge filling. Throughout the process, the pixel aspect ratio deviation is ensured to be no more than 0.1%. At the same time, the image grayscale range is stretched to 0-255, and then the grayscale distribution is unified by mean-variance normalization to eliminate the image brightness difference caused by the intensity fluctuation of the X-ray source and ensure that the image specifications are adapted to the input requirements of the subsequent model.

[0044] The technical solutions in this part are existing technologies in this field. Those skilled in the art can clearly obtain the relevant technical solutions from the existing technologies, and they will not be described in detail in this application.

[0045] S230. Perform noise reduction processing on the acoustic signal information to eliminate background noise.

[0046] S300. Label the preprocessed X-ray image information and acoustic signal information to construct a dataset.

[0047] In one possible implementation, step S300 involves labeling the preprocessed X-ray image information and acoustic signal information to construct a dataset, including: S310. Label the preprocessed X-ray image information and acoustic signal information to obtain labeled data.

[0048] In one possible implementation, S310. Labels are applied to the preprocessed X-ray image information and acoustic signal information to obtain labeled data, including: S311. Label the preprocessed X-ray image information.

[0049] S312. Label acoustic signal information with the same time series as the preprocessed X-ray image information with the same label.

[0050] Observe the X-ray images and label those with defects, indicating the defect type on the label. Locate the corresponding acoustic signal segments in the images over time and label them as well, indicating the defect type. Mark defect-free signals as good.

[0051] In one possible implementation, the labels include unfused, spherical, porous, and good.

[0052] S320. Divide the labeled data into training and validation sets. Classify the labeled data into training and validation sets in an 8:2 ratio.

[0053] S400. Input the preprocessed X-ray image information and acoustic signal information into the multimodal teacher model for training to obtain the ability to identify and locate defects based on multimodal information.

[0054] The multimodal teacher model includes: a backbone network, a multimodal feature fusion module, and a decision module.

[0055] After the backbone network reads the data, it segments the data for analysis. It has two branches, one for extracting image signal features and the other for extracting acoustic signal features. In the image signal branch, X-ray images from the training set are input, and the backbone network extracts image features, outputting an image feature vector. These image features specifically include geometric morphology features, grayscale distribution features, texture structure features, and dynamic correlation features.

[0056] In the acoustic signal branch, acoustic signal segments from the training set are input. The acoustic signals are first transformed into a time-frequency image through a short-time Fourier transform in the model, then audio features are extracted, and the processed output is an acoustic feature vector. The extracted audio features include time-frequency domain features, spectral features, Mel-frequency cepstral coefficients, and wavelet domain features.

[0057] First, the preprocessed acoustic signal is framed and windowed; then, the time-domain acoustic signal is converted into a time-frequency image through short-time Fourier transform, which intuitively presents the frequency change over time; next, the time-frequency domain, spectrum, MFCC, and wavelet domain features are extracted from the time-frequency image; then, the extracted features are normalized and dimensionality reduced; finally, the processed features are concatenated to generate a fixed-dimensional acoustic feature vector for subsequent use by the model.

[0058] The multimodal feature fusion module inputs the image obtained after feature extraction and the acoustic feature vector into the multimodal feature fusion module. An attention mechanism is added to the module to dynamically adjust the weight of the modal features. The feature vectors of the two modalities are concatenated to output the multimodal information feature vector.

[0059] The technical solutions in this part are existing technologies in this field. Those skilled in the art can clearly obtain the relevant technical solutions from the existing technologies, and they will not be described in detail in this application.

[0060] The decision module inputs the multimodal information feature vectors into the fully connected classifier, and after nonlinear mapping and output, obtains the probability distribution of the defect category.

[0061] In one possible implementation, S400. preprocessed X-ray image information and acoustic signal information are input into a multimodal teacher model for training to obtain defect identification and localization capabilities based on multimodal information, including: S410. Extract image feature vectors and acoustic feature vectors based on the preprocessed X-ray image information and acoustic signal information, respectively.

[0062] After the backbone network of the multimodal teacher model reads the data, it segments the data for analysis. It has two branches, one for extracting image signal features and the other for extracting acoustic signal features. In the image signal branch, X-ray images from the training set are input, and the backbone network extracts image features, outputting an image feature vector. In the acoustic signal branch, acoustic signal segments from the training set are input. The acoustic signals are first transformed into time-frequency images through a short-time Fourier transform in the model, then audio features are extracted, and finally, an acoustic feature vector is output.

[0063] S420. The image feature vector and the acoustic feature vector are fused to obtain the multimodal information feature vector.

[0064] The image feature vector and the acoustic feature vector are fused using the following formula:

[0065] in, It is a multimodal information feature vector; Image feature vector; For acoustic feature vectors; , For attention weights.

[0066] S430. Obtain the probability distribution of defect categories based on multimodal information feature vectors.

[0067] The fully connected layers of the multimodal teacher model take multimodal information feature vectors as input and calculate the predicted values ​​using the following formula:

[0068] in, These are the predicted values ​​for the fully connected layer; For the first The weight matrix of the class; It is a multimodal information feature vector; For the first The bias vector of the class.

[0069] The probability distribution of defect categories is calculated based on the predicted values ​​from the fully connected layer using the following formula:

[0070] in, For the first The probability of a class; For the first Predicted values ​​for the fully connected layer of the class.

[0071] S440. Output the results, including information number and defect probability, based on the defect category probability distribution based on multimodal information.

[0072] S500. Input the preprocessed acoustic signal information into the single-modal student model to obtain the trained single-modal student model.

[0073] The unimodal student model uses a lightweight ResNet architecture, which includes a feature extraction module and a classification decision module.

[0074] In the feature extraction module, the acoustic signal segments in the training set are input, the long segments are divided into multiple short segments, and converted into time-frequency images. These images are then passed through convolutional layers, pooling layers, and multiple residual feature extraction modules. After the feature map is compressed in the global average pooling layer, it is finally normalized to obtain the acoustic feature vector.

[0075] The decision module takes feature vectors as input to a fully connected classifier, performs nonlinear mapping, and outputs predicted values ​​for each type of defect, ultimately obtaining the predicted probability of the defect category. The output results are arranged by time, corresponding to the defect situation in each time period, to determine the defect location.

[0076] In one possible implementation, S500 inputs the preprocessed acoustic signal information into the single-modal student model to obtain the trained single-modal student model, including: S510. Extract acoustic feature vectors based on preprocessed acoustic signal information.

[0077] In the feature extraction module, the acoustic signal segments in the training set are input, the long segments are divided into multiple short segments, and converted into time-frequency images. These images are then passed through convolutional layers, pooling layers, and multiple residual feature extraction modules. After the feature map is compressed in the global average pooling layer, it is finally normalized to obtain the acoustic feature vector.

[0078] S520. Obtain the probability distribution of defect categories based on single-modal information based on acoustic feature vectors.

[0079] The fully connected layers of the student model take acoustic feature vectors as input and calculate the predicted values ​​using the following formula:

[0080] in, These are the predicted values ​​for the fully connected layer; For the first The weight matrix of the class; For acoustic feature vectors; For the first The bias vector of the class.

[0081] The probability distribution of defect categories is calculated based on the predicted values ​​from the fully connected layer using the following formula:

[0082] in, For the first The probability of a class; For the first Predicted values ​​for the fully connected layer of the class.

[0083] S530. Output the results, including the information number and the defect probability, based on the defect category probability distribution based on single-modal information.

[0084] S600. The defect identification and localization capabilities based on multimodal information are transferred to the trained unimodal student model through a knowledge distillation strategy.

[0085] In one possible implementation, S600 transfers the defect identification and localization capabilities based on multimodal information to the trained unimodal student model through a knowledge distillation strategy, including: S610. Extract the predicted values ​​calculated by the fully connected layer obtained from training the multimodal teacher model.

[0086] S620. Calculate the cross-entropy loss function based on the trained unimodal student model and the true labels.

[0087] The cross-entropy loss function is:

[0088] in, The cross-entropy loss function; For the first The labels of each sample; For the first The student model logit output for each sample.

[0089] S630. Calculate the soft-label distillation loss function based on the predicted values ​​calculated by the fully connected layer and the probability distribution of defect categories based on single-modal information.

[0090] The soft-label distillation loss function is:

[0091] in, This is the loss function for soft-label distillation; For the first The logit output of the teacher model for each sample.

[0092] S640. Calculate the single-mode distillation loss function based on the cross-entropy loss function and the soft-label distillation loss function.

[0093] The single-mode distillation loss function is:

[0094] in, This is the single-mode distillation loss function; for Parameters; for Parameters; The cross-entropy loss function; This is the loss function for soft-label distillation.

[0095] S650. Update the parameters of the student model based on the total loss function of single-mode distillation.

[0096] The beneficial effects of the laser additive manufacturing defect monitoring method provided by this invention are as follows: Compared with the prior art, this laser additive manufacturing defect monitoring method can simultaneously acquire X-ray image information and acoustic signal information during the laser powder bed melting process. These two types of information reflect the state of the manufacturing process from different dimensions, providing more comprehensive raw data support for subsequent defect analysis. X-ray image information can present details related to the microstructure such as molten pool dynamics and keyholes, while acoustic signal information can sense the internal conditions of the molten area. The combination of the two breaks the limitations of a single information source, making the perception of the manufacturing process more three-dimensional.

[0097] The preprocessed multimodal information is input into the multimodal teacher model for training, enabling the model to integrate the advantages of both types of information and learn richer and more accurate features related to defect identification and localization. By leveraging the complementarity of the two types of information, the multimodal teacher model can more accurately grasp the characteristic patterns of different defect types, thus possessing stronger defect identification and localization capabilities based on multimodal information, providing a high-quality "knowledge source" for subsequent knowledge transfer.

[0098] By inputting preprocessed acoustic signal information into a single-modal student model for training, the model can first acquire the basic ability to perform defect analysis based on acoustic signal information. Acoustic signal acquisition equipment is characterized by its low cost, ease of installation, good temporal resolution, and flexible deployment. Initial training of the single-modal student model based on this information can initially form a model foundation that meets the needs of convenient deployment in practical application scenarios, preparing for subsequent performance improvements by incorporating multimodal knowledge.

[0099] By employing a knowledge distillation strategy, the defect identification and localization capabilities of the multimodal teacher model are transferred to the trained unimodal student model. This allows the unimodal student model to retain the advantage of convenient acoustic signal information acquisition while achieving higher defect identification and localization accuracy. This transfer process enables the unimodal student model to draw upon the microstructural features learned by the multimodal teacher model from X-ray image information, compensating for the indirectness and singularity of defect feature representation in single acoustic signal information, and significantly improving the monitoring performance of the unimodal student model in practical applications.

[0100] Through this process, the resulting single-modal student model combines the flexibility and cost advantages of acoustic signal monitoring in deployment with the defect identification and positioning accuracy approaching that of a multimodal model. This allows it to better meet the real-time monitoring needs of parts manufacturing in conventional printers, effectively improving the practicality and reliability of defect monitoring in laser additive manufacturing and providing strong technical support for improving the quality of the final product.

[0101] Secondly, a data analysis-based construction safety monitoring system is provided, applied to the laser additive manufacturing defect monitoring method described in the first aspect, including: The information acquisition unit is used to acquire X-ray image information and acoustic signal information during the laser powder bed melting process; The information preprocessing unit preprocesses the X-ray image information and acoustic signal information to obtain preprocessed X-ray image information and acoustic signal information; The information labeling and dataset construction unit is used to obtain and label the preprocessed X-ray image information and acoustic signal information in order to construct a dataset. The multimodal teacher model training unit is used to input the preprocessed X-ray image information and acoustic signal information into the multimodal teacher model for training, so as to obtain the defect identification and localization capability based on multimodal information; A single-modal student model training unit is used to input the preprocessed acoustic signal information into a single-modal student model to obtain a trained single-modal student model. The knowledge distillation unit is used to transfer the defect identification and localization capabilities based on multimodal information to the trained unimodal student model through a knowledge distillation strategy.

[0102] Thirdly, a storage medium is provided storing a computer-readable program for implementing the laser additive manufacturing defect monitoring method described in the first aspect.

[0103] Fourthly, an electronic device is provided, including a memory and a processor, the memory being communicatively connected to the processor, the memory storing a computer-readable program for the processor to implement the laser additive manufacturing defect monitoring method as described in the first aspect.

[0104] Fifthly, an apparatus is provided, comprising the electronic device as described in the fourth aspect.

[0105] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0106] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0107] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

Claims

1. A method of laser additive manufacturing defect monitoring, characterized in that, The method comprises the steps of: acquiring X-ray image information and acoustic signal information in a laser powder bed melting process; preprocessing the X-ray image information and the acoustic signal information to obtain preprocessed X-ray image information and acoustic signal information; labeling the preprocessed X-ray image information and acoustic signal information to construct a data set; inputting the preprocessed X-ray image information and acoustic signal information into a multi-modal teacher model for training to obtain a defect recognition and positioning capability based on multi-modal information; inputting the preprocessed acoustic signal information into a single-modal student model to obtain a trained single-modal student model; migrating the defect recognition and positioning capability based on multi-modal information to the trained single-modal student model through a knowledge distillation strategy.

2. The laser additive manufacturing defect monitoring method of claim 1, wherein, The preprocessing of the X-ray image information and the acoustic signal information to obtain preprocessed X-ray image information and acoustic signal information comprises: aligning the X-ray image information and the acoustic signal information in each laser powder bed melting process; extracting a local area including a laser-powder interaction region in the X-ray image information and performing noise reduction and uniform image resolution; performing noise reduction processing on the acoustic signal information to eliminate background noise.

3. The laser additive manufacturing defect monitoring method of claim 2, wherein, The aligning of the X-ray image information and the acoustic signal information in each laser powder bed melting process comprises: aligning the X-ray image information and the acoustic signal information in each laser powder bed melting process in time sequence; grouping the X-ray image information and the acoustic signal information in a certain time.

4. The laser additive manufacturing defect monitoring method of claim 1, wherein, The labeling of the preprocessed X-ray image information and acoustic signal information to construct a data set comprises: labeling the preprocessed X-ray image information and acoustic signal information to obtain labeled data; dividing the labeled data into a training set and a validation set.

5. The laser additive manufacturing defect monitoring method of claim 4, wherein, The labeling of the preprocessed X-ray image information and acoustic signal information to obtain labeled data comprises: labeling the preprocessed X-ray image information; labeling the acoustic signal information with the same time sequence as the preprocessed X-ray image information with the same label.

6. The laser additive manufacturing defect monitoring method of claim 5, wherein, The label includes non-fusion, spheroidization, porosity, and good.

7. The laser additive manufacturing defect monitoring method of claim 4, wherein, The training of the preprocessed X-ray image information and acoustic signal information in a multi-modal teacher model to obtain a defect recognition and positioning capability based on multi-modal information comprises: extracting image feature vectors and acoustic feature vectors based on the preprocessed X-ray image information and acoustic signal information; fusing the image feature vectors and the acoustic feature vectors to obtain multi-modal information feature vectors; obtaining a defect category probability distribution based on multi-modal information based on the multi-modal information feature vectors; outputting a result including information number and defect probability according to the defect category probability distribution based on multi-modal information.

8. The laser additive manufacturing defect monitoring method of claim 7, wherein, The inputting of the preprocessed acoustic signal information into a single-modal student model to obtain a trained single-modal student model comprises: extracting acoustic feature vectors based on the preprocessed acoustic signal information; obtain a defect class probability distribution based on single-modal information based on the acoustic feature vector; output a result including information number and defect probability according to the defect class probability distribution based on single-modal information.

9. The laser additive manufacturing defect monitoring method of claim 8, wherein, The transferring the defect recognition and positioning ability based on multi-modal information to the trained single-modal student model through the knowledge distillation strategy includes: extracting a prediction value calculated by a full connection layer of the multi-modal teacher model; calculating a cross-entropy loss function of a real label based on the trained single-modal student model; calculating a soft label distillation loss function according to the prediction value calculated by the full connection layer and the defect class probability distribution based on single-modal information; calculating a single-modal distillation loss function based on the cross-entropy loss function and the soft label distillation loss function; updating parameters of the student model based on the single-modal distillation total loss function.

10. A laser additive manufacturing defect real-time monitoring system based on multi-modal knowledge distillation, applied to the laser additive manufacturing defect monitoring method according to any one of claims 1 to 9, characterized in that, comprise: an information acquisition unit configured to acquire X-ray image information and acoustic signal information in a laser powder bed fusion process; an information preprocessing unit configured to preprocess the X-ray image information and the acoustic signal information to obtain preprocessed X-ray image information and acoustic signal information; an information labeling and dataset construction unit configured to label the preprocessed X-ray image information and acoustic signal information to construct a dataset; a multi-modal teacher model training unit configured to input the preprocessed X-ray image information and acoustic signal information into a multi-modal teacher model to train the multi-modal teacher model to obtain a defect recognition and positioning ability based on multi-modal information; a single-modal student model training unit configured to input the preprocessed acoustic signal information into a single-modal student model to obtain a trained single-modal student model; a knowledge distillation unit configured to transfer the defect recognition and positioning ability based on multi-modal information to the trained single-modal student model through a knowledge distillation strategy.

11. A storage medium, characterized by A computer readable program is stored in the memory, and the computer readable program is used to implement the laser additive manufacturing defect monitoring method according to any one of claims 1-9.

12. An electronic device, comprising: The memory and the processor are connected in communication, and the memory stores a computer readable program for the processor to implement the laser additive manufacturing defect monitoring method according to any one of claims 1-9.

13. An apparatus, comprising: The electronic device according to claim 12 is included. The electronic device according to claim 12 is included.

Images