Laser powder bed melting defect detection method based on privacy enhancement federal learning

Abstract

The invention relates to a laser powder bed fusion defect detection method based on privacy enhanced federated learning, and the method comprises the steps: collecting a powder bed image in a laser powder bed fusion printing process of each participating user based on self-adaptive differential privacy federated learning, carrying out the corresponding preprocessing and calibration of image data, and constructing a respective data set; obtaining a global defect identification model by adopting federal learning of adaptive differential privacy; constructing a loss function and carrying out repeated training until the global model meets the requirements of defect detection performance and privacy protection; and each participating user downloads the updated global model from the central server, and the updated global model is used for detecting the powder laying defect in the fusion printing process of the respective laser powder bed on line.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of laser powder bed fusion, in particular to a laser powder bed fusion defect detection method based on privacy-enhanced federated learning. BACKGROUND

[0002] As one of the most widely used technologies in the field of additive manufacturing, Laser Powder Bed Fusion (LPBF) has shown significant advantages in weapon manufacturing, aerospace and medical fields. For example, in weapon manufacturing, LPBF can realize the integrated production of high-precision metal parts while meeting the security packaging needs of classified parts; in the aerospace field, its lightweight complex structure manufacturing capability combined with core process parameter encryption technology can effectively protect intellectual property rights; in the medical field, LPBF can not only customize personalized orthopedic implants, but also protect patient privacy through data desensitization technology. However, this technology faces severe quality stability challenges in actual application - the interaction of multiple physical fields (thermal-flow-solid coupling) easily leads to defects in the forming process. Especially in the powder laying stage, the complex interaction between the scraper and the powder bed often causes problems such as warping and uneven powder laying, while the steep temperature gradient generated by the high-energy laser beam in the melting stage exacerbates the molten pool fluctuation. These dynamic abnormalities, if not detected in real time, will accumulate layer by layer and eventually lead to printing failure.

[0003] Training a defect recognition model by combining multiple parties' powder bed image data can better improve the generalization and accuracy of powder bed defect recognition. However, federated learning methods still face three challenges in industrial scenarios: first, the scarcity and heterogeneity of powder bed defect data make it difficult to train a model with excellent performance, and the differences in data distribution across devices and processes make it difficult for a single model to adapt to complex working conditions; second, the conflict between the strict data privacy requirements in military, medical and other fields and the reliance of deep learning on data sharing is difficult to reconcile, traditional federated learning frameworks can achieve distributed training, but device heterogeneity makes it difficult for the model to converge, and data isolation requirements limit cross-institutional knowledge transfer; third, existing privacy protection technologies (such as differential privacy) lack quantitative evaluation of the importance of defect features, and global noise addition strategies can easily destroy key defect features, resulting in a significant decrease in model recognition accuracy.

[0004] The above information disclosed in the background section is only intended to enhance the understanding of the background of the present application, and therefore can contain information that is not prior art known to those of ordinary skill in the art. SUMMARY

[0005] The application provides a laser powder bed fusion defect detection method and device based on privacy-enhanced federated learning, applies adaptive differential privacy and federated learning to the defect identification field of the LPBF powder laying process, collects data from different factories or manufacturers, adds Gaussian noise when each participating user uploads gradient information to the central server to realize differential privacy and ensure data security. In addition, in order to solve the problem of obvious decline of identification performance caused by differential privacy, more noise is added to the powder bed background which is not concerned, and less noise is added to the foreground features containing defect information, so as to realize the balance between privacy protection and model performance. By designing the feature decoupling of foreground and background, the sensitive data is protected while the key defect features are reserved, aiming to realize the collaborative training of "available but invisible" process data.

[0006] A laser powder bed fusion defect detection method based on privacy-enhanced federated learning comprises:

[0007] S100: In the laser powder bed fusion printing process based on federated learning of adaptive differential privacy, the powder bed image is collected, the image data is preprocessed and calibrated, and the respective data set is constructed;

[0008] S200: Selecting a defect detection model Attention-Unet as the neural network of federated learning of adaptive differential privacy, the network is composed of an encoder, a decoder and a jump connection, and introducing a grouping convolution method based on Attention-Unet to realize the feature decoupling of powder bed image foreground and background information;

[0009] S300: After the loss function is constructed, each participating user first downloads the defect detection model from the central server, trains the defect identification model locally using the local data set, then uploads the model gradient information processed by adaptive differential privacy to the central server, and aggregates and updates the global defect identification model on the central server, and repeats the training step until the global step reaches 200 times to ensure that the model converges and reaches the training expectation;

[0010] S400: Each participating user downloads the updated global model from the central server for online detection of powder laying defects in the respective LPBF printing process.

[0011] In the laser powder bed fusion defect detection method based on privacy-enhanced federated learning, S100 comprises:

[0012] S101: The laser powder bed fusion printing process comprises powder spreading and melting alternately to manufacture a part, after each layer of powder laying process, before the melting process, an image acquisition device is used to obtain the powder bed image of the forming area;

[0013] S102: name and label the platform topography pattern of the forming area, use different color labels to label different defects and powder background of each powder bed image, and the label name is the same as the corresponding powder bed image;

[0014] S103: Each participating user constructs a training set, a validation set and a test set according to 8:1:1 from the powder bed image and the corresponding labeled image, which is used for subsequent local model training.

[0015] The laser powder bed melting defect detection method based on privacy-enhanced federated learning, wherein S200 comprises:

[0016] S201: an encoder, mainly used for capturing global context information of an image, comprising five convolutional blocks and four maximum pooling layers;

[0017] S202: a decoder, mainly used for gradually recovering the lost spatial information after downsampling, comprising four upsampling layers and four attention gate mechanisms, and finally adding a convolutional layer for adapting to the output channel number;

[0018] S203: an image with an input size of HxWx3 enters the model, wherein H and W are the length and width of the input image, respectively, HxWx64 is obtained through the first layer of convolution, and Similarly, five layers of convolution are performed to obtain Then, the first layer of upsampling is performed to obtain After four layers of upsampling, it is finally restored to HxWx64, and then restored to HxWx6 through 1x1 convolution;

[0019] S204: at the third layer of the encoder, the input is evenly divided into foreground and background parts by channel number, and a decoupling loss function is used to make the two parts tend to train corresponding features, and at the second layer of upsampling of the decoder, the two parts are merged by channel number for subsequent convolution;

[0020] The laser powder bed melting defect detection method based on privacy-enhanced federated learning, wherein S300 comprises:

[0021] S301: constructing a federated learning framework based on differential privacy: a trusted central end determines the privacy budget δ in differential privacy, the target indicators (mIoU and mDice) of the defect recognition model and the number of customers for joint training according to the privacy requirements and data characteristics of each participating user;

[0022] S302: defect recognition model initialization: each participating user prepares a local training data set and downloads a pre-trained defect recognition model from a central server;

[0023] S303: Local training of defect recognition model: Using AdamW optimizer and multi-loss descent method, each participating user uses the original graphics and annotation mask in the training set to perform local training of the local defect recognition model. After reaching the local training times, the gradient function in Pytorch is used to obtain the feature layer gradient information of the local defect recognition model;

[0024] S304: Adaptive differential privacy encryption: Each participating user decouples the features of the local model gradient information, separates the foreground and background features, and then adds less Gaussian noise to the foreground features and more Gaussian noise to the background features to improve the model prediction accuracy under the same privacy protection level;

[0025] S305: Gradient aggregation and global model update: After the central server receives the model gradient information encrypted by differential privacy from each participating user, the FedProx algorithm is used for weighted aggregation of the gradient information, and the global training and update of the defect recognition model on the central server are performed. After reaching the global training steps, the defect recognition model is distributed to each participating user;

[0026] S306: Variable learning rate and early stopping strategy: If the mIoU performance of the defect detection model on the validation set does not improve for multiple consecutive iteration periods, the early stopping strategy and the method of reducing the learning rate are used to deal with non-convergence to achieve the training expectation.

[0027] S307: Repeat steps S303 to S306 until the target indicators of the defect recognition model are reached under the premise of not exceeding the privacy budget δ, and obtain the global powder bed defect recognition model.

[0028] The global and local optimization functions of the adaptive differential privacy federated learning network in the laser powder bed fusion defect detection method based on privacy-enhanced federated learning are:

[0029]

[0030] where h m is the local optimization objective function of the mth client, is the weight and bias of the local defect detection model of the mth client in the tth global iteration, w t is the weight and bias of the global defect detection model in the tth global iteration, N represents the total number of images of all users, M represents the total number of participating clients, N m represents the number of images of user m, represents the loss function of the training model, μ is a penalty factor used to control the generation of local device divergence caused by heterogeneous defect data of powder laying images.

[0031] The AdamW optimizer decouples weight decay, making the regularization effect more stable and intuitive, preventing excessively large weight values, and helping to prevent model overfitting.

[0032] The laser powder bed fusion defect detection method comprises the following steps:

[0033] S3041: Adaptive gradient clipping: in the adaptive differential privacy federated learning network, the gradient clipping formula is as follows:

[0034]

[0035] Where g represents the gradient of the defect detection model, when the L2 norm of the gradient ||g||2 is less than or equal to C, the original gradient g is retained, and if ||g||2 is greater than C, g is clipped to C. When the gradient norm is less than the clipping factor, the standard deviation of the noise is multiplied by the L2 norm of the gradient instead of the clipping factor;

[0036] S3042: Gradient L2 sensitivity: in differential privacy, sensitivity measures the maximum influence of adding or deleting a single data point in the data set on the function output, and the L2 sensitivity Δ is defined as follows:

[0037]

[0038] Where g' represents the adjacent data set of the gradient information of the defect detection model, f(·) is a query function, and ||·||2 is an L2 norm;

[0039] S3043: Adaptive noise addition: if the original noise mechanism F(g) = f(g) + Z, satisfies (ε, δ)-DP, represents a Gaussian distribution, f(·) is a query function and its L2 sensitivity Δ2f≤1, then for any M-dimensional query function f'(·), if F'(g) = f'(g) + Z', Z' = (Z1,..., Z M ), satisfies F'(g) = f'(g) + Z' still satisfies (ε, δ)-DP, where N is a Gaussian noise, σ * is the standard deviation of the original noise, (Δ2f') i represents the L2 sensitivity of the i-th dimension of f', ε is the privacy budget, and δ is the relaxation factor, allowing the model to break the limit within a small range;

[0040] S3044: According to the importance of the foreground and background features of the powder bed pattern and S2043, the standard deviation of the gradient corresponding to the foreground features is Gaussian noise, the gradient of the background feature is added with a standard deviation of Where 0 < β < 1, β is the ratio of the standard deviation of the noise added to the foreground and background, Δ1 and Δ2 represent the sensitivity of the gradient information of the foreground and background parts respectively.

[0041] The laser powder bed melting defect detection method based on privacy-enhanced federated learning, S306 includes,

[0042] S3061: The variable learning rate method is ReduceLROnPlateau, which dynamically adjusts the learning rate according to the change of the validation set index. When the mIoU of the validation set does not improve significantly in a continuous number of training steps, the learning rate is automatically reduced by a certain proportion to make the model converge faster;

[0043] S3062: The early stopping strategy is that when the validation set index no longer improves in a continuous number of training steps, it is likely that the defect recognition model has not converged or has begun to overfit the training data, so the training is terminated to avoid invalid calculation.

[0044] In the laser powder bed melting defect detection method based on privacy-enhanced federated learning, in S300, each participating user uses a method of jointly descending multiple loss functions for model training, and the loss functions for realizing the classification task include cross-entropy loss, focal loss function and Des coefficient loss, and in S204, the loss function for realizing feature decoupling includes the following:

[0045] is the L2 norm loss function, u i is the i-th dimensional output component of the model background feature, L is the dimension of the output component, and the loss can make the background area of the model tend to capture the background features of the image;

[0046] is the cosine similarity loss, u is the output vector of the model background feature, v is the output vector of the model foreground feature, and ||·|| is the L2 norm, which can make the foreground area of the model tend to capture the foreground features of the image;

[0047] α, β, γ, μ, κ ≥ 0 are weight hyperparameters, and the total loss is obtained by linearly combining the loss functions, and the weight relationship between different loss functions is adjusted according to the experimental results to make the model achieve the best accuracy.

[0048] In the laser powder bed fusion defect detection method based on privacy-enhanced federated learning, the up-sampling and down-sampling parts of the feature decoupled loss function segmentation network adopt grouped convolution, and according to the number of channels, the feature maps are evenly divided into foreground and background two parts, and by adopting a multi-loss optimization strategy, each part is encouraged to learn the respective features.

[0049] A laser powder bed fusion defect detection device comprises,

[0050] A laser powder bed fusion module comprises,

[0051] A powder bed comprises,

[0052] A substrate on which powder is laid,

[0053] A scraper reciprocally levels the powder,

[0054] A supply bin provides powder,

[0055] A recycling bin contains remaining powder,

[0056] A laser source fuses the powder;

[0057] A laser powder bed fusion powder bed online detection system comprises,

[0058] An image acquisition device collects powder bed image data,

[0059] A federated learning module with adaptive differential privacy is connected to the image acquisition device to perform the laser powder bed fusion defect detection method.

[0060] In a laser powder bed fusion defect detection device, the image acquisition device comprises an industrial camera.

[0061] Compared with the prior art, the present application has the following advantages: the present application realizes effective decoupling of foreground-background features, so that the noise injection intensity can be dynamically adjusted according to the feature sensitivity. This adaptive noise allocation strategy can not only reduce the negative impact on model performance, but also ensure the same level of privacy protection. It provides a very promising solution framework for defect detection in LPBF printing tasks in key fields, while taking into account performance and privacy protection. It improves the convergence speed, alleviates the problem of data scarcity and enhances the privacy security of key fields, while taking into account performance and privacy protection, and provides a promising method for deploying LPBF technology in high security requirements working conditions. BRIEF DESCRIPTION OF DRAWINGS

[0062] Various other advantages and benefits of the present application will become apparent to those of ordinary skill in the art, upon reading the following detailed description of the preferred embodiment. The accompanying drawings are included to provide a better understanding of the preferred embodiment, and are not intended to constrain the application. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained from these drawings without creative labor for those of ordinary skill in the art. Moreover, the same reference numerals are used to represent the same components throughout the drawings.

[0063] In the drawings:

[0064] Figure 1 is a schematic diagram of a laser powder bed fusion defect detection device provided by an embodiment of the present disclosure, which moves upward in a cycle, the scraper spreads the metal powder uniformly to the forming bin from left to right, the industrial camera collects the powder bed image, then the laser source selects the melting, and finally the scraper scrapes the remaining powder to the recycling bin to complete a cycle of printing;

[0065] Figure 2 is a schematic diagram of the original powder bed image collected by the industrial camera in the LPBF printing process and the five typical defect images calibrated by an embodiment of the present disclosure;

[0066] Figure 3 is a schematic diagram of the Attention-Unet semantic segmentation model and the grouping convolution module used in the present disclosure to detect the LPBF powder spreading defects, the input original powder bed image is first subjected to convolution and maximum pooling operation, the feature decoupling is performed according to the input channel number at the third layer downsampling layer, and the average is divided into foreground and background parts, then the channel number is connected at the second layer upsampling layer for subsequent convolution operation, and the corresponding feature layers of upsampling and downsampling exist attention gate connection;

[0067] Figure 4 is a schematic diagram of the federated learning training based on differential privacy provided by an embodiment of the present disclosure, each participating client first completes local training, then uploads the encrypted gradient of the defect detection model to the central server instead of the original data, then performs gradient aggregation and update at the central end, and then issues to each client;

[0068] Figure 5 (a) in the foregoing, Figure 5 (b) in the foregoing is a comparison diagram of the performance evaluation indexes mIoU and mDice of the ADP-FL method in the training process and the effect of the federated learning and the federated learning based on differential privacy, Figure 5 (c) in the foregoing, Figure 5 (d) in the foregoing is a comparison diagram of the prediction performance indexes IoU of the defect detection model in the background and the five defects.

[0069] Figure 6 This is a flowchart illustrating a method provided in one embodiment of this disclosure.

[0070] The present invention will be further explained below with reference to the accompanying drawings and embodiments. Detailed Implementation

[0071] Specific embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While specific embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0072] It should be noted that certain terms are used in the specification and claims to refer to specific components. Those skilled in the art will understand that different terms may be used to refer to the same component. This specification and claims do not distinguish components based on differences in terminology, but rather on differences in function. The terms "comprising" or "including" used throughout the specification and claims are open-ended and should be interpreted as "comprising but not limited to." The following descriptions are preferred embodiments for carrying out the invention; however, these descriptions are for the purpose of understanding the general principles of the specification and are not intended to limit the scope of the invention. The scope of protection of this invention is determined by the appended claims.

[0073] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. The accompanying drawings do not constitute a limitation on the embodiments of the present invention.

[0074] like Figures 1 to 6 As shown, the laser powder bed melting defect detection method includes the following steps:

[0075] S100: During the laser powder bed fusion printing process, each participating user in the adaptive differential privacy federated learning acquires powder bed images, performs corresponding preprocessing and calibration on the image data, and constructs their own datasets.

[0076] S200: The defect detection model Attention-Unet was selected as the neural network for adaptive differential privacy federated learning. The network consists of an encoder, a decoder and skip connections. Grouped convolution was introduced on the basis of Attention-Unet to achieve feature decoupling of foreground and background information.

[0077] S300: After the loss function is constructed, each participating user first downloads the defect detection model from the central server, trains the defect recognition model locally using the local data set, then uploads the model gradient information processed by adaptive differential privacy to the central server, and aggregates and updates the global defect recognition model on the central server to obtain the global defect recognition model, and repeats the training step until the global step reaches 200 times to ensure that the model converges and reaches the training expectation;

[0078] S400: Each participating user downloads the updated global model from the central server for online detection of powder laying defects in the LPBF printing process.

[0079] In a preferred embodiment of the method, the LPBF manufacturing process in step S100 is mainly divided into powder spreading and melting stages, which are alternately performed to manufacture parts. The step of collecting powder bed image data using an image acquisition device includes:

[0080] S101: After each layer of powder laying process, before the melting process, an image acquisition device is used to obtain the powder bed image of the forming area and store it in the computer device;

[0081] S102: The platform topography pattern of the obtained forming area after powder laying is named and labeled, and different colors are used to label different defects and powder backgrounds in each powder bed image, and the label name should be the same as the corresponding image;

[0082] S103: Each participating user constructs a training set, a validation set and a test set according to 8:1:1 from the original image and the corresponding labeled image, which is used for subsequent local model training.

[0083] In a preferred embodiment of the method, the structure of the defect detection model Attention-Unet in S200 includes:

[0084] S201: The encoder is mainly used to capture the global context information of the image, which includes five convolutional blocks and four max-pooling layers;

[0085] S202: The decoder is mainly used to gradually recover the spatial information lost by downsampling, which includes four upsampling and four attention gate mechanisms, and finally adds a convolutional layer to adapt to the output channel number;

[0086] S203: The image with an input size of HxWx3 is input into the model, where H and W are the length and width of the input image, respectively. After the first convolution, HxWx64 is obtained, and after the second convolution, HxWx128 is obtained. Similarly, after five convolutions, HxWx512 is obtained. Then, after the first upsampling, Hx2Wx512 is obtained. After four up-sampling layers, it is finally recovered to HxWx64, and then recovered to HxWx6 through 1x1 convolution;

[0087] S204: At the third layer of the encoder, the input is evenly divided into foreground and background parts by channel number, and the two parts are made to tend to train the corresponding features using the decoupling loss function, and are merged by channel number at the second up-sampling layer of the decoder for subsequent convolution;

[0088] In a preferred embodiment of the method, the step of obtaining a global defect detection model by using federated learning with adaptive differential privacy in S300 comprises:

[0089] S301: Construct a federated learning framework based on differential privacy: a trusted central end determines the privacy budget δ in differential privacy, the target indicators (mIoU and mDice) of the defect recognition model, and the number of customers for joint training according to the privacy requirements and data characteristics of each participating user;

[0090] S302: Defect recognition model initialization: each participating user prepares a local training data set and downloads a pre-trained defect recognition model from the central server;

[0091] S303: Local training of defect recognition model: using the AdamW optimizer and the multi-Loss descent method, each participating user performs local training of the local defect recognition model using the original graphics and labeled masks in the training set. After reaching the local training times, the gradient function in Pytorch is used to obtain the feature layer gradient information of the local defect recognition model;

[0092] S304: Adaptive differential privacy encryption: each participating user decouples the features of the local model gradient information, separates the foreground and background features, and then adds less Gaussian noise to the foreground features and more Gaussian noise to the background features to improve the model prediction accuracy under the same privacy protection level;

[0093] S305: Gradient aggregation and global model update: after the central server receives the model gradient information encrypted by differential privacy uploaded by each participating user, the FedProx algorithm is used for weighted aggregation of the gradient information, and the global defect recognition model is trained and updated on the central server. After reaching the global training steps, the defect recognition model is distributed to each participating user;

[0094] S306: Use variable learning rate and early stopping strategy: if the mIoU performance of the defect detection model on the validation set does not improve for multiple consecutive iteration periods, use the early stopping strategy and reduce the learning rate to deal with non-convergence to achieve the training expectation.

[0095] S307: repeat S303 to S306 until the target index of the defect identification model is reached under the premise that the privacy budget is not exceeded, and a global powder bed defect identification model is obtained.

[0096] The global and local optimization functions of the adaptive differential privacy federated learning network in the laser powder bed fusion defect detection method are:

[0097]

[0098] where h m is the local optimization objective function of the mth client, is the weight and bias of the local defect detection model of the mth client in the tth global iteration, w t is the weight and bias of the global defect detection model in the tth global iteration, N represents the total number of images of all users, M represents the total number of participating clients, N m represents the number of images of user m, represents the loss function of the training model, and mu is a penalty factor for controlling the generation of local device divergence caused by heterogeneous defect data of powder laying images.

[0099] In the preferred embodiment of the method, in S201, the defect identification model is an Attention-Unet semantic segmentation model, which introduces an attention mechanism for processing segmentation tasks with complex background or unbalanced class distribution.

[0100] In the preferred embodiment of the method, S304 includes,

[0101] S3041: adaptive gradient clipping: in the adaptive differential privacy federated learning network, the gradient clipping formula is as follows:

[0102]

[0103] When the L2 norm of the gradient ||g||2 is less than or equal to C, the original gradient g is retained, and if ||g||2 is greater than C, g is clipped to the norm of C. When the gradient norm is less than the clipping factor, the standard deviation of the noise is multiplied by the L2 norm of the gradient instead of the clipping factor;

[0104] S3042: gradient L2 sensitivity: in differential privacy, sensitivity measures the maximum influence of adding or deleting a single data point in the data set on the function output, and the L2 sensitivity Delta is defined as follows:

[0105]

[0106] where g' represents the adjacent data set of the gradient information of the defect detection model, f(·) is a query function, and ||·||2 is an L2 norm;

[0107] S3043: Adaptive noise adding: if the original noise mechanism F(g) = f(g) + Z, satisfies (ε, δ)-DP, where represents a Gaussian distribution, f(·) is a query function and its L2 sensitivity Δ2f≤1, then for any M-dimensional query function f'(·), if F'(g) = f'(g) + Z', Z' = (Z1,..., Z M ), satisfies F'(g) = f'(g) + Z' still satisfies (ε, δ)-DP, where N is a Gaussian noise, σ * is the standard deviation of the original noise, (Δ2f') i represents the L2 sensitivity of the i-th dimension of f', ε is the privacy budget, δ is the relaxation factor, and the model is allowed to break the limit within a very small range;

[0108] S3044: According to the importance of the foreground and background features of the powder bed pattern and S2043, Gaussian noise with a standard deviation of is added to the gradient corresponding to the foreground feature, and Gaussian noise with a standard deviation of is added to the gradient corresponding to the background feature, where 0 < β < 1, β is the ratio of the standard deviations of the noise added to the foreground and background, Δ1 and Δ2 represent the sensitivities of the gradient information of the foreground and background parts, respectively.

[0109] In a preferred embodiment of the method, in S300, each participating user uses a method of jointly descending multiple loss functions for model training, the loss function for realizing the classification task includes cross-entropy loss, focal loss function and Des coefficient loss, and the loss function for realizing feature decoupling includes the following:

[0110] is an L2 norm loss function, u i is the i-th dimensional output component of the background feature of the model, L is the dimension of the output component, and the loss can make the background area of the model tend to capture the background features of the image;

[0111] is a cosine similarity loss, u is the output vector of the background feature of the model, v is the output vector of the foreground feature of the model, and ||·|| is an L2 norm. The loss can make the foreground area of the model tend to capture the foreground features of the image;

[0112] α, β, γ, μ, κ ≥ 0 are weight hyperparameters, the total loss is obtained by linearly combining the loss functions, and the weight relationship between different loss functions is adjusted according to the experimental results to make the model achieve the best precision.

[0113] In the preferred embodiment of the laser powder bed fusion defect detection method, the up-sampling and down-sampling parts of the feature decoupled loss function segmentation network use grouped convolution, according to the number of channels, the feature map is divided into foreground and background two parts, by using the multi-loss optimization strategy, each part is encouraged to learn the respective features.

[0114] A laser powder bed fusion defect detection device comprises,

[0115] A laser powder bed fusion module comprises,

[0116] A powder bed comprises,

[0117] A substrate on which the powder is laid,

[0118] A scraper reciprocally levels the powder,

[0119] A supply bin provides the powder,

[0120] A recycling bin contains the remaining powder,

[0121] A laser source fuses the powder;

[0122] A laser powder bed fusion powder bed online detection system comprises,

[0123] An image acquisition device collects powder bed image data,

[0124] A federated learning module with adaptive differential privacy is connected to the image acquisition device to perform the laser powder bed fusion defect detection method of any one of claims 1-10.

[0125] In a preferred embodiment of the laser powder bed fusion defect detection device, the image acquisition device comprises an industrial camera.

[0126] In one embodiment, the laser powder bed fusion defect detection method comprises the following steps:

[0127] S100: Under the differential privacy-based federated learning framework, each participant needs to collect powder bed images during the LPBF printing process with the help of an industrial camera, and use a monitoring device to export data, then pre-process, name and calibrate the images, make and save the corresponding data set;

[0128] S200: After the preparation of the local dataset is completed, the global defect detection model is downloaded from the central server under the training framework of federated learning. Then the loss function is constructed, each party trains the model using the local dataset, and uploads the model gradient processed by adaptive differential privacy to the central server to realize the iterative update of the global model. The training is repeated until the predetermined performance and privacy protection requirements are met;

[0129] S300: Each participant then downloads the updated global model from the central server and uses the optimal model to detect the powder laying defects occurring in the LPBF printing process online.

[0130] Preferably,

[0131] In step S100, LPBF is one of the most typical processes in the actual production application of additive manufacturing technology, which melts and sinters metal powder by controlling high-energy laser beams according to the specified path. In this process, the left powder supply bin rises to a fixed height, and then the scraper evenly spreads the powder from left to right to the forming lever. The excess powder will be swept to the excess material lever. Then laser selective scanning is performed, and after scanning, the back knife is used for the next cycle. Since LPBF operates in a closed and controlled industrial environment, key data can be encrypted to minimize potential security vulnerabilities.

[0132] In step S100, the user collects the powder bed image from the industrial camera and calibrates it. The steps for making the corresponding dataset are as follows:

[0133] S101: During the printing process, after each layer of powder is laid, the laser melts, and an industrial camera is used to collect the powder bed image of the forming area before laser melting and is named according to the printing layer number (example, '0.png'). The image is saved to the corresponding folder;

[0134] S102: After collecting the original image, the resolution size of the image is adjusted to 1024x1024 pixels, and then labelme is used to calibrate the powder bed image. Then the powder bed is kept black (0, 0, 0), the warping is yellow (255, 255, 0), the incomplete powder laying is red (255, 0, 0), the vibration is green (0, 255, 0), the dragging is cyan (0, 255, 255), and the dot matrix is blue (0, 0, 255). After calibration, the corresponding original image and the corresponding label image are placed in the image folder and the label folder respectively, and the name of the calibration image is the same as that of the corresponding original image. Finally, the image is shuffled and rearranged, and then the original dataset is divided into training set, test set and validation set according to the ratio of 8:1:1.

[0135] Preferably,

[0136] In step S200, the defect detection model is Attention-Unet, which is an improved model based on the classic U-Net and mainly used for semantic segmentation tasks. Its structure maintains the encoder-decoder form of U-Net, extracts multi-scale features layer by layer in the encoder, restores the spatial resolution by step-up sampling in the decoder, and fuses the features on the encoder side. The attention gate is used to filter and emphasize the features that are more meaningful to the current segmentation task in the skip connection, thereby reducing the interference of noise and irrelevant information on the decoder.

[0137] The Attention-Unet mainly consists of an encoder and a decoder. The encoder includes five convolutional blocks and four max-pooling layers, with a convolution kernel size of 3x3 and a step size of 1. After each convolution, batch normalization and an activation function (ReLU) are added. The max-pooling layer has a size of 2x2 and a step size of 2. The decoder includes four up-sampling layers and four attention gate mechanisms. A final convolutional layer is added to adapt the output channel number. The transpose convolution uses a 3x3 kernel with a step size of 1. The attention gate mechanism first maps both inputs to the same dimension through a 1x1 convolution. After addition, ReLU activation, another 1x1 convolution, and Sigmoid activation, the attention coefficients are obtained. Then, the corresponding inputs are added element-wise.

[0138] The input size of the powder bed image is HxWx3, which enters the model and goes through the first layer of convolution to get HxWx64, and the second layer of convolution to get Similarly, five layers of convolution are performed to get Then, the first layer of up-sampling is performed to get After four layers of up-sampling, it is finally restored to HxWx64, and then restored to HxWx6 through a 1x1 convolution. To implement adaptive differential privacy noise addition, the defect detection model needs to be divided into foreground and background parts. Therefore, at the third layer of the encoder, the channel number is evenly divided into foreground and background parts, and the loss function is used to make the two parts tend to train the corresponding features. At the second layer of up-sampling in the decoder, the channel number is combined to implement grouped convolution.

[0139] Preferably,

[0140] In step S200, federated learning is a new distributed machine learning method that trains models on multiple devices or servers and shares model parameters instead of raw data to protect privacy and security, which is suitable for industrial scenarios where data is sensitive but still needs to be jointly modeled. However, relying solely on federated learning is not enough to ensure sensitive information in critical areas. Using differential privacy can hide input and output by designing encryption protocols to protect privacy and security, while using foreground and background feature decoupling methods for adaptive noise addition to improve model accuracy.

[0141] The steps of federated learning are:

[0142] S201: initializing a model: after each local participant prepares a training data set and signs a joint training and privacy protection agreement, the central server initializes and pre-trains a global defect detection model, and then the local user downloads the global model from the central server to the local;

[0143] S202: local model training: each local participant uses local data to train the global defect detection model for a certain number of rounds, and uses gradient descent method according to the loss function during training. After training, the client does not directly upload the local data, so as to protect the data privacy;

[0144] S203: adaptive differential privacy encryption: each participant user decouples the feature of the local model gradient information, separates the foreground and background features, and then adds less Gaussian noise to the foreground feature and more Gaussian noise to the background feature, so as to improve the model prediction accuracy under the same privacy protection degree;

[0145] S204: gradient aggregation and global model update: after the central server receives the model gradient information encrypted by differential privacy from each participant user, the gradient information is aggregated according to the pre-set gradient weighting strategy, and the global model training and update on the central server are carried out, and then the defect identification model is distributed to each participant user;

[0146] S205: variable learning rate and early stopping strategy: if the mIoU performance of the defect detection model on the validation set does not improve for continuous multiple iteration periods, the early stopping strategy and the method of reducing the learning rate are used to deal with non-convergence to achieve the expected training.

[0147] S206: repeat steps S202 to S205 until the target indicators of the model performance are reached under the premise of not exceeding the privacy budget, and obtain the global powder bed defect identification model.

[0148] Preferably,

[0149] After the global iteration is completed, the optimal LPBF powder defect detection model under a certain privacy budget is obtained, and each participant can download the powder bed defect online identification for the printing process of the local device from the central server as needed.

[0150] Preferably,

[0151] The core idea of federated learning in step S201 is to average the local model weights and send them to the global model for updating. The method used is FedProx. Specifically, FedProx adds a proximal term to the loss function on the client side compared to the federated averaging algorithm, which limits the difference between local updates and the global model, thereby accelerating the convergence speed of the model and reducing the influence of heterogeneity between clients on the convergence of the global model. The global and local optimization functions are:

[0152]

[0153] where h m is the local optimization objective function of the mth client, is the weight and bias of the local defect detection model of the mth client in the tth global iteration, w t is the weight and bias of the global defect detection model in the tth global iteration, N represents the total number of images of all users, M represents the total number of participating clients, N m represents the number of images of user m, represents the loss function of the training model, and μ is a penalty factor used to control the divergence of local devices caused by heterogeneous defect data in the powder image.

[0154] Preferably,

[0155] In step S202, after each participating user downloads the global model from the central server, in order to solve the class imbalance problem between defects in the LPBF process, improve the accuracy of small target detection, and effectively separate foreground and background features, a method of jointly descending multiple loss functions is used for model training. The loss function for the classification task is: (1) Cross-Entropy Loss function is a commonly used loss function, which is used to measure the difference between two probability distributions; (2) Focal Loss is introduced to handle the severe imbalance between positive and negative samples in object detection tasks. When the number of negative samples is much larger than the number of positive samples, the learning process will be overwhelmed by negative samples that are easy to classify, thereby reducing the overall classification performance. Therefore, Focal Loss performs better when detecting small-scale defects in powder bed images; (3) Dice Loss measures similarity by calculating the intersection ratio and union ratio of predicted labels and true labels, which can optimize the area overlap of segmentation results, especially for small defect targets, and enhance the model's ability to detect details.

[0156] The loss function for feature decoupling includes: (1) L2Loss:

[0157]

[0158] is the L2-norm loss function, u is the output vector of the model background feature, v is the output vector of the model foreground feature, and ||·|| is the L2-norm, which can make the foreground area of the model tend to capture the foreground features of the image, and by adjusting the weight assigned to each loss function, the best training effect can be obtained. i is the i-th dimensional output component of the model background feature, L is the dimension of the output component, and the loss can make the background area of the model tend to capture the background features of the image; (2) Cosine-Similarity Loss:

[0159]

[0160] is the Cosine-Similarity Loss, u is the output vector of the model background feature, v is the output vector of the model foreground feature, and ||·|| is the L2-norm, which can make the foreground area of the model tend to capture the foreground features of the image, and by adjusting the weight assigned to each loss function, the best training effect can be obtained.

[0161]

[0162] α, β, γ, μ, κ are weight hyperparameters and α, β, γ, μ, κ ≥ 0, there is a certain trade-off between different loss functions, which can be adjusted according to the experimental results to make the total loss reach the best precision in the minimum time close to the model performance, thereby balancing the training and performance.

[0163] Preferably,

[0164] In step S203, the differential privacy noise adding framework includes: (1) adaptive gradient clipping: in federated learning with differential privacy, the main purpose of gradient clipping is not only to stabilize training, but also to adjust the size of the added noise to improve the performance of the model. The clipping formula is as follows:

[0165]

[0166] When the L2-norm of the gradient ||g||2≤C, the original gradient is retained, and if ||g||2>C, the original gradient g is clipped to the norm of C. (2) Adaptive noise adding: decoupling noise is added according to the importance of the foreground and background features of the powder bed image, Gaussian noise with a standard deviation of is added to the gradient corresponding to the foreground feature, and Gaussian noise with a standard deviation of is added to the gradient corresponding to the background feature, where 0<β<1, Δ1 and Δ2 represent the sensitivity of the gradient information of the foreground and background parts, respectively.

[0167] In one embodiment, as shown in Figure 1 the present disclosure provides a federated learning based on adaptive differential privacy for LPBF powder laying defect detection, comprising the following steps:

[0168] S100: In the LPBF printing process, each user participating in federated learning based on differential privacy needs to use an industrial camera to collect powder bed images and use a monitoring device to export and pre-process the images accordingly.

[0169] In this embodiment, as shown in FIG. 1, a data collection device (such as an industrial camera) is used in the LPBF printing process to capture powder bed images after powder laying is completed and before laser action. Image processing methods such as resizing and labeling are used to create a data set. Figure 2

[0170] Different participating users have different printing equipment, working conditions, and monitoring equipment, so the final powder laying data set has characteristics such as unbalanced defective data and heterogeneity, which makes the final model performance poor. Therefore, one of the purposes of this disclosure is to solve the problem of data distribution deviation and difficulty in sharing of each participating user.

[0171] S200: Construct a semantic segmentation model for detecting powder laying defects on a central server and pre-train it. Each participating user first downloads the global defect detection model from the central server, uses their own local data set for local training and optimization, uploads the model gradient information processed by adaptive differential privacy to the central server, and performs global model update on the central server until the model performance and privacy protection requirements are met.

[0172] In this step, the advantages of both federated learning and differential privacy are combined. For those skilled in the art, federated learning first trains on local devices to obtain a local model. Then, distributed model weights from different devices or servers are aggregated on a central server to achieve global model training while effectively protecting data privacy. For manufacturing fields with high privacy requirements, differential privacy protects individual privacy by adding a controllable amount of noise to the calculation results, ensuring that the contribution of any single data point is indistinguishable. According to the definition of differential privacy, analysis of the database and subsequent integration with other information will not expose individual privacy. It is worth noting that the differential privacy mechanism is not affected by post-processing and is not damaged by any algorithm.

[0173] ​However, directly applying differential privacy will lead to a significant decrease in accuracy. This decrease is mainly due to the fact that the uniform noise addition strategy does not take into account the characteristics of the powder bed data in LPBF. Specifically, differential privacy is usually achieved by injecting noise into the data, while in LPBF, noise must be introduced at each layer of the gradient. This process can severely affect the accuracy, especially for defect categories represented by only a few pixels, which is contrary to the ultimate goal of defect detection. To solve this problem, our method proposes an adaptive noise addition strategy for foreground-background imbalance in LPBF. By dynamically allocating noise, this method can both ensure equal privacy protection and reduce the negative impact on model performance, thereby improving the accuracy of defect detection.

[0174] S300: Each participating user downloads the updated global model from the central server for online detection of powder laying defects in their respective LPBF printing process;

[0175] In this step, after the global optimal model is trained by the central server, each participating user can download the model to the local. When using their respective LPBF equipment to process parts, the optimal model is used to identify powder laying defects in the printing process in real time, realizing online detection of powder laying quality process.

[0176] As shown in Figure 1 , each participating user performs the LPBF printing process, collects powder laying defect patterns and makes corresponding data sets, including the following steps:

[0177] S101: After the end of each layer of powder laying process and before the melting process, use the image acquisition device to obtain the powder bed image of the forming area and store it in the computer device, and name and save it according to the format of acquisition time and layer number (e.g. "1.png"), and then perform laser selective melting;

[0178] S102: After each layer of melting process and before powder laying process, the feed bin is raised to a certain height, then the scraper evenly spreads the powder from left to right to the forming lever, and the excess powder will be swept to the excess material lever;

[0179] S103: Name and label the obtained powder laying forming area platform topography pattern, use different color labels to label different defects and powder background of each powder bed image, and the label name should be the same as the corresponding image (e.g. "1.png") saved in the special folder;

[0180] The data set of the embodiment is obtained by printing AlSi10Mg using an easyplus three-dimensional printer, including representative defect images derived by monitoring equipment. A total of 406 powder laying images were collected, the resolution of which was adjusted to 1024x1024 pixels, and then the powder bed image was labeled for defects using Labelme: the powder bed was labeled as black (0, 0, 0), the warping defect was labeled as yellow (255, 255, 0), the incomplete powder laying was mapped as red (255, 0, 0), the vibration defect was labeled as green (0, 255, 0), the dragging defect was labeled as cyan (0, 255, 255), and the dot matrix defect was labeled as blue (0, 0, 255). After labeling, the original image and the corresponding label image were saved to folders named image and label, respectively. The images in the image folder were rearranged, and the original data set was divided into a training set, a test set and a validation set in a ratio of 8:1:1. Defect annotation supervised training was performed. One powder bed image may contain multiple defect types. The statistical information of the powder laying defect data set is shown in Table 1:

[0181] Table 1 Number of powder laying defect images

[0182] Class Warping Incomplete powdering Shake marks Dragging Dot matrix Number 377 52 193 229 134

[0183] To simulate the local data set in FL, the training set images were further divided into four clients, and each client received 81 images. The validation set and the test set were each allocated 41 images.

[0184] S201: In constructing the federated learning framework based on differential privacy, first of all, the trusted central end collects the privacy requirements and data characteristics of each participating user, and analyzes the type, distribution and sensitivity of the data in detail. Based on this information, the central end formulates a joint training framework and a privacy protection plan that can both accommodate the characteristics of diversified data sources and meet the privacy requirements of each user. Specifically, the plan introduces a differential privacy mechanism while jointly training, protects the original data by adding appropriate noise to prevent sensitive information from being leaked. Subsequently, the central server designs and determines the joint training model based on the privacy-processed data uploaded by each participating node, continuously optimizes the model parameters, and ensures a balance between overall performance and individual privacy protection. The entire framework realizes collaborative learning across multiple data sources while effectively reducing privacy risks, and builds a safe and efficient federated learning environment;

[0185] S202: Model initialization: each participating user prepares the local training data set, and downloads the pre-trained defect recognition model from the central server;

[0186] In this step, as Figure 3As shown, the significance segmentation network for powder laying defect detection needs to be able to identify tiny targets and complex shape details. In this example, an Attention-Unet model is used, which introduces an attention gate mechanism to dynamically focus on important areas. In order to decouple the foreground and background feature information for subsequent separate noise addition, grouping convolution is used. The core idea is to divide the input channels into multiple groups, and each group is only connected to the corresponding partial filter (convolution kernel). In this way, the channels of different groups are independent of each other, providing convenience for subsequent independent noise addition operations, reducing the parameters that each group needs to learn, and significantly reducing the computational complexity of the model while maintaining similar performance to the original model. In this example, grouping convolution is used in both the upsampling and downsampling processes, and the feature maps are divided into foreground and background parts according to the number of channels for subsequent convolution operations.

[0187] S203: Model local training: each participating user trains the model locally using the training set, and obtains the feature layer gradient information of the local model after reaching the local training times;

[0188] In this step, in order to train the semantic segmentation model for defect detection, it is necessary to set the powder laying defect segmentation performance evaluation index. In this example, IoU and Dice are selected, and their definitions are as follows:

[0189]

[0190] where TP represents the true positive rate predicted by the model, FN represents the false negative rate predicted by the model, and FP represents the false positive rate predicted by the model. Actual segmentation performance indicators include but are not limited to these two.

[0191] The defect segmentation task of the powder bed is a multi-classification problem. In view of the class imbalance problem and the decoupling of foreground and background features, each participating user uses a method of jointly descending multiple loss functions to train the model. The loss function for implementing the classification task is:

[0192] (1) Cross-Entropy Loss as the basic classification loss function, effectively drives the parameter optimization of the classification model by quantifying the KL divergence between the model prediction probability distribution and the true label distribution; (2) Focal Loss is introduced to address the class imbalance problem in object detection tasks. By introducing an adjustable focus factor, it dynamically reduces the loss weight of easy-to-classify samples (especially the large proportion of negative samples), thereby alleviating the problem of model training being dominated by a large number of simple negative samples. This feature makes it outstanding in micro-defect detection in powder bed fusion and other additive manufacturing processes, and can significantly improve the representation ability of sub-millimeter-level pores, cracks, and other small-scale abnormalities; (3) Dice Loss is designed based on the Intersection over Union (IoU) of the predicted mask and the true label. This function directly optimizes the overlap of the segmentation region and performs outstandingly in small target segmentation scenarios such as microscopic image analysis and pathological section detection, especially when dealing with weak edge features and low-contrast defects, it can effectively enhance the model's ability to capture geometric details. The core difference between the three types of loss functions is that Cross-Entropy Loss focuses on pixel-level probability calibration, Focal Loss emphasizes difficult sample learning, and Dice Loss focuses on macro-region consistency optimization.

[0193] The loss function that realizes feature decoupling is: (1) L2 Loss:

[0194]

[0195] L2 Loss, u i is the i-th dimensional output component of the model background feature, L is the dimension of the output component, and this loss can make the background area of the model tend to capture the background features of the image; (2) Cosine-Similarity Loss:

[0196]

[0197] Cosine-Similarity Loss, u is the output vector of the model background feature, v is the output vector of the model foreground feature, and ||·|| is the L2 norm. This loss can make the foreground area of the model tend to capture the foreground features of the image.

[0198]

[0199] α, β, γ, μ, κ are weight hyperparameters, and there is a certain trade-off between different loss functions. They can be adjusted according to experimental results to make the total loss reach the best accuracy in the minimum time close to the model performance.

[0200] S204: Adaptive differential privacy encryption: Each participating user decouples the local model gradient information by feature, separates the foreground and background features, and then adds less Gaussian noise to the foreground features and more Gaussian noise to the background features to improve the model prediction accuracy under the same privacy protection level;

[0201] In this step, as shown in Figure 4 , the laser powder bed data is protected by privacy through an adaptive differential privacy method. The private data of the central differential privacy client can still be leaked, so noise needs to be added to the local model. The main process is roughly divided into four steps: encoding, perturbation, aggregation, and analysis. First, the client data is processed using the local differential privacy mechanism in the encoding process, then noise is added to the encoded data, then it is sent to the server side to aggregate the perturbed data of different users, and finally the training results of the model are statistically analyzed. The definition of differential privacy is: for adjacent data sets g and g' (obtained by adding or removing some data points, the object of this patent is the gradient information of the model), under a certain degree of privacy leakage probability δ, it is considered that the data set conforms to the differential privacy if it meets the following random algorithm M:

[0202] Pr[F(g)∈S]≤e ε Pr[F(g′)∈S]+δ

[0203] Where ε is called the privacy parameter, which is used to measure the "amount of privacy" provided. A smaller ε means that very similar inputs need to be provided to produce very similar outputs, thus providing a higher level of privacy. A larger ε allows for less similar outputs, thus providing less privacy. And δ is used to limit the probability of model change, called the relaxation factor, which allows the model to break the limit within a small range. S is the output space of the mechanism F(·), and Pr[·] represents the output probability. The noise adding mechanism adopted by the present disclosure is the Gaussian mechanism, and the noise adding object is the model gradient. Although the accuracy of the Gaussian mechanism is not as good as that of the Laplace mechanism, the L2 sensitivity in powder bed image processing is much lower than the L1 sensitivity, so the noise added by the Gaussian mechanism is much smaller. The definition of the Gaussian mechanism based on federated learning is as follows:

[0204]

[0205] Where f(·) is the query function, Δ2f is the L2 sensitivity of f(·), represents the sampling result of the Gaussian distribution with mean 0 and variance σ 2 .

[0206] Adaptive noise adding includes: (1) adaptive gradient clipping: first, gradient clipping limits the maximum norm of the gradient to ensure stable gradient update and help the model converge, and the clipping formula is as follows:

[0207]

[0208] When the L2 norm of the gradient ||g||2≤C, the original gradient g is retained, and if ||g||2>C, the original gradient g is clipped to the norm of C. In federated learning with differential privacy, the purpose of gradient clipping is not only to stabilize training, but also to control the noise addition scale to improve model performance, and to control the contribution of individual clients to the global model to avoid privacy leakage. Gradient clipping is generally performed after local update of each client rather than on the global model. By clipping the local gradient of each client, the influence of each client on the global model update can be controlled to prevent some gradients from leaking too much individual information. Gradient clipping together with subsequent noise addition forms a mechanism to protect user data privacy. To evaluate the maximum impact on the original data and analysis results after adding noise, the sensitivity needs to be defined. In differential privacy, sensitivity measures the maximum impact of adding or removing a single data point on the function output. The definition of L2 sensitivity is:

[0209]

[0210] g and g' are adjacent data sets, and sensitivity determines the scale of noise. The higher the sensitivity, the greater the influence of a single data point, and therefore more noise needs to be added to ensure privacy. Therefore, the level of sensitivity directly affects the size of the noise. According to the training characteristics of the semantic segmentation model, as the global step increases, the L2 norm of the model gradient decreases, the influence of noise increases, and the convergence of the model is affected, resulting in excessive addition of noise and waste. Therefore, when the gradient norm is less than the clipping factor, the standard deviation of the noise is multiplied by the L2 norm of the gradient to achieve an adaptive gradient clipping noise addition method to improve the accuracy of the model.

[0211] (2) Adaptive noise addition: In LPBF powder laying defect detection, the importance difference of feature regions makes the adaptive noise addition method particularly critical. Compared to the traditional uniform noise addition method, the present disclosure adds noise by decoupling foreground and background, which can effectively balance privacy protection and model performance, enabling the model to accurately capture defect features under privacy protection conditions, thereby improving the robustness and accuracy of defect detection. To calculate the overall privacy protection level in real time, the present disclosure uses a differential privacy framework based on Rényi divergence, providing a more flexible privacy control method. For a given probability distribution F, the αth order Rényi divergence is defined as:

[0212]

[0213] α is the divergence order, which determines the sensitivity and strictness of the privacy measure. The definition of Rényi differential privacy (α, ε)-RDP is as follows: for adjacent data sets g and g', if the output F(·) satisfies:

[0214] D α (F(g)||F(g′))≤ε(α)

[0215] then it meets (α, ε)-RDP. If ||f(·)||2≤1, the Gaussian distribution is brought into the Rényi divergence to obtain the expression of the privacy budget:

[0216]

[0217] The serial composition mechanism of differential privacy gives the total privacy consumption when publishing multiple results protected by the differential privacy mechanism on the same input data. If F1(x) satisfies (ε1, δ1)-DP and F2(x) satisfies (ε2, δ2)-DP, then the mechanism G(x) = (F1(x), F2(x)) of publishing these two results also satisfies differential privacy (ε1+ε2, δ1+δ2)-DP. It can be seen that the privacy budget is also linearly increased, and the serial composition of differential privacy is the cornerstone of adaptivity.

[0218] According to the above theorem, if the original noise mechanism F(g) = f(g) + Z, satisfies (ε, δ)-DP, f(·) is a query function and its L2 sensitivity Δ2f≤1, then for any M-dimensional query function f'(·), if F'(g) = f'(g) + Z', Z' = (Z1,..., Z M ), satisfies then F'(g) = f'(g) + Z' still satisfies (ε, δ)-DP, where N is the Gaussian noise, σ * is the standard deviation of the original noise, (Δ2f′) i represents the L2 sensitivity of the i-th dimension of f', ε is the privacy budget, and δ is the relaxation factor, which allows the model to break the limit within a small range.

[0219] In the feature decoupling adaptive noise addition task of powder images, the foreground contains most of the defect semantic information and a small amount of background information related to the powder bed. On the contrary, the background contains limited defect-related information but includes a large amount of powder bed background information. The foreground information mainly contains important information about part defects, so it is hoped to inject less noise, while the background information is relatively less valuable, so more noise needs to be injected to ensure that privacy is fully protected. Assuming that the noise added to the foreground is σ1, the noise added to the background is σ2, and the original noise is σ *, σ1= βσ2, 0 < β < 1 can be obtained:

[0220]

[0221] S205: Gradient aggregation and global model update: After the central server receives the model gradient information uploaded by each participating user, the gradient information is aggregated according to the pre-set gradient weighting strategy, and the global model training and updating on the central server are performed, and after completion, the defect recognition model is distributed to each participating user;

[0222] In S205 of this step, after the central server receives the model gradient information uploaded by each user, aggregation and updating are performed, and weighted averaging is performed using the FedProx framework. Compared with the federated averaging algorithm, a proximal term is added to the loss function on the client side, which limits the difference between local update and global model, thereby accelerating the convergence speed of the model and reducing the influence of heterogeneity between clients on the convergence of the global model. The global and local optimization functions are:

[0223]

[0224] where h(·) is the local optimization objective function, is the weight and bias of the defect detection model of the mth client in the tth global iteration, h represents the local optimization objective function, N represents the total number of image quantities of all users, N m represents the number of images of user m, represents the loss function of the training model, and μ is the penalty factor.

[0225] In order to verify the effectiveness of the framework in actual industrial production conditions, the powder laying defect image dataset is used for experimental verification. The hyperparameters of the defect detection model trained by the federated learning framework based on adaptive differential privacy proposed by the present disclosure are as follows:

[0226] Table 2 Hyperparameters of model training

[0227] Model parameters Training hyperparameters Model name Attention-Unet Optimizer AdamW Input size 1024 Learning rate 2e-4 Number of clients 4 Local steps 3 Batch size 2 Global steps 200 Gradient clipping 1 Weight decay 1e-4 Noise scale 0.02 Momentum 0.8 Adaptive factor 0.3 L2Loss weight 1e-4 CosLoss weight 0.4

[0228] To verify the generality of the proposed ADP-FL framework in LPBF powder bed defect detection, this study compared four methods: centralized learning (CL), federated learning (FL), federated learning with differential privacy (DP-FL), and ADP-FL. CL involves storing and training all data centrally. This approach usually achieves the best performance because the model has direct access to the entire training dataset, but it requires centralized storage of data, which poses a huge challenge to data privacy and security. In the field of LPBF manufacturing, powder bed data often represents the core business secrets of the industrial production process. Therefore, centralized storage poses a great risk of privacy leakage, so it is necessary to use the DP-FL method for model training. And in order to ensure the fairness and effectiveness of the experiment, the same hyperparameter settings were used for all methods except the independent variable.

[0229] To demonstrate the effectiveness of Attention-Unet in LPBF powder bed defect detection and the generality of the adaptive method in other segmentation networks. Therefore, four networks (Unet, Unet++, Attention-Unet, RCNN-Unet) were selected, and the adaptive framework was added to each. Gaussian noise was added and the noise intensity was ensured to be consistent to observe the experimental results.

[0230] Table 3 Powder bed defect segmentation result evaluation table

[0231] Metrics Unet Unet++ RCNN-Unet Attention-Unet mIoU 0.5961±0.0005 0.6518±0.0010 0.6663±0.0013 0.6854±0.0034 mDice 0.6826±0.0008 0.7718±0.0014 0.7852±0.0017 0.7982±0.0031

[0232] As can be seen from the above table, the performance of Attention-Unet is particularly outstanding, with mIoU and mDice higher than other networks. This is mainly due to its attention mechanism, which can focus well on the defect area and reduce the impact of noise on model defect recognition.

[0233] The training results of the four frameworks are as follows Figure 5As shown in Table 3, among all evaluation indicators, the performance of CL is the best, with the fastest convergence speed and the highest final stable mIoU and mDice scores. This is because centralized learning can access the complete dataset, ensuring more uniform data distribution. FL performs slightly worse, with slightly lower final stable values than CL, because the training data of different clients is different. However, since no raw data is exchanged, FL can provide stronger privacy protection. For DP-FL, the introduction of Gaussian differential privacy noise enhances privacy protection but reduces the accuracy of model updates, leading to further performance degradation. The foreground-background adaptive noise injection method we propose effectively alleviates the performance degradation caused by noise by dynamically adjusting the noise level. Therefore, the accuracy and dice coefficient of the final model both exceed DP-FL, while retaining strong privacy protection capabilities. Adding noise inevitably leads to some degree of performance degradation; however, through adaptive noise injection, the model performance is significantly improved, with an increase of about 2.4% in mIoU and about 1.9% in mDice. In addition, the loss value of adaptive noise injection is still lower than that of fixed noise injection during most of the training process, which proves its effectiveness.

[0234] Table 4 Privacy budget and performance indicators of four training methods

[0235] Method σ ε δ mIoU mDice CL - - - 0.7906±0.0006 0.8739±0.0003 FL - - - 0.7645±0.0011 0.8567±0.0014 DP-FL 0.02 1401.28 1e-5 0.6854±0.0034 0.7982±0.0031 ADP-FL 0.02 1401.28 1e-5 0.7094±0.0039 0.8173±0.0025

[0236] In actual LPBF engineering applications, the steps for using the method proposed in the present disclosure are as follows. First, each participating user clearly defines their privacy requirements and prepares the local powder image dataset, and then signs a joint learning framework-related agreement with the central server. Next, each user downloads the global model from the server to the local for training, and then uses the feature decoupling and adaptive noise injection method to process the gradient information uploaded to the central server for differential privacy. After training is complete, each participating user downloads the global optimal model to perform online quality monitoring of LPBF powder defects. In summary, the proposed method provides a promising solution for LPBF powder bed defect detection tasks that balances high performance and privacy security, successfully separating foreground and background features, and then adaptively adjusting the noise injection intensity based on the sensitivity of each feature. This dynamic noise allocation strategy not only reduces the interference of noise on model performance, improves model detection performance and convergence speed, but also ensures a corresponding level of privacy protection.

[0237] In the laser powder bed melting defect detection method based on privacy-enhanced federated learning,

[0238] S100: Federated learning based on adaptive differential privacy. During the laser powder bed fusion printing process, each participating user collects powder bed images and pre-processes and calibrates the image data to construct their own data set. The key to this step is to ensure that each participant can collect high-quality data sets to provide a foundation for subsequent model training.

[0239] S200: Select the Attention-Unet defect detection model as the neural network, which consists of an encoder, a decoder, and a skip connection, and introduces a grouped convolution method to achieve feature decoupling of foreground and background information. Attention-Unet enhances the model's ability to handle complex backgrounds or imbalanced class distribution tasks through attention mechanisms. Grouped convolution further improves the model's ability to handle foreground and background information, helping to improve defect detection accuracy.

[0240] S300: Construct the loss function and perform global model update. After the loss function is constructed, each participating user first downloads the defect detection model from the central server, trains the defect recognition model locally using their own local data set, then uploads the model gradient information processed by adaptive differential privacy to the central server, and aggregates and updates the global defect recognition model on the central server. By processing the model gradient information with adaptive differential privacy, both user data privacy and the model's generalization ability and prediction accuracy can be effectively improved.

[0241] S400: Online detection of powder laying defects in LPBF printing process. Each participating user downloads the updated global model from the central server for online detection of powder laying defects in their own LPBF printing process. This stage realizes the practical application of the model, enabling real-time monitoring of quality problems during the printing process and timely detection and correction of defects.

[0242] Image acquisition and preprocessing (S100): After each layer of powder laying process, use image acquisition equipment to obtain powder bed images of the forming area and store them in computer equipment; name and label the obtained powder laying forming area platform topography image. These operations ensure the quality and usability of the data, which is the basis for building an efficient defect detection model.

[0243] Selection and optimization of defect detection model (S200): The encoder is mainly used to capture the global context information of the image, containing five convolution blocks and four max-pooling layers; the decoder is mainly used to gradually recover the spatial information lost by downsampling, containing four upsampling and four attention gate mechanism connected decoders. This structure design enables the model to effectively extract and utilize global information while preserving details, thereby improving the accuracy of defect detection.

[0244] Adaptive differential privacy federated learning (S300), adaptive differential privacy encryption: each participating user decouples the feature of local model gradient information, separates the foreground and background features, then adds less Gaussian noise to the foreground feature and more Gaussian noise to the background feature to improve the model prediction accuracy under the same privacy protection level. This method not only enhances privacy protection, but also reduces the impact on model performance by adding noise selectively.

[0245] The method of joint descent of multiple loss functions is used for model training (S300), the loss functions for realizing the classification task include cross-entropy loss, focus loss function and Des coefficient loss, and the loss functions for realizing feature decoupling include L2 norm loss and cosine similarity loss. By combining different types of loss functions, the model performance can be more comprehensively evaluated, especially for the class imbalance problem, and this method can significantly improve the accuracy of small target detection.

[0246] The loss function segmentation network of feature decoupling (S204), the grouping convolution is used in the up-sampling and down-sampling parts, according to the number of channels, the feature map is evenly divided into foreground and background two parts, by using the multi-loss optimization strategy, each part is encouraged to focus on learning its own features. The design of grouping convolution reduces the computational complexity, and at the same time improves the learning efficiency and accuracy of the model.

[0247] Although the embodiments of the present application are described above in combination with the drawings, the present application is not limited to the above specific embodiments and application fields, and the above specific embodiments are only illustrative and guiding, but not limiting. Those skilled in the art can make many forms under the inspiration of the present application and without departing from the scope protected by the claims of the present application, which all belong to the protection of the present application.

Claims

1. A privacy-enhanced federated learning based laser powder bed fusion defect detection method, characterized in that, The method comprises the following steps: S100: Each participating user of the federated learning based on adaptive differential privacy collects powder bed images during the laser powder bed fusion printing process, and pre-processes and calibrates the image data to construct respective data sets; S200: Selecting a defect detection model Attention-Unet as the neural network of the federated learning based on adaptive differential privacy, the network is composed of an encoder, a decoder and a jump connection, and a grouping convolution method is introduced on the basis of Attention-Unet to realize feature decoupling of powder bed image foreground and background information; S300: After the loss function is constructed, each participating user first downloads the defect detection model from the central server, trains the defect recognition model locally using the local data set, then uploads the model gradient information processed by adaptive differential privacy to the central server, and aggregates and updates the defect detection model weight and bias in the central server to obtain a global defect recognition model, and the training step is repeated until the global step reaches 200 times to ensure that the model converges and meets the training expectation; S400: Each participating user downloads the updated global model from the central server for online detection of powder laying defects in the LPBF printing process.

2. The privacy-enhanced federated learning based laser powder bed fusion defect detection method of claim 1, wherein, Preferably, S100 comprises: S101: The laser powder bed fusion printing process comprises powder spreading and melting alternately to manufacture a part, after the powder spreading process of each layer is completed, before the melting process, an image acquisition device is used to obtain a powder bed image of the forming area; S102: The platform topography of the forming area is named and labeled, different colors are used to label different defects and powder backgrounds of each powder bed image, and the label name is the same as the corresponding powder bed image; S103: Each participating user constructs a training set, a validation set and a test set according to 8:1:1 from the powder bed image and the corresponding labeled image, which is used for subsequent local model training.

3. The method of claim 1, wherein, S200 comprises: S201: The encoder comprises five convolution blocks and four maximum pooling layers; S202: The decoder comprises four upsampling and four attention gate mechanisms, and finally adds a convolution layer for adapting the output channel number; S203: input the image with size HxWx3 into the model, where H and W are the length and width of the input image respectively, HxWx64 is obtained through the first layer of convolution, HxWx128 is obtained through the second layer of convolution Similarly, HxWx512 is obtained through five layers of convolution Then, HxWx1024 is obtained through the first layer of upsampling After four layers of upsampling, it is finally restored to HxWx64, and then restored to HxWx6 through 1x1 convolution S204: At the third layer of the encoder, the channel number is evenly divided into foreground and background two parts, and a decoupling loss function is used to encourage the two parts to train corresponding features, and at the second layer of the decoder, the channel number is combined to restore the original channel number for subsequent convolution operation.

4. The method of claim 3, wherein the method is a privacy-enhanced federated learning based laser powder bed fusion defect detection method. In step S201, the convolution kernel size is 3x3, the step is 1, and after each convolution, batch normalization and an activation function (ReLU) are connected, the maximum pooling layer size is 2x2, and the step is 2.

5. The method of claim 3, wherein the method is a privacy-enhanced federated learning based laser powder bed fusion defect detection method. In step S202, a 3x3 transpose convolution is used, the step is 1, and the features after upsampling in the decoder and the corresponding encoder features are respectively mapped to the same channel number by 1x1 convolution and spliced.

6. The method of claim 1, wherein, S300 comprises: S301: Gradient aggregation and global model update: After the central server receives the defect detection model gradient information uploaded by each participating user after differential privacy encryption, the FedProx algorithm is used for weighted aggregation of gradient related weights and bias, and global training and update of the defect recognition model on the central server is performed. After the global training step is reached, the weights and bias of the defect recognition model are distributed to each participating user; S302: Variable learning rate and early stopping strategy: If the mIoU performance of the defect detection model on the validation set does not improve for multiple consecutive iteration periods, the early stopping strategy and the method of reducing the learning rate are used to deal with non-convergence to achieve the expected training.

7. The method of claim 6, wherein the method further comprises: The S302 comprises: S3021: The method of variable learning rate is ReduceLROnPlateau, which dynamically adjusts the learning rate according to the change of the validation set index. When the mIoU of the validation set does not improve significantly in consecutive training steps, the learning rate is automatically reduced by a certain proportion to make the model converge faster; S3022: The early stopping strategy is that when the validation set index no longer improves in consecutive training steps, it is likely that the defect recognition model has not converged or has begun to overfit the training data, so the training is terminated to avoid invalid calculation.

8. The method of claim 6, wherein the method further comprises: The global and local optimization functions of the FedProx algorithm are: where h m (·) is the local optimization objective function of the mth customer, is the weight and bias of the local defect detection model of the mth customer in the tth global iteration, w t is the weight and bias of the global defect detection model in the tth global iteration, N represents the total number of images of all users, M represents the total number of participating customers, N m represents the number of images of user m, represents the loss function of the trained model, μ is a penalty factor used to control the generation of local device divergence caused by heterogeneous defect data of the powder image.

9. A laser powder bed fusion defect detection apparatus, characterized in that, It includes, A laser powder bed fusion module comprising, A powder bed comprising, A substrate on which the powder is laid, A scraper that reciprocally levels the powder, A feed bin that provides powder, A recycling bin that contains remaining powder, A laser source that fuses the powder; A laser powder bed fusion powder bed online detection system comprising, An image acquisition device that collects powder bed image data, A federated learning module with adaptive differential privacy connected to the image acquisition device to perform the laser powder bed fusion defect detection method of any one of claims 1-8.

10. The laser powder bed fusion defect detection apparatus of claim 9, wherein, The image acquisition device comprises an industrial camera.

Images