Industrial quality inspection image-oriented adaptive clustering federated learning method

Abstract

The invention discloses a self-adaptive clustering federated learning method for industrial quality inspection images, and belongs to the technical field of industrial quality inspection image recognition model training. Different clients have the problem of poor federal learning effect caused by heterogeneous diversity of industrial quality inspection image data. For an industrial quality inspection system comprising a plurality of clients, model training is realized on the basis of the industrial quality inspection system based on clustering federal learning; in the clustering federation learning process, firstly, clients are preliminarily divided into a plurality of clusters according to the difference of data volumes of the clients, and the clusters are marked as a plurality of large clusters; based on a plurality of preliminarily divided large clusters, clustering the large clusters into small clusters by calculating the similarity of client model weights; before each round of training, performing clustering updating according to the actual condition of dynamic change of the client equipment in the industrial production environment; in each round of training, the participation proportion of the client is dynamically adjusted according to the detection loss of each cluster, and then the training process is completed.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition model training for industrial quality inspection, and relates to a clustering federated learning method for industrial quality inspection images. Background Art

[0002] Artificial intelligence technology is leading the rapid development of various industries. Despite the vast amount of information in the era of big data, much valuable data has not been effectively linked, leading to the phenomenon of data silos. How to securely and efficiently integrate this information and fully tap its potential value has become a key challenge currently faced. To this end, Federated Learning (FL) has emerged as an innovative distributed machine learning method. Federated Learning enables clients to jointly train and learn global optimization models without exchanging local data. Although federated learning brings many advantages, the problem of data heterogeneity and diversity still has a significant impact on its performance. Due to the heterogeneous diversity of client data distribution, the client's local loss function may be inconsistent with the optimization objective of the global model, which in turn causes the global model to deviate from the optimal solution, posing a challenge to the overall performance of the model.

[0003] To address the challenges brought by data heterogeneity and diversity, cluster federated learning (CFL) has gradually attracted attention in recent years. Its core idea is to use clustering methods to divide clients with similar data distributions into the same cluster, and jointly optimize the model specifically for the specific data distribution within the cluster, thereby alleviating the adverse effects of heterogeneity and diversity on the global model. Although existing studies have verified the effectiveness of CFL in improving model performance to a certain extent, in practical applications, especially in industrial visual inspection systems, the following key issues have not been fully addressed: 1) Industrial quality inspection images collected by different production lines or equipment have significant distribution differences. Direct clustering often fails to achieve ideal grouping results, which in turn affects the effectiveness of subsequent model training; 2) In industrial production environments, equipment status and acquisition conditions may change frequently. How to update the clustering structure in a timely manner when clients are dynamically added or removed, and ensure the accuracy and stability of the clustering results, becomes a major challenge; 3) Methods in which all or random clients participate in training cannot fully consider the actual quality and contribution of the image data of each device, resulting in low training efficiency and slow model convergence. Summary of the Invention

[0004] The present invention aims to solve the problem of poor federated learning effect and limited recognition ability of the training model caused by the heterogeneous diversity of industrial quality inspection image data on different clients during the industrial quality inspection process.

[0005] The adaptive clustering federated learning method for industrial quality inspection images includes the following steps:

[0006] The industrial quality inspection system contains multiple clients, each corresponding to a production line or equipment. The industrial quality inspection system implements model training based on cluster federated learning. During the cluster federated learning process, the clients are initially divided into several clusters based on the differences in the amount of client data, which are recorded as several large clusters. Based on the several large clusters initially divided, the large clusters are clustered into small clusters by calculating the similarity of the client model weights. Before each round of training, the clustering is updated according to the actual situation of dynamic changes in client equipment in the industrial production environment. In each round of training, the client participation ratio is dynamically adjusted according to the detection loss of each cluster to complete the training process.

[0007] Furthermore, the clients are initially divided into several clusters based on the difference in client data volume. The process of dividing the clients into several large clusters includes:

[0008] The server first sorts the client samples in descending order, starting with the client with the largest sample size, and selects the clients with the smallest sample size difference with the current group to join. Each time a client is added, the sample size difference within the current large cluster is checked; if the difference exceeds the dynamic division threshold, it stops adding clients to the cluster, starts creating a new large cluster, and continues to divide, thereby preliminarily dividing all clients into several large clusters.

[0009] Furthermore, the dynamic division threshold is as follows:

[0010]

[0011] Where Th1 is the dynamic partition threshold; λ1 is an adjustable constant; n represents the number of clients in the client set; D i represents the sample size in client i.

[0012] Furthermore, in the process of sequentially selecting the client with the smallest difference in sample size from the current group to join, a greedy algorithm is used to select the client with the smallest difference in sample size from the current group to join.

[0013] Furthermore, based on the initial division of several large clusters, the process of clustering the large clusters into small clusters by calculating the similarity of the client model weights includes:

[0014] After the large clusters are divided, the client model weights are reduced in dimensionality. Then, the L2 distance is used as the similarity measure. The Euclidean distance between the client models is calculated to evaluate their similarity. Each large cluster is clustered into small clusters based on the similarity.

[0015] Furthermore, in the process of dimensionality reduction of the client model weights, principal component analysis is used for dimensionality reduction.

[0016] Furthermore, clustering updates are implemented based on JS divergence during clustering updates according to the actual situation of dynamic changes of client devices in an industrial production environment.

[0017] Furthermore, the process of implementing clustering update based on JS divergence includes:

[0018] Use two probability distributions to represent the data distribution of two different clients and calculate the JS divergence D JS , through the JS divergence D corresponding to the initial data distribution JS Calculate the offset threshold Th2 = D JS ·λ2, where λ2 is the scaling factor; when the client's JS divergence exceeds the offset threshold, the cluster redistribution operation is triggered.

[0019] Furthermore, when the JS divergence of a client exceeds the offset threshold, the cluster reallocation operation is triggered, and the client detected as needing cluster adjustment is assigned to a cluster according to the following scheduling principle;

[0020] Scheduling principles:

[0021]

[0022] Among them, D c_New represents the sample size of unassigned clients, represents the sample size of the i-th large cluster, ω c_New Indicates that the client local update model weight is not assigned, Represents the cluster model weight.

[0023] Furthermore, in each round of training, the client participation ratio is dynamically adjusted according to the detection loss of each cluster. A loss-driven biased client selection mechanism is used for adjustment. The loss-driven biased client selection method is as follows:

[0024] During each round of training, the server calculates the average loss of all clusters and calculates the client selection ratio P of each cluster based on the average loss:

[0025]

[0026] Where P is the selection ratio, and G i,j The loss and the average loss of all clusters;

[0027] The client selection ratio is dynamically adjusted according to the average loss of the cluster to complete the training process.

[0028] Beneficial effects:

[0029] 1. To address the heterogeneous diversity of image data caused by equipment differences across multiple production lines in industrial visual inspection systems, an adaptive clustering algorithm for industrial quality inspection images is proposed. This algorithm deeply integrates the production line's visual acquisition system and employs a dual clustering strategy: First, it minimizes differences in data sample sizes between production line clusters to divide image data from different equipment models into large clusters, eliminating distribution shifts caused by differences in equipment parameters. Second, it calculates image feature distances to achieve small cluster division, effectively alleviating the interference of non-independent and identically distributed image data from multiple production lines on inspection model training. Furthermore, a clustering update mechanism designed based on JS divergence monitors changes in image histogram distribution in real time and dynamically adjusts the clustering structure to adapt to the characteristics of newly connected visual sensor data.

[0030] 2. To address the slow model convergence problem of traditional client selection methods in distributed industrial inspection systems, a loss-driven biased client selection method is proposed. By analyzing the detection loss of each device cluster in real time, the proportion of clients participating in training in different clusters is dynamically adjusted, thereby constructing a selection strategy that matches the distribution of industrial image features, effectively improving model stability and training accuracy.

[0031] 3. In industrial quality inspection tasks with different working conditions simulated by four datasets, a multi-scenario and multi-model experimental setup was adopted to simulate the actual industrial inspection environment and evaluate the performance of LDBCS-ACFL. The experimental results show that LDBCS-ACFL exhibits significant superiority in various scenarios with heterogeneous data diversity, verifying the effectiveness and advantages of the framework. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Schematic diagram of the parameter divergence of federated learning and clustered federated learning models in different scenarios.

[0033] Figure 2 This is the LDBCS-ACFL framework diagram.

[0034] Figure 3 The visualization results of the model similarity distance matrix in different industrial quality inspection scenarios, where (a) represents the balanced data scenario and (b) represents the unbalanced data scenario.

[0035] Figure 4 Schematic diagram of clustering update strategy.

[0036] Figure 5 Partial comparison results of the accuracy of LDBCS-ACFL and the baseline algorithm on different datasets in a static environment.

[0037] Figure 6 Comparison results of the test accuracy of LDBCS-ACFL and the baseline algorithm on the FEMNIST dataset in various scenarios in dynamic environments. DETAILED DESCRIPTION

[0038] To address the problems and challenges in this technology, we propose an adaptive clustering federated learning framework based on loss-driven client selection (LDBCS-ACFL). This framework aims to enhance the flexibility and effectiveness of clustering federated learning in processing data heterogeneity and diversity (including data imbalance), the dynamic adaptability of clustering algorithms, and client selection for industrial quality inspection image applications, thereby further improving the performance of CFL models. The following describes an adaptive clustering federated learning method for industrial quality inspection images in conjunction with specific implementations.

[0039] In industrial visual inspection systems, the image data collected by different production lines or equipment are obviously heterogeneous and diverse. In order to realize distributed image model training while protecting the data privacy of each production line, the federated learning framework is introduced to achieve collaborative optimization of multi-source data. Assume that the system contains multiple clients, each client corresponds to a production line or equipment, and the client set of the federated learning system C = {c1,c2…c n}, the local sample dataset of client i is D i , the total number of samples is D, and the global optimization goal of federated learning for industrial quality inspection image applications is:

[0040]

[0041] Among them, |D i | represents the set D i The cardinality (number of elements), that is, the set D i The number of samples in |D i |Used in scenarios with weighted loss functions, where the amount of data in different subsets may be unbalanced and contributions need to be balanced through weighting; represents the weighted loss function.

[0042] Given the loss function L(ω,x,y) of a single sample in the client sample, where (x,y) represents the features and corresponding labels of the sample, the local optimization goal of federated learning is:

[0043]

[0044] The FedAvg algorithm coordinates communication between the server and the client in T rounds, and in each round the client uses the latest global model ω t Perform local optimization, and finally the server obtains a new global model by aggregating the model updates uploaded by the client t+1 The client performs multiple rounds of stochastic gradient descent (SGD) locally according to the learning rate η. For the divergence, implement local model update:

[0045]

[0046] The FedAvg algorithm is affected by data heterogeneity and diversity, which can easily cause the global model to overly favor the data distribution of certain clients during training. This can reduce the model's generalization ability and negatively impact overall training accuracy. To address this issue, the FedProx algorithm proposed by Li et al. introduces a regularization term in local model updates, ensuring that local updates not only rely on local data loss but also maintain parameter consistency with the global model. However, global consistency optimization methods often slow convergence and increase algorithmic complexity by forcing local models to update toward the global model.

[0047] like Figure 1 As shown in the figure, traditional federated learning has obvious model parameter divergence problems in scenarios with heterogeneous data. For independent and identically distributed (IID) scenarios, FL's global model can be stably optimized through average updates of clients and adapt to homogeneous data distributions. However, in Non-IID (non-independent and identically distributed) scenarios, due to the significant differences in data distributions among clients, a single global model is difficult to adapt effectively, resulting in highly dispersed parameter update directions, which seriously affects the performance of the global model. To address this challenge, CFL (clustered federated learning) clusters clients with similar data distributions into the same cluster in Non-IID scenarios, allowing clients within each cluster to share and optimize a cluster model. In this way, the optimization objectives of each cluster can better fit its specific data distribution characteristics, thereby transforming a single global optimization objective into a cluster multi-objective optimization, alleviating the negative impact of data heterogeneity.

[0048] Assume that n clients are divided into m clusters G = {G1, G2…G m}, the cluster model is ω G ={ω1,ω2…ω m The global optimization goal of clustering federated learning for industrial quality inspection images is:

[0049]

[0050] Research in clustering federated learning continues to emerge, driving the continuous evolution of its methods and applications. Ghosh et al. proposed the Iterative Federated Clustering Algorithm (IFCA), which estimates client cluster membership by minimizing loss, but requires broadcasting the cluster model and presetting the number of clusters in each round. Xie et al. proposed the Multi-center Federated Learning Algorithm (FeSEM), which uses L2 distance-based randomized expectation maximization to calculate the parameter distance between the client model and the cluster model and update the cluster centers. However, the curse of dimensionality is prone to occur in high-dimensional data. Duan et al. proposed the Flexible Clustering Federated Learning Algorithm (FlexCFL), which uses cold start and client migration strategies to cope with changes in data distribution. However, it requires presetting the number of clusters and offset threshold, and the random selection of clients is unstable.

[0051] Traditional clustering federated learning frameworks usually adopt a strategy of having all clients or randomly selected clients participate in training. Although the participation of all clients in training can maximize the use of data resources, it will significantly increase communication costs; although random selection of clients can reduce communication overhead to a certain extent, it fails to effectively identify and utilize key clients, resulting in low training efficiency and limiting the performance of some cluster models. The present invention proposes a biased client selection method and selects clients to participate in training based on a predetermined method. When dealing with the problem of data heterogeneity and diversity, clustering federated learning can not only significantly improve training efficiency and effectiveness, but also reduce communication overhead, which is more in line with the actual industrial quality inspection image needs.

[0052] The adaptive clustering federated learning method for industrial quality inspection images described in this embodiment is actually an adaptive clustering federated learning method based on loss-driven biased client selection (LDBCS-ACFL). That is, the present invention proposes an LDBCS-ACFL framework, which aims to overcome the limitations of existing clustering federated learning (CFL) methods and promote the widespread application and in-depth development of CFL in the field of industrial quality inspection images. To solve the problems caused by data heterogeneity and diversity in industrial quality inspection images, the present invention designs an adaptive clustering algorithm for industrial quality inspection images, realizes refined grouping of clients, and can update the clustering structure in real time according to dynamic changes in clients. Specifically, after the server broadcasts the initialization of the global model, pre-training is first performed; each client is trained based on local industrial quality inspection image data and uploads the sample size and model parameters. The server first adopts a strategy of minimizing the difference in sample size within the cluster to alleviate the data imbalance problem; then, through adaptive similarity clustering based on model parameter distance calculation, clients with similar data distribution are divided into the same cluster, thereby effectively addressing the challenges brought by data heterogeneity and diversity in industrial quality inspection images. In order to adapt to the dynamic changes of client devices in actual applications, the algorithm further designs a clustering update mechanism based on JS divergence. Before each round of training, the JS divergence of the data distribution of each client is calculated and compared with the preset offset threshold to detect changes in data distribution, thereby optimizing the clustering structure in real time while taking into account the dynamic addition of new clients. In order to improve the model instability caused by traditional client selection methods, the present invention also introduces a loss-driven biased client selection mechanism. In each round of training, the client participation ratio is dynamically adjusted according to the detection loss of each cluster, thereby accelerating the convergence speed of the cluster model with larger loss. The overall structure of the LDBCS-ACFL framework is as follows: Figure 2 shown.

[0053] The present embodiment is an adaptive clustering federated learning method for industrial quality inspection images. The adaptive clustering algorithm proposed in the specific learning process plays a key role in clustering federated learning, especially for the common data heterogeneity and diversity problems in the field of industrial quality inspection images. The existing CFL framework mainly focuses on processing Non-IID data, but in actual industrial applications, not only does the Non-IID problem caused by label offset exist, but unbalanced data is also common and has an important impact on model performance. If actual industrial quality inspection images are directly used as processing objects for verification, the lack of clear judgment criteria for actual industrial quality inspection images is not conducive to the description of the processing effect of the present invention. In order to ensure that the effect has a relatively unified recognition, this embodiment uses open source image data as industrial quality inspection images for experiments; at the same time, in order to verify the impact of different data distributions (such as balanced data and unbalanced data) on the model similarity matrix, the present invention uses the MNIST data set as an example to conduct a comparative experiment: in the experiment, 10 clients are divided into two groups of different data distributions according to different label assignments, and trained using a fully connected neural network. The experimental results are presented in a visual manner, where Figure 3 (a) shows the similarity matrix calculated using L2 distance in a balanced data scenario, where the two clusters can be divided more accurately; Figure 3 (b) shows that in the imbalanced data scenario, the metrics for clients 0 to 4 in the similarity matrix are disrupted, resulting in a significant decrease in clustering effectiveness. This result indicates that imbalanced data has a significant impact on the accuracy of client clustering, which further affects the performance of subsequent model training.

[0054] To this end, the present invention proposes an adaptive clustering algorithm for industrial quality inspection images from the two perspectives of unbalanced data and Non-IID data (label offset). Its innovation is mainly reflected in two stages. In the first stage, the refined client clustering algorithm comprehensively considers the diversity of industrial quality inspection image data, including factors such as data imbalance and label distribution offset, and refines the client grouping by constructing optimization objectives to achieve optimization of clustering effect; in the second stage, the clustering update strategy is designed based on the JS divergence in response to the actual situation of dynamic changes in client devices in industrial production environments. This strategy monitors the client data distribution in real time before each round of training, and detects changes in data distribution in a timely manner by calculating the JS divergence and comparing it with the preset offset threshold, and dynamically adjusts the clustering structure. At the same time, the algorithm also has a mechanism for handling new clients joining to ensure that the clustering results can always reflect the latest actual data distribution status.

[0055] In the process of clustering federated learning, a dual clustering strategy is used for clustering:

[0056] The dual clustering strategy employs a hierarchical clustering process to achieve more accurate and flexible client clustering. First, in the preliminary clustering phase, clients are divided into several large clusters based on differences in client data volume. This effectively addresses data imbalance and lays the foundation for subsequent refined processing. Next, in the refined clustering phase, based on the preliminary clustering results, the cluster structure is further refined by calculating the similarity of client model weights, forming multiple small clusters. This achieves multi-level refined clustering. The specific implementation details of the dual clustering strategy are detailed in Algorithm 1.

[0057]

[0058]

[0059] The traditional greedy algorithm relies on a fixed sample size difference threshold for grouping, but this approach lacks flexibility when facing different data distributions. To overcome the limitations of hard threshold settings, the present invention implements a dynamic partitioning threshold based on the client sample size standard deviation, which can flexibly adjust the grouping granularity according to the changes in sample size and adapt to scenarios with different sample size distributions. Assume that the client i dataset is D i , the dynamic partition threshold Th1 is calculated as follows:

[0060]

[0061] Where λ1 is an adjustable constant, and the partitioning threshold is customized in different scenarios according to the degree of data imbalance to ensure better adaptive effect. n represents the number of clients in the client set; D i represents the sample size in client i;

[0062] The server first sorts clients in descending order based on sample size, ensuring that clients with larger sample sizes are assigned to groups first. It then uses a dynamic partitioning threshold to improve the greedy algorithm (improved dynamic threshold) to further flexibly control the granularity of large cluster grouping. Starting with the client with the largest sample size, clients with smaller sample size differences from the current group are selected to join. Each time a client is added, the sample size difference within the current large cluster is checked. If the difference exceeds the dynamic partitioning threshold, the server stops adding clients to the cluster and creates a new large cluster. This ensures that the sample size difference within each large cluster is as small as possible.

[0063] After the large clusters are divided, the negative impact of unbalanced data on model similarity clustering is effectively alleviated. When performing refined clustering on the basis of large clusters, calculating the similarity of the model weights between each client and other clients will consume a lot of computing resources, especially in high-dimensional space, the computing overhead is extremely large. In order to reduce this overhead and improve computing efficiency, the present invention uses principal component analysis (PCA) to reduce the dimension of the client model weights. PCA can extract the main components in the data and remove redundant information, thereby compressing the data dimension to a lower space. On this basis, the L2 distance is used as the similarity measure, and the Euclidean distance between client models is calculated to evaluate their similarity. In this way, not only the time and space complexity of the similarity calculation are reduced, but also the accuracy of the distance metric is improved, effectively avoiding the dimensionality disaster problem. The Euclidean distance of the model weights of different clients after dimensionality reduction is defined as:

[0064]

[0065] where ω i ′ and ω j ′ are the model weight vectors of client i and client j after PCA dimensionality reduction, with dimension m, ω′ i,k and ω′ j,k They are the weight vector ω i ′ and ω j The kth element in ′. The similarity matrix M′ is defined as:

[0066] M′=d(ω i ′,ω j ′) i,j∈n (7)

[0067] To avoid the limitations of traditional clustering algorithms in determining the number of clusters, the OPTICS algorithm is introduced for adaptive clustering based on the similarity matrix M′. It does not require a preset number of clusters and is insensitive to input parameters. It can automatically determine the number of clusters according to the density distribution of data points and can complete the allocation of small clusters more reasonably and stably.

[0068] The dual clustering strategy assigns clients to small clusters and performs adaptive clustering (dynamic partitioning) on these small clusters. This strategy not only handles changes in client data distribution during training, but also adapts to new clients joining. The following JS divergence calculation is performed locally on each client.

[0069] The clustering update strategy based on JS divergence when calculating the JS divergence of each client data distribution before each round of training is as follows:

[0070] Taking into account the dynamic changes of clients during the training process of industrial quality inspection image application scenarios, including the addition of new clients and changes in client data distribution, the adaptive clustering algorithm realizes real-time clustering update optimization by designing a clustering update strategy based on JS divergence. JS divergence is a measurement method for measuring the similarity between two probability distributions, and is particularly sensitive to changes in category distribution, which enables it to effectively detect changes in client data distribution in federated learning. Its calculation relies only on the direct estimation of the probability distribution, avoiding complex optimization processes and high computational costs, and has higher computational efficiency; JS divergence avoids computational instability problems caused by zero-probability events like KL divergence through symmetry and averaging, and has stronger robustness. The client judges the degree of change in data distribution through a detection mechanism. Therefore, in order to measure the similarity between data distributions, the present invention adopts JS divergence as the key indicator of the clustering update strategy to help judge the degree of change in client data distribution, which is defined as follows:

[0071]

[0072] Among them, P and Q represent the current distribution and the basic distribution respectively. In the scenario of federated learning, these two probability distributions represent the data distribution of different clients. x is the index of each category in the probability distribution. P(x) and Q(x) represent the probability values of P and Q under x. Figure 4 As shown in the figure, when judging changes in data distribution, the clustering update strategy gives priority to sample size fluctuations. The cluster reallocation operation is triggered only when the label category changes and the JS divergence exceeds the dynamic threshold, avoiding unnecessary cluster adjustments caused by sample size fluctuations, thereby improving the efficiency of the framework.

[0073] The clustering update strategy is based on the initial data distribution JS divergence D JS The offset threshold Th2 is defined together with the proportional coefficient λ2, and the threshold can be flexibly adjusted according to the specific scenario. The offset threshold Th2 is defined as follows:

[0074] Th2=D JS ·λ2 (9)

[0075] When the JS divergence of a client exceeds the offset threshold, a cluster reallocation operation is triggered. Clients that are detected as needing cluster adjustment will be assigned to clusters according to the scheduling principle (10) just like new clients.

[0076]

[0077] Among them, G i represents the i-th largest cluster, G i,j represents the jth small cluster in the i-th large cluster, D c_Newrepresents the sample size of unassigned clients, represents the sample size of the i-th large cluster, ω c_New Indicates that the client local update model weight is not assigned, Represents the cluster model weight. Unassigned clients include clients whose JS divergence exceeds the threshold and require cluster re-adjustment, as well as clients newly added during training. Through this cluster update strategy, the framework can handle cluster update adjustments due to changes in data distribution in real time and handle the addition of new clients, making the entire framework more adaptable to dynamic changes in the real environment.

[0078] In each round of training, the client participation ratio is dynamically adjusted according to the detection loss of each cluster. During this process, a loss-driven biased client selection mechanism is used for adjustment. The loss-driven biased client selection method is as follows:

[0079] Taking into account the communication cost of all clients participating in the training and the instability of randomly selecting clients to participate in the training, the present invention uses cluster loss to dynamically allocate the selection ratio of clients in each cluster according to the characteristics of the cluster structure. The cluster loss reflects the performance of the cluster model. The larger the cluster loss, the lower the accuracy of the cluster model. The reason is usually that the number of client samples in the cluster is small, resulting in poor local model update effect. Through the loss-driven biased client selection method, more clients can be involved in clusters with poor model performance, thereby accelerating the convergence of the cluster model and improving the overall model effect.

[0080] During each round of training, the server calculates the average loss of all clusters and calculates the client selection ratio of each cluster based on formula (11):

[0081]

[0082] Where P is the selection ratio, and G i,jloss and the average loss of all clusters. The client selection ratio is dynamically adjusted according to the average loss of the cluster, which has better flexibility. Unlike the traditional method of defining the selection ratio by the ratio of cluster loss to average loss, the design of the present invention ensures the smoothness and stability of the client selection ratio by taking the maximum value with 1. The advantage of this design is that even if the loss of a cluster is small or close to the average loss, it will not cause insufficient model updates or slow down the training speed due to too low a selection ratio. Such a strategy avoids excessive ratio adjustments due to excessive fluctuations in single-round losses, reduces instability in the training process, and ensures that the client selection ratio of each cluster is reasonable and stable. Through this loss-driven biased client selection method, the present invention effectively improves the shortcomings of traditional client selection strategies, further improves model performance, and enhances the stability and convergence speed of the training process.

[0083] In order to fully verify the effect of the present invention, this embodiment conducts a comparative experiment with the following existing methods.

[0084] FedAvg: A federated learning algorithm based on model parameter averaging. FedAvg optimizes the model by training on the client and periodically aggregating the parameters of the global model.

[0085] FeSEM: A multi-center federated learning algorithm, built on a clustered federated learning framework based on L2 distance. FeSEM optimizes the client distribution within the cluster by randomly minimizing the expected difference between clients and groups, thereby improving the model's generalization capabilities.

[0086] IFCA: Iterative Clustering Federated Learning algorithm, which uses a strategy of alternating between estimating cluster identities and minimizing a loss function. This method iteratively adjusts client assignments, gradually improving the accuracy of the cluster model.

[0087] FlexCFL: A clustering federated learning algorithm tailored to changing data distributions. Based on a hybrid clustering framework combining factored cosine similarity and Euclidean distance, FlexCFL aims to address the challenges of data heterogeneity and distribution shift in federated learning. FlexCFL dynamically adapts to changes in data distribution, optimizing the model training process and improving model robustness.

[0088] At the same time, in order to verify the effectiveness and robustness of the present invention for different industrial quality inspection images, the present invention also uses open source datasets other than MNIST to simulate different industrial quality inspection images for processing, thereby illustrating the effect.

[0089] MNIST: A classic dataset of handwritten digits, containing 70,000 28x28 pixel grayscale images of handwritten digits from 10 categories. This paper simulates the product number recognition and classification task on this dataset and constructs a client dataset with heterogeneous data diversity by randomly selecting two types of labeled data.

[0090] FEMNIST: An extended version of the MNIST dataset, containing 800,000 28x28 pixel images of handwritten letters, covering 62 characters (26 uppercase letters, 26 lowercase letters, and 10 numbers). In this paper, this dataset is used to simulate the product number recognition and classification task. By applying the FedProx algorithm to resample the original data, adjusting the categories to 10 ('a'-'j'), and randomly selecting three types of labeled data, we construct a client dataset with heterogeneous data diversity.

[0091] CIFAR-10: A standard image classification dataset containing 60,000 32x32 pixel color images from 10 categories. This paper simulates a multi-classification task for industrial parts on this dataset. The client randomly selects four categories of labeled data to construct a training scenario characterized by heterogeneous data diversity.

[0092] Synthetic: This synthetic dataset, proposed by Shamir et al., is specifically designed for federated learning. It simulates scenarios where multiple clients collaborate on training tasks under realistic challenges such as data heterogeneity. In this paper, this dataset is used to simulate a multi-classification task for industrial parts. Client data is generated using the FlexCFL algorithm, with hyperparameters set to α = 1 and β = 1 to control for statistical heterogeneity between clients.

[0093] To verify the effectiveness of LDBCS-ACFL, we used three different models: MCLR, MLP, and CNN. The specific model design for each dataset is shown in Table 1:

[0094] Table 1 Model designs under different data sets

[0095]

[0096]

[0097] Hyperparameter Settings: In the experimental setup, the number of clients is M = 100, the number of clients selected per round is m = 40, the number of communication rounds is set to T = 100, the number of local training rounds for each client is E = 10, the batch size of local updates is B = 10, and the learning rate η is set based on FlexCFL and adjusted for different datasets: η = 0.01 for MNIST and Synthetic, η = 0.02 for FEMNIST, and 0.005 for CIFAR-10. In addition, the data swap probability Swap_p is set to 0.05. All other settings follow the FlexCFL algorithm. Specifically, in the LDBCS-ACFL framework, this study uses the following hyperparameters: min_size = 4, xi = 0.2, λ1 = 0.5, and λ2 = 0.2 to ensure robustness and effectiveness in diverse data heterogeneity scenarios.

[0098] The experimental content of the present invention is divided into static environment experiments and dynamic environment experiments, aiming to comprehensively evaluate the performance of LDBCS-ACFL in practical application scenarios such as industrial quality inspection images and product number recognition. In the experiment, the data heterogeneity and diversity simulation covers two situations: unbalanced data and Non-IID (label offset), which respectively reflect the common data uneven distribution and label distribution offset problems in industrial visual inspection and product recognition tasks. In the static environment experiment, the data held by each client remains stable, simulating the fixed data distribution in industrial quality inspection images or product number recognition tasks; in the dynamic environment experiment, considering the continuous changes in equipment status and acquisition conditions in actual industrial production, three dynamic settings are further designed: randomly exchanging all data of some clients (all), randomly exchanging part of the data of some clients (part), and incremental data (increment) experiments. The incremental data experiment refers to the fact that under the specified training rounds (0, 25, 50, and 75 rounds), the amount of original data held by the client is 1 / 4, 2 / 4, 3 / 4, and 1 times respectively. In cluster federated learning, due to the presence of multiple cluster models, this paper uses the weighted accuracy of the FlexCFL algorithm as a measure of overall model performance. The final result of each experiment is based on the highest test accuracy achieved, and all results are taken as the median of three independent experiments to reduce the interference of accidental factors. To intuitively demonstrate the comparative effect of LDBCS-ACFL and the four baseline methods in terms of model accuracy and convergence speed, Tables 2 and 3 list the test accuracy results under different experimental settings, respectively, with the best results shown in bold. Figure 5 and Figure 6It shows some comparisons of the test accuracy of various methods on the MNIST, FEMNIST (product number recognition task) and CIFAR-10 and Synthetic (industrial parts multi-classification task) datasets.

[0099] (1) Static environment test:

[0100] Table 2 shows the results of 12 experiments in a static environment, with LDBCS-ACFL achieving optimal performance in 11 of them. In scenarios with high data heterogeneity and diversity, the baseline algorithm struggles to determine the correct number of clusters, resulting in poor initial clustering results and, consequently, ineffective formation of a specialized clustering model, ultimately impacting model training performance. We further analyzed the performance of LDBCS-ACFL on various datasets. Compared to the optimal baseline results, LDBCS-ACFL achieved improvements of 2.04% on the MNIST dataset, 5.51% on the FEMNIST dataset, 3.31% on the CIFAR-10 dataset, and 3.06% on the Synthetic dataset. With the exception of LDBCS-ACFL and FlexCFL, the other algorithms exhibited poor model stability across multiple scenarios, preventing them from maintaining optimal performance in complex scenarios with heterogeneous data. In particular, the FeSEM algorithm, which relies solely on L2 distance, has poor adaptability to data heterogeneity and diversity, resulting in irrational client grouping. Furthermore, the EM algorithm struggles to converge to the global optimal solution in scenarios with heterogeneous data, further validating the effectiveness of the dual clustering strategy on L2-based clustering methods. On the Synthetic dataset, LDBCS-ACFL performed 0.95% lower than IFCA in the MCLR model experiment. This may be due to the fact that IFCA formed a better clustering model during initialization, while LDBCS-ACFL may have lost some key information during the small cluster clustering process due to dimensionality reduction, thus affecting the clustering effect. However, the performance difference in a single scenario cannot mask the significant advantages of dimensionality reduction in reducing computational overhead. Another reason is that the MCLR model is relatively simple and fails to capture the complex relationships in the data, which affects its performance.

[0101] Table 2 Comparison of test accuracy between LDBCS-ACFL and four baseline algorithms in static environment

[0102]

[0103]

[0104] The loss-driven biased client selection in LDBCS-ACFL effectively guides the training direction of each cluster model by increasing the selection ratio of clients in the cluster with larger loss in the early stage of training. Figure 5 The convergence rate of LDBCS-ACFL on the MNIST dataset is slightly slower than that of FedAvg and FlexCFL. This phenomenon can be attributed to the fact that loss-driven biased client selection adjusts the proportion of larger clusters during early training, resulting in relatively slow learning of the overall model. However, on other datasets, LDBCS-ACFL's convergence rate is not affected and is actually accelerated. This difference is due to the fact that as dataset complexity increases, other baseline algorithms are unable to effectively address the problem of insufficient cluster model training due to insufficient sample size. When faced with more complex tasks, these baseline algorithms often struggle to ensure that each cluster model fully converges by randomly selecting clients, thus affecting their overall model performance and convergence rate. In contrast, LDBCS-ACFL's loss-driven biased client selection avoids this problem by rationally adjusting the proportion of selected clients, ensuring that cluster models with larger losses are fully trained.

[0105] (2) Dynamic environment experiment:

[0106] In dynamic environment experiments, all baseline methods adopted the client migration algorithm in FlexCFL and preset an offset value of 0.2. Table 3 shows the results of 36 experiments in dynamic environments, among which LDBCS-ACFL achieved the best performance. The clustering update strategy in LDBCS-ACFL uses a JS divergence-based calculation method, which is more sensitive to label changes and further refines the data distribution change problem into three cases, avoiding unnecessary client scheduling. Experimental results show that LDBCS-ACFL better adapts to various scenarios with heterogeneous and diverse data, can achieve better clustering results, and demonstrates higher efficiency in subsequent model training.

[0107] Table 3 Comparison of test accuracy between LDBCS-ACFL and four baseline algorithms in dynamic environment

[0108]

[0109]

[0110] The clustering structure in a dynamic environment is constantly changing, and random client selection may cause some cluster models to fail to be fully trained, thus affecting the performance of the overall model. This problem is particularly evident in the "increment" scenario, where the addition of new data in a specified round may increase the difficulty of training the insufficiently trained clusters, thereby impairing model performance. The loss-driven biased client selection strategy implemented in the present invention can provide a reasonable training method for cluster models within a limited number of training rounds, effectively alleviating this problem and ensuring a more efficient convergence process.

[0111] In summary, the proposed adaptive clustering federated learning framework LDBCS-ACFL, based on loss-driven biased client selection, provides an efficient and flexible solution to the data heterogeneity and client dynamics challenges commonly encountered in industrial quality inspection imagery. By designing an adaptive clustering algorithm tailored to data heterogeneity, this framework effectively addresses imbalanced data and non-IID problems. It also optimizes the clustering structure in real time using a clustering update strategy based on JS divergence, effectively addressing the dynamic changes in device data in actual production environments. Furthermore, the introduced loss-driven biased client selection method ensures that each cluster model is fully trained, significantly improving overall model accuracy and convergence speed while enhancing model stability. Multi-scenario, multi-model experiments based on four public image datasets demonstrate that LDBCS-ACFL outperforms other baseline algorithms under various data heterogeneity settings, fully demonstrating its potential and practical effectiveness in industrial visual inspection systems. These findings provide a solid theoretical basis and practical support for the promotion and in-depth application of CFL technology in the field of industrial quality inspection imagery.

[0112] The present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. An adaptive clustering federated learning method for industrial quality inspection images, characterized by: The following steps are involved: The industrial quality inspection system contains multiple clients, each corresponding to a production line or equipment. Model training is implemented based on cluster federated learning. During cluster federated learning, the clients are initially divided into several clusters based on the amount of data from the clients, which are then recorded as several large clusters. Based on several initially divided large clusters, the large clusters are clustered into small clusters by calculating the similarity of client model weights. Before each round of training, the clustering is updated according to the actual dynamic changes of client devices in the industrial production environment. In each round of training, the client participation ratio is dynamically adjusted according to the detection loss of each cluster to complete the training process.

2. The adaptive clustering federated learning method for industrial quality inspection images according to claim 1, characterized in that: First, the clients are initially divided into several clusters based on the difference in client data volume. The process of dividing them into several large clusters includes: The server first sorts the client samples in descending order, starting with the client with the largest sample size, and selects the clients with the smallest sample size difference with the current group to join. Each time a client is added, the sample size difference within the current large cluster is checked; if the difference exceeds the dynamic division threshold, it stops adding clients to the cluster, starts creating a new large cluster, and continues to divide, thereby preliminarily dividing all clients into several large clusters.

3. The adaptive clustering federated learning method for industrial quality inspection images according to claim 2, characterized in that: The dynamic partitioning threshold is as follows: Where Th1 is the dynamic partition threshold; λ1 is an adjustable constant; n represents the number of clients in the client set; D i represents the sample size in client i.

4. The adaptive clustering federated learning method for industrial quality inspection images according to claim 3 is characterized in that: In the process of selecting the client with the smallest difference in sample size from the current group to join, a greedy algorithm is used to select the client with the smallest difference in sample size from the current group to join.

5. The adaptive clustering federated learning method for industrial quality inspection images according to any one of claims 2 to 4, characterized in that: Based on the initial division of several large clusters, the process of clustering the large clusters into small clusters by calculating the similarity of the client model weights includes: After the large clusters are divided, the client model weights are reduced in dimensionality. Then, the L2 distance is used as the similarity measure. The Euclidean distance between the client models is calculated to evaluate their similarity. Each large cluster is clustered into small clusters based on the similarity.

6. The adaptive clustering federated learning method for industrial quality inspection images according to claim 5, characterized in that: In the process of dimensionality reduction of client model weights, principal component analysis is used for dimensionality reduction.

7. The adaptive clustering federated learning method for industrial quality inspection images according to claim 5, characterized in that: In the process of clustering update according to the actual situation of dynamic changes of client devices in industrial production environment, clustering update is realized based on JS divergence.

8. The adaptive clustering federated learning method for industrial quality inspection images according to claim 7, characterized in that: The process of implementing clustering updates based on JS divergence includes: Use two probability distributions to represent the data distribution of two different clients and calculate the JS divergence D JS , through the JS divergence D corresponding to the initial data distribution JS Calculate the offset threshold Th2 = D JS ·λ2, where λ2 is the scaling factor; when the client's JS divergence exceeds the offset threshold, the cluster redistribution operation is triggered.

9. The adaptive clustering federated learning method for industrial quality inspection images according to claim 8, characterized in that: When the JS divergence of a client exceeds the offset threshold, the cluster reallocation operation is triggered. The client that is detected as needing cluster adjustment is assigned to a cluster according to the following scheduling principles; Scheduling principles: Among them, D c_New represents the sample size of unassigned clients, represents the sample size of the i-th large cluster, ω c_New Indicates that the client local update model weight is not assigned, Represents the cluster model weight.

10. The adaptive clustering federated learning method for industrial quality inspection images according to claim 9, characterized in that: In each round of training, the client participation ratio is dynamically adjusted according to the detection loss of each cluster. A loss-driven biased client selection mechanism is used for adjustment. The loss-driven biased client selection method is as follows: During each round of training, the server calculates the average loss of all clusters and calculates the client selection ratio P of each cluster based on the average loss: Where P is the selection ratio, and G i,j The loss and the average loss of all clusters; The client selection ratio is dynamically adjusted according to the average loss of the cluster to complete the training process.

Images