An Adaptive Hierarchical Distributed Learning Method and System for Heterogeneous Data in Industrial Production Lines

Abstract

This application provides an adaptive hierarchical distributed learning method and system for heterogeneous industrial production line data, relating to the field of distributed learning technology in industrial manufacturing. The method includes: constructing a federated learning framework for global training; uniformly aggregating low-level common visual features based on global model parameters to construct a shared feature space; performing mid-level adaptive clustering based on feature and gradient similarity to obtain a cluster-shared model; performing high-level fine-grained re-clustering and personalized optimization within clusters to construct personalized models; integrating model parameters from each layer to construct a tree-like multi-level aggregation structure and performing dynamic optimization; and finally ensuring training stability and generalization performance through cross-layer collaboration and convergence optimization. This application, through multi-level aggregation and dynamic clustering mechanisms, effectively solves problems such as semantic conflicts, low aggregation efficiency, and training instability caused by heterogeneous industrial production line data, significantly improving the model's convergence speed, stability, and generalization ability in cross-production line scenarios.

Description

Technical Field

[0001] This application relates to the field of distributed learning technology in industrial manufacturing, and in particular to an adaptive hierarchical distributed learning method and system for heterogeneous data in industrial production lines. Background Technology

[0002] With the rapid development of complex scenarios such as intelligent manufacturing, the Industrial Internet, and edge computing, the demand for cross-production line distributed learning is becoming increasingly urgent. Thanks to the rapid evolution of distributed learning technology, the industrial sector can avoid data privacy issues and improve the generalization ability of models across production lines through collaborative training of multiple production line nodes. However, existing distributed learning methods face a series of challenges: on the one hand, due to significant differences in product appearance, process flow, and equipment configuration among different production lines, existing global model aggregation methods are prone to semantic conflicts; on the other hand, due to the heterogeneity of data distribution across different production line nodes, existing distributed learning methods cannot adaptively adjust to the differences in features between production lines, resulting in low training efficiency and instability. To overcome these challenges, an adaptive hierarchical distributed learning method oriented towards the heterogeneity of industrial production line data is designed to achieve more stable and highly generalizable cross-production line learning. Summary of the Invention

[0003] This application provides an adaptive hierarchical distributed learning method and system for heterogeneous data in industrial production lines. It aims to address the problems of data heterogeneity, low aggregation efficiency, and unstable training in existing cross-production line joint learning methods by providing a hierarchical and adaptive distributed learning approach. This method effectively alleviates semantic conflicts during global model aggregation by introducing a multi-level aggregation structure and a dynamic clustering mechanism. It also adaptively adjusts the model based on the feature differences between different production lines, improving model stability and convergence speed. Furthermore, this application can meet the personalized needs of different production lines while ensuring a globally shared model, thereby improving the efficiency and generalization ability of cross-production line learning.

[0004] Firstly, this application provides an adaptive hierarchical distributed learning method for heterogeneous industrial production line data, including:

[0005] A federated learning framework and a global training process are constructed. The federated learning framework includes a central server and multiple production line nodes. The global training process is used to determine global model parameters.

[0006] Based on global model parameters, construct global underlying model parameters and a unified feature space;

[0007] Based on the global underlying model parameters and the unified feature space, the shared model parameters of the mid-level clusters are obtained;

[0008] Within each mid-level sub-cluster and its shared model parameters, high-level fine-grained re-clustering and personalized optimization are performed. The re-clustering operation is executed, and a personalized parameter decomposition mechanism is introduced to construct a personalized model for each node. The parameters are optimized by minimizing the personalized loss function to obtain the high-level personalized model parameters.

[0009] With the global bottom-level model parameters as the root node, the mid-level cluster shared model parameters as the intermediate nodes, and the high-level personalized model parameters as the leaf nodes, a tree-like multi-level aggregation structure is formed. A dynamic aggregation weight update mechanism and an aggregation depth adjustment mechanism are introduced to achieve cross-layer adaptive optimization.

[0010] Based on a tree-like multi-level aggregation structure, cross-layer collaboration and convergence optimization are performed. Collaborative updates of model parameters at different levels are achieved through feature distillation and gradient consistency constraints, and the learning rate is dynamically adjusted through an adaptive convergence control mechanism.

[0011] Furthermore, the global training process includes having each production line node independently train the model using local private data, and uploading the model parameter update results to the central server. The central server receives and merges the parameter updates to complete the global model parameter update, and then distributes the updated global model parameters to each production line node.

[0012] Furthermore, based on the global model parameters, a global underlying model parameter and a unified feature space are constructed, including:

[0013] Based on global model parameters, the methods for constructing global underlying model parameters and a unified feature space include: on the basis of the obtained global model parameters, each production line node performs global general feature extraction and underlying unified aggregation, extracts the general visual features of the product image, uploads the underlying feature parameters to the central server, and uses a global unified aggregation mechanism to fuse the parameters, thereby constructing a global underlying model parameter and a unified feature space shared across production lines.

[0014] Furthermore, the global underlying model parameters are determined using the following formula:

[0015] ;

[0016] in, These are the parameters of the global underlying model. The weighting coefficients for production line nodes are determined based on the sample size or data quality of the production line nodes, and satisfy the following conditions: The total number of production line nodes participating in the aggregation. Number the production line nodes. For the first The underlying parameters of each production line node.

[0017] Furthermore, based on the global underlying model parameters and the unified feature space, the methods for obtaining the shared model parameters of the mid-level clusters include:

[0018] Based on the global underlying model parameters and unified feature space, a mid-level adaptive clustering based on feature and gradient similarity is performed. The feature distribution similarity and gradient direction similarity between nodes are calculated to construct a joint similarity matrix. Based on the joint similarity matrix, an adaptive clustering algorithm is executed to divide the production line nodes into semantically consistent sub-clusters. Local parameter aggregation is performed within each sub-cluster to obtain the mid-level cluster shared model parameters.

[0019] Furthermore, the formula for calculating the feature distribution similarity is as follows:

[0020] ;

[0021] in, Representing production line nodes The mean of the middle-layer features; Representing production line nodes The characteristic covariance matrix; The trace operation of a matrix is ​​represented; exp represents an exponential function with the natural constant as the base. Indicates production line node Similarity of feature distributions;

[0022] The formula for calculating the gradient direction similarity is:

[0023] ;

[0024] in: These represent the mid-level gradient vectors of production line node i and production line node j, respectively. This represents the gradient direction similarity between production line node i and production line node j;

[0025] Based on the similarity of feature distributions and gradient directions, a joint similarity matrix is ​​constructed using the following formula:

[0026] ;

[0027] in, Indicates production line node With production line nodes Overall semantic similarity; These are weighting coefficients used to balance the importance of gradient similarity and feature similarity;

[0028] Based on the joint similarity matrix, the production line nodes are divided into several semantically consistent subclusters, and local parameter aggregation is performed within each cluster:

[0029] ;

[0030] in, For the first The middle layer of each cluster shares model parameters; The weight coefficient of the production line node within the cluster; Indicates the first A set of production line nodes for each cluster; This represents the number of clusters. For the first The middle layer of each production line node shares model parameters.

[0031] Furthermore, within each mid-level sub-cluster and its shared model parameters, high-level fine-grained re-clustering and personalized optimization are performed. The re-clustering operation is executed, and a personalized parameter decomposition mechanism is introduced to construct a personalized model for each node. Parameter optimization is performed by minimizing the personalized loss function to obtain the high-level personalized model parameters, including:

[0032] At the high-level stage, each sub-cluster uses its shared model as initial weights to establish a high-level personalized model:

[0033] ;

[0034] in, This represents the high-level structural hierarchy of the model. Indicates production line node Initial parameters of the high-level model, Indicates the first Cluster The middle layer shares model parameters;

[0035] In cluster Internal calculation of the high-level joint similarity matrix between production line nodes:

[0036] ;

[0037] in, This represents the similarity between production line node i and production line node j in the high-level gradient direction; Production line nodes With production line nodes High-level feature distribution similarity; , which is the weight coefficient in high-level clustering, used to balance structural similarity and consistency of update direction;

[0038] Perform intra-layer spectral clustering or hierarchical clustering operations based on the high-level joint similarity matrix to group the clusters. It is decomposed into several fine-grained subclusters;

[0039] After fine-grained sub-cluster partitioning is completed, a personalized parameter decomposition mechanism is introduced at each production line node; whereby, for each production line node... Its high-level model parameters consist of a shared component and a personalized bias component, expressed as:

[0040] ;

[0041] in, For production line nodes High-level personalized model parameters; For production line nodes Shared parameters of the fine-grained sub-clusters; For production line nodes Personalized parameter bias;

[0042] During local training, production line nodes minimize the following personalized loss function. Update:

[0043] ;

[0044] in, This represents the production line task loss function; This is the personalized regularization coefficient, used to limit the magnitude of personalized bias; Used to prevent overfitting;

[0045] Each production line node will update the personalized parameter bias. The shared parameter gradient information is fed back to the central server, which then performs personalized weighted aggregation within each fine-grained sub-cluster, updating the sub-cluster shared parameters.

[0046] ;

[0047] in, Indicates the first The first in the middle cluster A high-level fine-grained sub-cluster; Personalized aggregation weights for production line nodes. For the updated production line nodes Shared parameters of the fine-grained sub-clusters;

[0048] Will Redistribute to the corresponding production line node, and determine the production line node. High-level personalized model parameters.

[0049] Furthermore, the dynamic aggregation weight update mechanism is expressed as follows:

[0050] ;

[0051] in, For production line nodes In the The gradient variance of a layer is used to reflect the training stability of that node during the training process of the current layer; This is a temperature coefficient used to control variance sensitivity; For the updated aggregation weights, production line nodes with smaller gradient variances receive higher aggregation weights; exp represents an exponential function with the natural constant as its base. The gradient variance of the j-th production line node participating in the aggregation at the same layer is used to calculate the denominator normalization term;

[0052] The aggregation depth adjustment mechanism is expressed as follows:

[0053] ;

[0054] in For the first Aggregation depth during round training; For the first Aggregation depth during round training; This represents the global gradient variance for the current round. Convergence threshold and oscillation threshold.

[0055] Furthermore, the loss function of the characteristic distillation is defined as:

[0056] ;

[0057] in, Characteristic distillation loss, The number of samples; For the high-level model to the first Feature representation extracted from each sample; This represents the feature representation of the same sample in the mid-level model; Represents the square of the Euclidean distance;

[0058] The gradient consistency constraint is expressed as follows:

[0059] ;

[0060] In the formula, This represents the total number of levels in the model. and For the first and The average gradient vector of the +1 layer model; The loss value reflects the degree of consistency between the direction and magnitude of the interlayer gradient;

[0061] The adaptive convergence control mechanism dynamically adjusts the learning rate by monitoring the cross-layer gradient variance and the loss descent rate.

[0062] ;

[0063] in, and For the first Wheel and First +1 round of learning rate; This is the learning rate adjustment coefficient; The average gradient variance across layers; Changes in losses between adjacent rounds; These are the convergence and oscillation thresholds, respectively.

[0064] Secondly, this application provides an adaptive hierarchical distributed learning system for heterogeneous industrial production line data, used to implement the method described above, the system comprising:

[0065] The framework process construction module is configured to build a federated learning framework and a global training process. The federated learning framework includes a central server and multiple production line nodes, and the global training process is used to determine global model parameters.

[0066] The first parameter determination module is configured to construct global underlying model parameters and a unified feature space based on global model parameters;

[0067] The second parameter determination module is configured to obtain the shared model parameters of the mid-level clusters based on the global bottom-level model parameters and the unified feature space.

[0068] The third parameter determination module is configured to perform high-level fine-grained re-clustering and personalized optimization within each mid-level sub-cluster and its shared model parameters. It performs re-clustering operations and introduces a personalized parameter decomposition mechanism to build a personalized model for each node. It optimizes parameters by minimizing the personalized loss function to obtain the high-level personalized model parameters.

[0069] The aggregation structure building module is configured to form a tree-like multi-level aggregation structure with global bottom-level model parameters as the root node, mid-level cluster shared model parameters as intermediate nodes, and high-level personalized model parameters as leaf nodes. It also introduces a dynamic aggregation weight update mechanism and an aggregation depth adjustment mechanism to achieve cross-layer adaptive optimization.

[0070] The cross-layer optimization module is configured to perform cross-layer collaborative and convergent optimization based on a tree-like multi-level aggregation structure. It achieves collaborative updates of model parameters at different levels through feature distillation and gradient consistency constraints, and dynamically adjusts the learning rate through an adaptive convergence control mechanism.

[0071] The adaptive hierarchical distributed learning method for heterogeneous industrial production line data provided in this application has at least the following beneficial effects:

[0072] 1) This application achieves a dynamic balance between global sharing and personalized optimization by dividing the model into bottom, middle, and high layers according to semantic depth and adopting a differentiated aggregation strategy. This mechanism enables the model to maintain the consistency of general features while taking into account the uniqueness and adaptability of each production line, significantly improving the stability of cross-production line collaborative learning.

[0073] 2) This application dynamically identifies semantic consistency between different production lines and performs adaptive clustering by fusing feature distribution and gradient direction similarity. This method effectively reduces model bias and aggregation instability caused by data heterogeneity, and improves the robustness and convergence performance of distributed models in complex industrial scenarios.

[0074] 3) This application constructs a tree-like multi-level aggregation structure from global to local, and introduces adaptive weights and dynamic optimization mechanisms between different layers. This structure can automatically adjust the aggregation depth and participation range during training, thereby significantly improving the model's convergence speed and cross-production line generalization ability. Attached Figure Description

[0075] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0076] Figure 1 A flowchart illustrating an adaptive hierarchical distributed learning method for heterogeneous industrial production line data, provided in an embodiment of this application;

[0077] Figure 2 A flowchart illustrating the federated learning framework and global training process provided in this application embodiment;

[0078] Figure 3 A flowchart for obtaining global underlying model parameters provided in this application embodiment;

[0079] Figure 4 A flowchart for obtaining parameters of the mid-layer cluster sharing model provided in this application embodiment;

[0080] Figure 5 A flowchart for obtaining high-level personalized model parameters provided in this application embodiment;

[0081] Figure 6 A flowchart illustrating the construction and optimization of a tree-like multi-level aggregation structure provided in this application embodiment;

[0082] Figure 7 A flowchart illustrating the dynamic optimization of a tree-like multi-level aggregation structure provided in this application embodiment;

[0083] Figure 8 A flowchart of cross-layer collaboration and convergence optimization provided for embodiments of this application;

[0084] Figure 9 The embodiments of this application provide an adaptive hierarchical distributed learning system for heterogeneous data in industrial production lines.

[0085] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0086] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0087] The collection, storage, use, processing, transmission, provision, and disclosure of relevant data and information in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0088] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0089] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0090] This application provides an adaptive hierarchical distributed learning method for heterogeneous data in industrial production lines, such as... Figure 1 The diagram shows a flowchart of an adaptive hierarchical distributed learning method for heterogeneous industrial production line data provided in an embodiment of this application. The continuous learning model hot update method for industrial product defect detection includes the following steps S10 to S60.

[0091] S10: Construct a federated learning framework and a global training process. The federated learning framework includes a central server and multiple production line nodes. The global training process is used to determine the global model parameters.

[0092] Step S10 is the foundation of this adaptive hierarchical distributed learning method, implemented based on the Federated Learning (FL) framework. This framework includes a central server (Server) and multiple production line nodes (Clients). Each production line node independently trains its model using its private data within its local production line environment and uploads the updated parameter results of its local model to the central server. The central server receives and merges the parameter update information from each production line node to complete the global model update and then distributes the updated model parameters to each production line node.

[0093] In the industrial manufacturing sector, different production lines exhibit significant differences in product appearance, shooting angles, lighting conditions, and process flows, resulting in highly heterogeneous training data distribution. Traditional centralized modeling methods cannot share knowledge across multiple production lines, while federated learning frameworks can achieve distributed collaborative modeling without sharing raw data. This invention addresses the visual inspection and process data analysis tasks of industrial production lines by constructing a federated learning system consisting of a central server and multiple production line nodes. Each production line node is deployed on an edge computing terminal, acquiring local production line images or sensor signals, performing local model updates, and participating in global aggregation, thereby achieving unified optimization and knowledge sharing across production line models.

[0094] In some embodiments, such as Figure 2 As shown, step S10 is specifically implemented through the following steps S101-S105.

[0095] S101: Establish a federated learning framework.

[0096] The Federated Learning Framework comprises the following components:

[0097] Central Server: Responsible for initializing global model parameters, performing parameter aggregation, and controlling communication and training rounds;

[0098] Production line node (Client): Each production line node corresponds to a specific industrial production line, holds local production line images or sensor data, and is used to perform local model training and parameter updates.

[0099] Communication module: Enables the uploading and downloading of model parameters or gradients, but does not transmit raw production data to protect data privacy and production confidentiality;

[0100] Aggregation module: Performs global parameter updates on the server side and provides the foundation for subsequent hierarchical aggregation.

[0101] Under this federated learning framework, each federated round consists of four stages: global broadcast, local update, model upload, and global aggregation.

[0102] S102: Global training initialization.

[0103] At the beginning of training, the central server randomly initializes the global model parameters, denoted as . This parameter is distributed to all production line nodes as the initial parameter for the local model of each production line node: .in: The total number of production line nodes participating in the training; For the first Initial parameters of the local model for each production line node; These are the initial global model parameters.

[0104] S103: Local model training phase.

[0105] At each production line node Above, utilize local production line sample sets The model is trained locally by minimizing the local loss function. Update parameters using the following formula:

[0106] ;

[0107] in: For the sample The task loss function (such as cross-entropy, mean squared error, etc.); Output for the model; This indicates the expected operation.

[0108] Each production line node performs several rounds of local gradient descent updates, as shown in the following formula:

[0109] ;

[0110] in: For production line nodes The local learning rate; For gradient operators; This indicates the current local iteration round.

[0111] S104: Parameter Upload and Global Aggregation.

[0112] After completing local training, each production line node will update the parameters. and its gradient difference Uploaded to the central server.

[0113] The server assigns aggregation weights based on the sample size or data quality of each production line node. Perform a global weighted aggregation operation, as shown in the following formula:

[0114] ;

[0115] in: These are the global model parameters obtained through aggregation; For production line nodes Aggregate weight coefficient; This represents the number of production line nodes participating in the aggregation.

[0116] After aggregation, the server will distribute the new global model parameters to each production line node again, and enter the next round of training, realizing multiple rounds of interactive iteration.

[0117] During parameter synchronization, the central server and each production line node transmit model parameters via an encrypted communication channel. To improve communication efficiency and reduce bandwidth consumption, this invention can employ model parameter sparsity, gradient quantization, or Top-K selection strategies, uploading only the most important parameter updates. The system supports both synchronous and asynchronous communication modes: in synchronous mode, all production line nodes complete the upload and then aggregate the parameters; in asynchronous mode, the server can perform aggregation in advance after updates from some production line nodes arrive, thereby improving training efficiency and system fault tolerance.

[0118] S105: Training iteration and convergence determination.

[0119] The federated training process repeats steps S103 and S104 above until the convergence condition is met or the maximum number of rounds is reached. .

[0120] The convergence condition can be defined based on the rate of decrease in global model loss or the change in gradient variance, as shown in the following formula:

[0121] ;

[0122] in: This results in a global loss. This sets a preset convergence threshold. When the convergence condition is met, the final global model parameters are output. This serves as input for subsequent hierarchical aggregation and personalized optimization.

[0123] After completing the above federated learning process, this embodiment introduces a semantic hierarchical structure and an adaptive aggregation mechanism on the basis of global model aggregation: unified aggregation of general visual features is performed at the bottom layer (global layer); adaptive clustering aggregation based on feature and gradient similarity is performed at the middle layer; and fine-grained re-clustering and personalized optimization are performed at the top layer. This forms a complete hierarchical federated learning closed loop of Local update → Global aggregation → Hierarchical optimization.

[0124] S20: Based on global model parameters, construct global underlying model parameters and a unified feature space.

[0125] In each industrial production line node, the underlying network of the visual model is used to extract common visual features of product images, including low-level semantic features such as texture, structure, edges, and contours. The low-level feature parameters extracted from each industrial production line node are uploaded to a central server, where a globally unified aggregation mechanism is used for parameter averaging or weighted fusion. The goal of this low-level aggregation is to establish a shared visual foundation representation across production lines, providing a consistent feature space for subsequent mid-to-high-level feature learning.

[0126] Specifically, after completing the above federated learning training process, the system obtains the global model parameters. To further extract common visual features and achieve underlying semantic unification across production lines, this invention moves to the next stage of global feature modeling and underlying unified aggregation.

[0127] The distributed model employs a hierarchical coding structure based on the Visual Transformer (ViT). The model consists of an embedding layer, several self-attention encoding layers, and a task output layer. Based on semantic depth and feature abstraction, the model parameters are divided into three categories: bottom layer, middle layer, and top layer. The bottom layer is responsible for general visual feature extraction, the middle layer is used for industry-specific or process-level feature representation, and the top layer focuses on personalized optimization of product-level differentiated features. This hierarchical division provides a structural foundation for subsequent differentiated aggregation and multi-level optimization.

[0128] In some embodiments, such as Figure 3 As shown, step S20 is specifically implemented through the following steps S201-S204.

[0129] S201: General Feature Extraction.

[0130] In each distributed production line node, a Vision Transformer (ViT) is deployed as the underlying visual feature extraction model. This model divides the input image into fixed-size patches, performs linear embedding on each patch, and then adds positional embedding to preserve spatial relationship information. Subsequently, a multi-head self-attention mechanism is used to calculate the global dependencies between the patch images, thereby extracting general visual features of the product image. The general features extracted by this model include:

[0131] Texture characteristics: reflect surface roughness, graininess, and repeating patterns;

[0132] Structural features: Characterizing the overall shape and local structural layout of a product's appearance;

[0133] Contour features: Describe the boundary information and morphological change patterns of the target area.

[0134] The extracted feature parameters are denoted as the set of underlying model weights for each production line node. It includes the embedding layer weights, self-attention layer weights, and normalization coefficients of ViT, among which... Number the production line nodes. This indicates the lowest level of features.

[0135] S202: Feature parameter upload.

[0136] After completing local feature extraction and parameter updates, each production line node uploads the underlying model parameters to the central server. The uploaded content includes:

[0137] The underlying weight parameters of the ViT model include the embedding layer weights and the self-attention layer weights;

[0138] The corresponding gradient statistics, such as gradient mean and variance;

[0139] The number of production line node samples and data distribution information are used for aggregation weight allocation.

[0140] To improve communication efficiency, parameters can be sparsified, quantized, or compressed in blocks during the upload process to reduce transmission bandwidth overhead.

[0141] S203: Unified aggregation at the underlying level.

[0142] After receiving the underlying parameters from each production line node, the central server performs a global unified aggregation operation, using a weighted average method to merge the production line node parameters:

[0143] ;

[0144] in, These are the parameters of the global underlying model. The weighting coefficients for production line nodes are determined based on the sample size or data quality of the production line nodes, and satisfy the following conditions: The total number of production line nodes participating in the aggregation. For the first The underlying parameters of each production line node.

[0145] This aggregation mechanism integrates the differences between different production line nodes in the underlying feature space, thereby forming a unified underlying feature representation shared across production lines.

[0146] S204: Construction of a unified feature space.

[0147] After aggregation is complete, the central server will send the global underlying parameters. The data is distributed to each production line node, serving as the common feature extraction basis for subsequent mid- and high-level training. This step enables each production line node to learn within the same underlying feature space, effectively reducing feature shift and semantic conflicts caused by differences in data distribution.

[0148] S30: Based on the global underlying model parameters and the unified feature space, the shared model parameters of the mid-level clusters are obtained.

[0149] To address the visual and process differences between different production lines, the gradient direction similarity and feature distribution similarity of the model parameters for each production line node are calculated. A joint similarity matrix is ​​constructed to measure the semantic consistency among production line nodes. Subsequently, an adaptive clustering algorithm (such as spectral clustering or hierarchical clustering) is executed based on the similarity matrix to dynamically divide the production line into several semantically consistent sub-clusters. Within each sub-cluster, local parameter aggregation is performed to obtain a mid-level shared model with common industry or process characteristics.

[0150] Specifically, the middle layer performs adaptive clustering based on feature and gradient similarity. By jointly measuring the feature distribution and gradient direction of different production line nodes, semantically consistent clustering is achieved, thereby establishing a shared model at the industry or process level. This step S30 aims to solve the problem of inconsistent model update directions caused by differences in data distribution and process conditions between different production lines, ensuring that the model can adaptively perform clustering and aggregation in the middle layer representation stage.

[0151] In some embodiments, such as Figure 4 As shown, step S30 is specifically implemented through the following steps S301-S306.

[0152] S301: Feature and gradient information extraction.

[0153] After step S10 is completed, each production line node has obtained globally unified underlying model parameters. And based on this, further local data training continues. During the training process, production line nodes... New mid-level model parameters will be generated. and the corresponding gradient information .in, Indicates the production line node number; This represents the middle-level structure hierarchy of the model; Specifically, the first The parameter set of each production line node in the middle layer network; Indicates the first The gradient vector of each production line node on the intermediate layer parameters This is the local loss function for this production line node. These features and gradient information will serve as input to the clustering stage to determine the semantic similarity between production line nodes.

[0154] S302: Calculation of feature distribution similarity.

[0155] To characterize the visual semantic similarity between different production line nodes, this embodiment calculates the feature distribution similarity between production line nodes based on the distribution of mid-level feature vectors.

[0156] Specifically, production line nodes are set. With production line nodes The mean vectors of the middle layer features are respectively The covariance matrices are respectively Then, the Gaussian distribution distance is used to define the feature distribution similarity:

[0157] ;

[0158] in, Representing production line nodes The mean of the middle-layer features; Representing production line nodes The characteristic covariance matrix; Represents the trace operation of a matrix; Indicates production line node Similarity of feature distributions; The closer the value is to 1, the more similar the visual features are.

[0159] S303: Gradient direction similarity calculation.

[0160] To further measure the consistency of different production line nodes in the training direction, this embodiment calculates the gradient direction similarity between production line nodes:

[0161] ;

[0162] in: Let i and j represent the mid-level gradient vectors of production line node i and production line node j, respectively; the numerator is the gradient dot product, reflecting the consistency of direction; the denominator is the normalization term, used to eliminate differences in gradient magnitude. This represents the gradient direction similarity between production line node i and production line node j; The closer the value is to 1, the more consistent the update direction is.

[0163] S304: Construction of joint similarity matrix.

[0164] Taking into account both visual feature similarity and gradient direction consistency, this embodiment defines a joint similarity matrix:

[0165] ;

[0166] in: Indicates production line node With production line nodes Overall semantic similarity; These are weighting coefficients used to balance the importance of gradient similarity and feature similarity; when When the size is large, more emphasis is placed on consistency in training direction; when When the size is smaller, more emphasis is placed on visual semantic similarity. This is achieved through a matrix. The system can obtain the semantic association degree between all production line nodes, providing a foundation for subsequent clustering.

[0167] S305: Adaptive Clustering and Intra-cluster Aggregation.

[0168] Based on the joint similarity matrix, the production line nodes are divided into several semantically consistent sub-clusters using the adaptive spectral clustering algorithm. ,in This represents the number of clusters.

[0169] The production nodes within each sub-cluster exhibit high similarity in visual features and gradient directions. Based on this, local parameter aggregation is performed within the cluster:

[0170] ;

[0171] in: For the first The middle layer of each cluster shares model parameters; , which is the weight coefficient of the production line node within the cluster, and is usually related to the number of samples of the production line node or the gradient stability; Indicates the first A set of production line nodes for each cluster.

[0172] S306: Clustering results update and feedback.

[0173] After clustering is completed, the central server will share the intermediate model parameters of each cluster. The data is distributed to the corresponding production line node set for the next round of training. Simultaneously, it is periodically recalculated based on changes in the update direction of the production line nodes. This enables dynamic adjustment of clustering results, thereby ensuring the adaptability and stability of the clustering structure during training.

[0174] S40: Within each mid-level sub-cluster and its shared model parameters, perform high-level fine-grained re-clustering and personalized optimization. Perform the re-clustering operation and introduce a personalized parameter decomposition mechanism to build a personalized model for each node. Optimize the parameters by minimizing the personalized loss function to obtain the high-level personalized model parameters.

[0175] For each sub-cluster that has already been clustered, re-clustering and personalized parameter updates are performed at the high-level semantic layer to further refine the model structure. Adaptive learning rates and aggregation weights are introduced within each sub-cluster, and the parameter update ratio is adjusted based on the differences in tasks at production line nodes. This constructs a personalized model. ,in For cluster-shared parameters, These are fine-tuning parameters specific to production line nodes, used to enhance adaptability to specific product or production line characteristics.

[0176] Step S40 aims to achieve high-level fine-grained re-clustering and personalized optimization. After completing the mid-level adaptive clustering and intra-cluster parameter aggregation, this embodiment further performs fine-grained re-clustering and personalized parameter optimization at the high-level semantic level. This step aims to improve the model's adaptability to the specific characteristics of each production line, enabling the model to not only capture common information but also to make fine adjustments for the unique characteristics of individual production lines, thereby achieving layer-by-layer optimization from inter-class sharing to production line node-level personalization.

[0177] In some embodiments, such as Figure 5 As shown, step S40 can be implemented through the following steps S401-S404.

[0178] S401: High-level model initialization.

[0179] In the intermediate stage, the intermediate shared model parameters for each sub-cluster have been obtained. ,in Indicates the first There are several clusters. At the high-level stage, each sub-cluster uses its shared model as initial weights to establish a high-level personalized model: .in, This represents the high-level structural hierarchy of the model. Indicates production line node Initial parameters of the high-level model, Cluster The middle-level shared model parameters. This initialization ensures that the starting point for high-level optimizations has a consistent semantic basis, while providing a shared reference for personalized updates.

[0180] S402: High-level feature extraction and re-clustering.

[0181] During high-level training, each production line node continues to update its high-level feature representation based on local production line data, generating a production line node-level feature mean vector. With gradient vector To capture more subtle semantic differences between production lines within a cluster, this invention performs a fine-grained re-clustering operation on the production line nodes within the cluster. Specifically, in the cluster... Internal calculation of the high-level joint similarity matrix between production line nodes :

[0182] ;

[0183] in: This represents the similarity between production line node i and production line node j in the high-level gradient direction; Production line nodes With production line nodes High-level feature distribution similarity; : This is the weighting coefficient in high-level clustering, used to balance structural similarity and consistency of update direction.

[0184] Based on the matrix Perform intra-layer spectral clustering or hierarchical clustering operations to transform the original clusters. Decomposed into several fine-grained subclusters ,in For the first Number of re-clusterings within a cluster.

[0185] S403: Personalized parameter optimization.

[0186] After fine-grained sub-cluster partitioning is completed, this embodiment introduces a personalized parameter decomposition mechanism at each production line node. For each production line node... Its high-level model parameters consist of shared components and personalized bias components: .in: For production line nodes Complete high-level model parameters; These are shared parameters for the fine-grained sub-cluster to which this production line node belongs; This is a unique parameter bias specific to this production line node, used to learn the production line's unique characteristics.

[0187] During local training, production line nodes are updated by minimizing the following personalized loss function:

[0188] ;

[0189] in: Let be the production line task loss function, and be the classification cross-entropy. This is the personalized regularization coefficient, used to limit the magnitude of personalized bias; This is used to prevent overfitting and maintain the controllability of personalized updates. By optimizing the above loss function, both shared knowledge transfer and personalized adaptation of production line nodes can be taken into account simultaneously.

[0190] S404: Personalized aggregation and feedback.

[0191] After the high-level training concludes, each production line node will update its personalized parameters. The shared parameter gradient information is fed back to the central server. The central server performs personalized weighted aggregation within each fine-grained sub-cluster, updating the sub-cluster shared parameters.

[0192] ;

[0193] in: Indicates the first The first in the middle cluster A high-level fine-grained sub-cluster; The personalized aggregation weights for production line nodes are determined based on the number of samples in the production line nodes or the model convergence rate. (Updated) The data will be redistributed to the corresponding production line nodes, forming a personalized learning loop across different rounds.

[0194] S50: It uses the global bottom-level model parameters as the root node, the shared model parameters of the mid-level clusters as the intermediate nodes, and the personalized model parameters of the high-level layers as the leaf nodes to form a tree-like multi-level aggregation structure. It also introduces a dynamic aggregation weight update mechanism and an aggregation depth adjustment mechanism to achieve cross-layer adaptive optimization.

[0195] The model employs a tree-like multi-level aggregation structure, with the bottom-level global aggregation result as the root production line node, the middle-level sub-cluster models as intermediate production line nodes, and the high-level personalized models as leaf production line nodes. An adaptive weight update mechanism is introduced during training to dynamically adjust the aggregation depth and the scope of participation of production line nodes based on the convergence degree, gradient variance, or validation error of each layer of the model. This tree-like aggregation structure creates a top-down multi-level information flow between the global and local levels, achieving a balance between generality and personalization in the model.

[0196] Specifically, after completing high-level fine-grained re-clustering and personalized optimization, this embodiment further constructs a tree-like multi-level aggregation structure to achieve adaptive collaboration and dynamic optimization of the cross-layer model. This step aims to uniformly manage the hierarchical relationships between the bottom-level global features, the middle-level industry features, and the high-level personalized features. By introducing adaptive weights and a dynamic aggregation mechanism, it achieves efficient convergence and balanced optimization of the model in the multi-layer space.

[0197] In some embodiments, such as Figure 6 As shown, step S50 can be implemented through the following steps S501-S505.

[0198] S501: Construct a tree-like hierarchical structure.

[0199] In the preceding steps, the parameters of the three layers of the model have been obtained: the bottom-level global model parameters. ;Mid-level clusters share model parameters High-level personalized model parameters Based on this, this embodiment constructs a tree-like multi-level aggregation structure. Its formal definition is:

[0200] ;

[0201] Where: global model parameters As the root node, it represents the globally common feature layer; the middle-layer clusters share model parameters. As an intermediate node, it represents an industry or process layer aggregation production line node; and As leaf nodes, they represent the high-level sub-cluster shared model and the personalized model, respectively. This tree structure reflects the parameter inheritance and optimization path from general to industry to individual, achieving ordered transmission of hierarchical semantics.

[0202] S502: Interlayer aggregation and information transfer.

[0203] To maintain consistency across different model layers, a cross-layer aggregation mechanism is used. Its core idea is to achieve knowledge sharing and gradient feedback through bidirectional parameter transfer between layers. The formula is as follows:

[0204] ;

[0205] in: These are the parameters for the upper-level model; specifically, they are the global and mid-level parameters. Specifically, the parameters for the lower-level models are shared by the middle and higher levels. This is a weighted average of the node parameters of the sub-production line; The cross-layer information fusion coefficient controls the degree to which the upper layer absorbs feedback from the lower layer. This mechanism allows the upper-layer model to dynamically absorb feature distribution information learned from the lower layer during training, while maintaining the stability of global features.

[0206] S503: Dynamic aggregate weight update.

[0207] To address the differences in convergence speed and gradient variance among different model levels during training, this embodiment introduces a dynamic aggregate weight adjustment mechanism. Let the... The aggregation weight of the layer is Adaptive updates are performed based on the model gradient variance and convergence state:

[0208] ;

[0209] in: For production line nodes In the The gradient variance of a layer is used to reflect training stability; This is a temperature coefficient used to control variance sensitivity; In the updated aggregation weights, production nodes with smaller gradient variance receive higher aggregation weights. This adaptive weighting mechanism dynamically adjusts the parameter participation of different layers during training, thereby accelerating overall convergence and improving model stability.

[0210] S504: Adaptive adjustment of aggregation depth and range.

[0211] At different stages of model training, the differences between production line nodes may gradually converge or expand. To adapt to this dynamic change, this embodiment designs an aggregation depth adjustment mechanism to automatically adjust the number of aggregation layers based on the validation error or inter-layer gradient consistency index.

[0212] ;

[0213] in: For the first Aggregation depth (number of layers) during round training; This represents the global gradient variance for the current round. Convergence and Oscillation Thresholds. When the system detects that the gradients of each layer tend to be consistent (with small variance), the aggregation depth is automatically increased to achieve broader cross-layer knowledge fusion; conversely, when the differences widen, the aggregation range is contracted to maintain the local stability of the model.

[0214] S505: Dynamic optimization process.

[0215] Based on the above mechanism, the dynamic optimization process of the tree-like multi-level aggregation structure is as follows: Figure 7 As shown, the specific steps include S5051-S5055.

[0216] S5051: Each node continues local training based on the high-level personalized model to generate gradient information;

[0217] S5052: Perform weighted aggregation within each layer and update shared parameters within the layer;

[0218] S5053: Perform cross-layer parameter fusion to enable lower-layer knowledge to be fed back upwards;

[0219] S5054: Dynamically adjust aggregation weights and depth based on gradient variance;

[0220] S5055: Output the updated global model and parameters of each layer, and proceed to the next training iteration.

[0221] S60: Based on a tree-like multi-level aggregation structure, it performs cross-layer collaboration and convergence optimization. It achieves collaborative updates of model parameters at different levels through feature distillation and gradient consistency constraints, and dynamically adjusts the learning rate through an adaptive convergence control mechanism.

[0222] By employing cross-layer feature distillation or gradient constraints, information transfer and consistency constraints between lower and higher-level parameters are achieved. An adaptive learning rate adjustment mechanism is utilized in global iterations to ensure stable convergence of the model within a multi-layered aggregation structure. The final output is a globally optimized model and a customized model, achieving high stability and high generalization performance in heterogeneous data environments across production lines.

[0223] In some embodiments, such as Figure 8 As shown, step S60 can be implemented through the following steps S601-S603.

[0224] S601: Translayer characteristic distillation mechanism.

[0225] Because different model levels differ in semantic depth and feature resolution, this invention introduces a cross-layer feature distillation mechanism to enhance inter-layer consistency. In this mechanism, higher-level models act as teacher models, and lower-level models act as student models, achieving knowledge transfer by minimizing the differences in feature distribution between layers. The distillation loss function is defined as follows:

[0226] ;

[0227] in: The number of samples; For the high-level model to the first Feature representation extracted from each sample; This represents the feature representation of the same sample in the mid-level model; This represents the squared Euclidean distance. By minimizing... The mid-level model can absorb the abstraction capabilities of high-level semantic features, while the high-level model maintains the stability of the feature spaces of the bottom and mid-level layers, thus forming semantic synergy between the upper and lower layers.

[0228] S602: Gradient consistency constraint mechanism.

[0229] To further ensure the coordination of parameter update directions at different levels of the model, this invention introduces a gradient consistency regularization mechanism.

[0230] Let the first The average gradient in the parameter space of the layer model is Then the cross-layer gradient consistency constraint term is defined as:

[0231] ;

[0232] in: This represents the total number of levels in the model (bottom layer - middle layer - top layer). For the first The average gradient vector of the layer model; It reflects the consistency between the direction and magnitude of the gradients between layers. In the joint optimization process, the overall loss function is defined as: ;in: The main loss function for the task, such as classification or regression error; , These are weighted coefficients that control the importance of feature distillation and gradient constraints, respectively. By introducing this multi-level constraint into global training, the model can maintain consistent gradient flow directions across different levels, avoiding convergence conflicts and convergence oscillations.

[0233] S603: Adaptive convergence control mechanism.

[0234] To address the dynamic changes between multi-layer models during training, this embodiment further introduces an adaptive convergence control mechanism. This mechanism dynamically adjusts the learning rate by monitoring the cross-layer gradient variance and the loss descent rate. :

[0235] ;

[0236] in: For the first The learning rate for each iteration; This is the learning rate adjustment coefficient; The average gradient variance across layers; Changes in losses between adjacent rounds; These represent the convergence and oscillation thresholds, respectively. When the system detects a decrease in gradient oscillations and a continuous decrease in the loss function, it automatically slightly increases the learning rate to accelerate convergence; conversely, when the gradient variance is too large or the loss increases, it automatically decreases the learning rate to prevent oscillations.

[0237] Combining the above three mechanisms, this embodiment forms a complete cross-layer optimization closed-loop process: the high-level model transmits abstract semantic features to the middle layer (feature distillation); the middle-level model corrects the update direction of the bottom-level features through gradient consistency constraints; the global learning rate is adaptively adjusted according to the cross-layer gradient changes; after optimization, the parameters of each layer are synchronously updated to the corresponding production line nodes of the tree structure, and the next round of collaborative training begins.

[0238] This application also provides an adaptive hierarchical distributed learning system for heterogeneous industrial production line data, used to implement the methods described in any of the above embodiments, such as... Figure 9 As shown, the system includes:

[0239] The framework process construction module 901 is configured to build a federated learning framework and a global training process. The federated learning framework includes a central server and multiple production line nodes, and the global training process is used to determine global model parameters.

[0240] The first parameter determination module 902 is configured to construct global underlying model parameters and a unified feature space based on global model parameters;

[0241] The second parameter determination module 903 is configured to obtain the shared model parameters of the mid-level clusters based on the global bottom-level model parameters and the unified feature space.

[0242] The third parameter determination module 904 is configured to perform high-level fine-grained re-clustering and personalized optimization within each mid-level sub-cluster and its shared model parameters, perform re-clustering operations, introduce a personalized parameter decomposition mechanism, construct a personalized model for each node, optimize parameters by minimizing the personalized loss function, and obtain high-level personalized model parameters.

[0243] The aggregation structure building module 905 is configured to form a tree-like multi-level aggregation structure with global bottom-level model parameters as the root node, mid-level cluster shared model parameters as intermediate nodes, and high-level personalized model parameters as leaf nodes. It also introduces a dynamic aggregation weight update mechanism and an aggregation depth adjustment mechanism to achieve cross-layer adaptive optimization.

[0244] The cross-layer optimization module 906 is configured to perform cross-layer collaborative and convergent optimization based on a tree-like multi-level aggregation structure. It achieves collaborative updates of model parameters at different levels through feature distillation and gradient consistency constraints, and dynamically adjusts the learning rate through an adaptive convergence control mechanism.

[0245] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An adaptive hierarchical distributed learning method for heterogeneous industrial production line data, characterized in that, include: A federated learning framework and a global training process are constructed. The federated learning framework includes a central server and multiple production line nodes. The global training process is used to determine global model parameters. Based on global model parameters, construct global underlying model parameters and a unified feature space; Based on the global underlying model parameters and the unified feature space, the shared model parameters of the mid-level clusters are obtained; Within each mid-level sub-cluster and its shared model parameters, high-level fine-grained re-clustering and personalized optimization are performed. The re-clustering operation is executed, and a personalized parameter decomposition mechanism is introduced to construct a personalized model for each node. The parameters are optimized by minimizing the personalized loss function to obtain the high-level personalized model parameters. With the global bottom-level model parameters as the root node, the mid-level cluster shared model parameters as the intermediate nodes, and the high-level personalized model parameters as the leaf nodes, a tree-like multi-level aggregation structure is formed. A dynamic aggregation weight update mechanism and an aggregation depth adjustment mechanism are introduced to achieve cross-layer adaptive optimization. Based on a tree-like multi-level aggregation structure, cross-layer collaboration and convergence optimization are performed. Collaborative updates of model parameters at different levels are achieved through feature distillation and gradient consistency constraints. The learning rate is dynamically adjusted through an adaptive convergence control mechanism. Each production line node corresponds to a specific industrial production line and holds local production line images or sensor data for local model training and parameter updates. Based on global model parameters, the methods for constructing global underlying model parameters and a unified feature space include: on the basis of the obtained global model parameters, each production line node performs global general feature extraction and underlying unified aggregation, extracts the general visual features of the product image, uploads the underlying feature parameters to the central server, and uses a global unified aggregation mechanism to fuse the parameters, thereby constructing a global underlying model parameter and a unified feature space shared across production lines.

2. The adaptive hierarchical distributed learning method for heterogeneous industrial production line data as described in claim 1, characterized in that, The global training process includes having each production line node independently train the model using local private data, and uploading the updated model parameters to the central server. The central server receives and merges the parameter updates to complete the global model parameter update, and then distributes the updated global model parameters to each production line node.

3. The adaptive hierarchical distributed learning method for heterogeneous industrial production line data as described in claim 2, characterized in that, The global underlying model parameters are determined using the following formula: in, These are the parameters of the global underlying model. The weighting coefficients for production line nodes are determined based on the sample size or data quality of the production line nodes, and satisfy the following conditions: The total number of production line nodes participating in the aggregation. Number the production line nodes. For the first The underlying parameters of each production line node.

4. The adaptive hierarchical distributed learning method for heterogeneous industrial production line data as described in claim 1, characterized in that, Based on the global underlying model parameters and a unified feature space, the methods for obtaining shared model parameters for mid-level clusters include: Based on the global underlying model parameters and unified feature space, a mid-level adaptive clustering based on feature and gradient similarity is performed. The feature distribution similarity and gradient direction similarity between nodes are calculated to construct a joint similarity matrix. Based on the joint similarity matrix, an adaptive clustering algorithm is executed to divide the production line nodes into semantically consistent sub-clusters. Local parameter aggregation is performed within each sub-cluster to obtain the mid-level cluster shared model parameters.

5. The adaptive hierarchical distributed learning method for heterogeneous industrial production line data as described in claim 4, characterized in that, The formula for calculating the similarity of the feature distributions is: in, Representing production line nodes The mean of the middle-layer features; Representing production line nodes The characteristic covariance matrix; The trace operation of a matrix is ​​represented; exp represents an exponential function with the natural constant as the base. Indicates production line node Similarity of feature distributions; The formula for calculating the gradient direction similarity is: in: These represent the mid-level gradient vectors of production line node i and production line node j, respectively. This represents the gradient direction similarity between production line node i and production line node j; Based on the similarity of feature distributions and gradient directions, a joint similarity matrix is ​​constructed using the following formula: in, Indicates production line node With production line nodes Overall semantic similarity; These are weighting coefficients used to balance the importance of gradient similarity and feature similarity; Based on the joint similarity matrix, the production line nodes are divided into several semantically consistent subclusters, and local parameter aggregation is performed within each cluster: in, For the first The middle layer of each cluster shares model parameters; The weight coefficient of the production line node within the cluster; Indicates the first A set of production line nodes for each cluster; This represents the number of clusters. For the first The middle layer of each production line node shares model parameters.

6. The adaptive hierarchical distributed learning method for heterogeneous industrial production line data as described in claim 1, characterized in that, Within each mid-level sub-cluster and its shared model parameters, high-level fine-grained re-clustering and personalized optimization are performed. The re-clustering operation is executed, and a personalized parameter decomposition mechanism is introduced to construct a personalized model for each node. Parameter optimization is performed by minimizing the personalized loss function to obtain the high-level personalized model parameters, including: At the high-level stage, each sub-cluster uses its shared model as initial weights to establish a high-level personalized model: in, This represents the high-level structural hierarchy of the model. Indicates production line node Initial parameters of the high-level model, Indicates the first Cluster The middle layer shares model parameters; In cluster Internal calculation of the high-level joint similarity matrix between production line nodes: in, This represents the similarity between production line node i and production line node j in the high-level gradient direction; Production line nodes With production line nodes High-level feature distribution similarity; , which is the weight coefficient in high-level clustering, used to balance structural similarity and consistency of update direction; Based on the high-level joint similarity matrix, perform intra-layer spectral clustering or hierarchical clustering operations to group the clusters. Decomposed into several fine-grained subclusters; After fine-grained sub-cluster partitioning is completed, a personalized parameter decomposition mechanism is introduced at each production line node; whereby, for each production line node... Its high-level model parameters consist of a shared component and a personalized bias component, expressed as: in, For production line nodes High-level personalized model parameters; For production line nodes Shared parameters of the fine-grained sub-clusters; For production line nodes Personalized parameter bias; During local training, production line nodes minimize the following personalized loss function. Update: in, This represents the production line task loss function; This is the personalized regularization coefficient, used to limit the magnitude of personalized bias; Used to prevent overfitting; Each production line node will update the personalized parameter bias. The shared parameter gradient information is fed back to the central server, which then performs personalized weighted aggregation within each fine-grained sub-cluster, updating the sub-cluster shared parameters. in, Indicates the first The first in the middle cluster A high-level fine-grained sub-cluster; Personalized aggregation weights for production line nodes. For the updated production line nodes Shared parameters of the fine-grained sub-clusters; Will Redistribute to the corresponding production line node, and determine the production line node. High-level personalized model parameters.

7. The adaptive hierarchical distributed learning method for heterogeneous industrial production line data as described in claim 1, characterized in that, The dynamic aggregation weight update mechanism is expressed as follows: in, For production line nodes In the The gradient variance of a layer is used to reflect the training stability of that node during the training process of the current layer; This is a temperature coefficient used to control variance sensitivity; For the updated aggregation weights, production line nodes with smaller gradient variances receive higher aggregation weights; exp represents an exponential function with the natural constant as its base. The gradient variance of the j-th production line node participating in the aggregation at the same layer is used to calculate the denominator normalization term; The aggregation depth adjustment mechanism is expressed as follows: in For the first Aggregation depth during round training; For the first Aggregation depth during round training; This represents the global gradient variance for the current round. Convergence threshold and oscillation threshold.

8. The adaptive hierarchical distributed learning method for heterogeneous industrial production line data as described in claim 1, characterized in that, The loss function of the characteristic distillation is defined as: in, Characteristic distillation loss, The number of samples; For the high-level model to the first Feature representation extracted from each sample; This represents the feature representation of the same sample in the mid-level model; Represents the square of the Euclidean distance; The gradient consistency constraint is expressed as follows: In the formula, This represents the total number of levels in the model. and For the first and The average gradient vector of the +1 layer model; The loss value reflects the degree of consistency between the direction and magnitude of the interlayer gradient; The adaptive convergence control mechanism dynamically adjusts the learning rate by monitoring the cross-layer gradient variance and the loss descent rate. in, and For the first Wheel and the first +1 round of learning rate; This is the learning rate adjustment coefficient; The average gradient variance across layers; Changes in losses between adjacent rounds; These are the convergence and oscillation thresholds, respectively.

9. An adaptive hierarchical distributed learning system for heterogeneous data in industrial production lines, characterized in that, The system for implementing the method as described in any one of claims 1 to 8 comprises: The framework process construction module is configured to build a federated learning framework and a global training process. The federated learning framework includes a central server and multiple production line nodes, and the global training process is used to determine global model parameters. The first parameter determination module is configured to construct global underlying model parameters and a unified feature space based on global model parameters; The second parameter determination module is configured to obtain the shared model parameters of the mid-level clusters based on the global bottom-level model parameters and the unified feature space. The third parameter determination module is configured to perform high-level fine-grained re-clustering and personalized optimization within each mid-level sub-cluster and its shared model parameters. It performs re-clustering operations and introduces a personalized parameter decomposition mechanism to build a personalized model for each node. It optimizes parameters by minimizing the personalized loss function to obtain the high-level personalized model parameters. The aggregation structure building module is configured to form a tree-like multi-level aggregation structure with global bottom-level model parameters as the root node, mid-level cluster shared model parameters as intermediate nodes, and high-level personalized model parameters as leaf nodes. It also introduces a dynamic aggregation weight update mechanism and an aggregation depth adjustment mechanism to achieve cross-layer adaptive optimization. The cross-layer optimization module is configured to perform cross-layer collaborative and convergent optimization based on a tree-like multi-level aggregation structure. It achieves collaborative updates of model parameters at different levels through feature distillation and gradient consistency constraints, and dynamically adjusts the learning rate through an adaptive convergence control mechanism.

Images