An unsupervised model field adaptation method for plant-level knowledge security migration

Abstract

The application discloses a kind of factory-level knowledge security migration-oriented unsupervised model field self-adapting method, it is applied to target domain factory to train target domain model, it includes: the source domain model of source domain factory is obtained, and its feature extractor and classifier are obtained by training using labeled source domain dataset;Using source domain model initializes the feature extractor and classifier of target domain model;Using the feature extractor of source domain model calculates the feature of target domain sample, and constructs feature library;The parameter of the classifier of target domain model is fixed, and the feature library is used to train target domain model in batches;Wherein, after each batch training updates model parameter, the feature of current training batch sample is recalculated using the feature extractor of current target domain model, updates feature library, and then the feature library of next batch sample is used to train target domain model.The application realizes high-precision field alignment under the condition that source domain data is not needed.

Description

Technical Field

[0001] This invention belongs to the field of neural network technology, specifically relating to an unsupervised model domain adaptation method for secure transfer of factory-level knowledge. Background Technology

[0002] Wheel type recognition is a crucial step in the manufacturing process of aluminum alloy wheels. In a mixed-line production and packaging environment, the wheel type recognition result is sent to the robotic arm to retrieve the corresponding gripping formula, thereby completing the gripping and placement of the wheel for subsequent processes. Incorrect wheel type recognition can lead to a mismatch between the wheel and the gripping formula, resulting in incorrect gripping positions and serious safety incidents such as wheel detachment or entanglement between the robotic arm and the wheel bracket. Therefore, accurate wheel type recognition is of paramount importance for the efficient and stable production of industrial systems.

[0003] As a branch of machine learning, deep learning technology is widely used in industrial manufacturing. Generally, stringent industrial production standards and low fault tolerance require a large amount of labeled data for training to ensure model reliability. However, due to factors such as actual production conditions and data acquisition environments, the data distribution collected from different sub-factories of the same manufacturer often exhibits significant domain differences. This makes it difficult to reuse a model trained in one factory across multiple factories. Consequently, other factories also need to bear the high time and manpower costs of data acquisition and labeling, raising the application threshold of deep learning technology. Unsupervised Domain Adaptation (UDA) is an effective way to solve the above problems. It achieves class distribution alignment by reducing the difference between the distribution of unlabeled samples in the target domain and samples in the source domain, thereby transferring rich knowledge from the source domain to the target domain for recognition tasks. Reference [1] proposes an optimization objective that combines diversity maximization and entropy minimization, replacing the traditional adversarial learning and achieving the best performance; Liu et al. proposed a progressive multi-source domain adaptive network (PMSDAN), which integrates multiple source domains into an auxiliary domain to match the distribution of the target domain, reducing the distribution difference between the multi-source domain and the target domain [2]. In the industrial field, Huang et al. [3] used adversarial learning to learn domain-invariant features in multi-domain transfer fault diagnosis and applied them to automotive transmission health monitoring; Chen et al. proposed an open-set DA network and introduced a weighted learning strategy to solve the open-set diagnosis problem of rotary bearing faults under multi-source conditions [4].

[0004] These methods are typically based on the fundamental assumption that source domain data is accessible. However, on the one hand, in the wheel manufacturing process, the storage volume of well-labeled source domain images is enormous. For example, one wheel manufacturer accumulated terabytes of data over 20 days of operation. In resource-constrained factory environments, high-quality and efficient data transmission and application face challenges. On the other hand, with the increasing digitalization and intelligence of industrial production, the importance of data security and privacy protection is constantly growing, making data confidentiality an indispensable task for enterprises. Therefore, data from different factories is usually independent, with only the model being accessible. Traditional learning paradigms based on data alignment, feature alignment, or cross-domain adversarial learning are difficult to implement effectively. How to find the optimization objective of the model using only a well-trained model from the source domain and unlabeled data from the target domain to reduce domain differences and achieve good model transfer has become a key problem in wheel manufacturing. Summary of the Invention

[0005] To address the practical problems of high data annotation costs and difficulties in cross-factory knowledge transfer in the existing large-scale wheel hub manufacturing process, and considering the privacy protection issues of networked collaborative manufacturing, this invention provides an unsupervised model domain adaptation method for secure transfer of factory-level knowledge, achieving high-precision domain alignment without the need for source domain data.

[0006] To achieve the above technical objectives, the present invention adopts the following technical solution:

[0007] An unsupervised model domain adaptation method for secure knowledge transfer at the factory level is applied to train a target domain model in a factory, including:

[0008] Step 1: Obtain the source domain model of the source domain factory; wherein, the source domain model is trained using a labeled source domain dataset and includes a feature extractor and a classifier.

[0009] Step 2: Initialize the feature extractor and classifier of the target domain model using the feature extractor and classifier of the source domain model, respectively.

[0010] Step 3: Use the feature extractor of the source domain model to calculate the features of each target domain sample and build a complete feature library;

[0011] Step 4: Fix the parameters of the classifier for the target domain model and train the target domain model in batches using the feature library;

[0012] Specifically, after updating the model parameters in each batch of training, the features of the current training batch of samples are recalculated using the feature extractor of the current target domain model, the feature library is updated, and then the target domain model is trained using the feature library of the next batch of samples.

[0013] Furthermore, the loss for batch training of the target domain model includes single-class deterministic loss. The loss is measured using the information entropy loss of the probability output for each category:

[0014]

[0015] Where p is the output probability vector of sample x, p∈R C C represents the number of categories; p j Let p be the j-th element, representing the probability that sample x belongs to the j-th class;

[0016] By minimizing the information entropy of the predicted probabilities of each category, the target domain model can output deterministic values ​​for each category, i.e., in the form of one-hot encoding.

[0017] Furthermore, one of the losses for batch training of the target domain model is measured using the full-class diversity loss;

[0018] First, calculate the average probability output p of all target domain samples. avg :

[0019]

[0020] Where, n t p represents the number of all samples in the target domain. i The output representing the classification probability of the i-th sample is a vector consisting of the probabilities corresponding to the C categories;

[0021] Then, information entropy is calculated as the diversity loss across all categories.

[0022]

[0023] Furthermore, the loss for batch training of the target domain model includes the K-nearest neighbor contrastive loss. The calculation formula is:

[0024]

[0025] k + {·}=KNN(q,bank fea )

[0026] k - {·}=KNN - (q, bank) fea )

[0027] Where q represents the current query sample being trained, and KNN represents the K-nearest neighbor operator. - This represents the K-nearest neighbor inverse operator, used to find the K furthest samples; k+ {·} represents the set of positive examples, k i+ Represents the set of positive examples k + The i-th sample in {·}, k - {·} represents the set of negative examples, k j- Represents the set of negative examples k - The j-th sample in {·}; bank fea This refers to the feature library.

[0028] Furthermore, the total loss of the target domain model trained in batches for:

[0029]

[0030] in, Represents single-class deterministic loss. Represents the loss of diversity across all categories. Represents the K-nearest neighbor contrastive loss; p is the output probability vector of sample x, p∈R C C represents the number of categories; p j Let p be the j-th element, representing the probability that sample x belongs to the j-th class; p avg The output probability is the average of all target domain samples; q represents the query sample currently being trained, and k... i+ Represents the set of positive examples k + The i-th sample in {·}, k j- Represents the set of negative examples k - The j-th sample in {·}; K represents the number of positive or negative samples found using the K-nearest neighbor algorithm.

[0031] Furthermore, during the process of updating the model parameters after each batch of training, the feature extractor is updated based on momentum:

[0032]

[0033] Where α is a hyperparameter, This represents the feature extractor obtained during the k-th iteration of training. This represents the feature extractor obtained from the (k-1)th and kth iterations and the update.

[0034] Furthermore, K-nearest neighbor comparison loss The value of K in the model gradually increases with the number of training iterations:

[0035] K = K0 + λ k ·k

[0036] Where K0 is the initial value, λ k This represents the growth rate during the k-th training iteration, where k represents the current training iteration number.

[0037] Furthermore, the source domain model and the target domain model are used to perform type recognition of the wheel types produced by their respective factories based on the input image.

[0038] Beneficial effects

[0039] This invention provides an unsupervised model domain adaptation method for secure knowledge transfer at the factory level. Utilizing single-class deterministic loss, global diversity loss, and K-nearest neighbor comparison loss, it achieves high-precision domain alignment without requiring source domain data. This invention addresses the problems of high data labeling costs due to domain differences and potential data security challenges under traditional transfer paradigms in large-scale wheel hub production. In the embodiments of this invention, its application in the practical industrial case of wheel hub production verifies the superiority of the method. In the future, this method is expected to be widely applied to other tasks in industrial manufacturing. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the overall framework of the method described in the embodiments of this application.

[0041] Figure 2 This is a data illustration of an embodiment of this application applied to an industrial wheel hub case, where (a) shows five typical wheels collected from Factory No. 1, and (b) shows five typical wheel hubs collected from Factory No. 2. Detailed Implementation

[0042] The embodiments of the present invention will be described in detail below. These embodiments are based on the technical solutions of the present invention and provide detailed implementation methods and specific operation processes to further explain the technical solutions of the present invention.

[0043] This invention addresses the practical problems of high data annotation costs and difficulties in cross-plant knowledge transfer during large-scale wheel manufacturing, while also considering privacy protection issues in networked collaborative manufacturing. It proposes a framework for a secure unsupervised model domain adaptation method (SUMAF) for secure plant-level knowledge transfer. The following analysis outlines the approach proposed in this invention: First, the model optimization objective under this specific problem setting is analyzed; then, corresponding loss functions are proposed for each optimization objective; finally, a progressive nearest neighbor selection strategy is used during the model training phase to ensure the accuracy of early nearest neighbor selection.

[0044] 1. Problem Setting

[0045] In traditional unsupervised domain adaptation tasks, the source domain dataset can usually be obtained. All data is labeled, and the target domain dataset can be represented as follows: All data are unlabeled. Under the assumption of closed sets, the source and target domains contain the same categories, but their data distributions differ. This task consists of two steps: first, training a source domain model using labeled data from the source domain. s Then, based on the source domain data, the source domain model, and the target domain data, the target domain model is trained to obtain the target domain model. t In mathematics:

[0046]

[0047]

[0048] Unlike traditional unsupervised domain adaptation, this method involves accessing only the source domain model when the source domain data is inaccessible. If domain migration is achieved using only the source domain model, not only can the amount of data transmitted be significantly reduced, but the privacy and security of data from each plant can also be fully protected. Therefore, unlike traditional unsupervised domain adaptation, unsupervised domain adaptation without source data can be represented as follows:

[0049]

[0050] Among them, f s With c s f represents the feature extractor and classifier of the source domain model, respectively. t With c t These represent the feature extractor and classifier of the target domain model, respectively.

[0051] 2. Optimize target analysis

[0052] Since traditional paradigms based on data / feature alignment or adversarial learning have failed, it is first necessary to clarify the optimization objective, i.e., how to optimize the model to reduce domain discrepancies. This invention starts with the mathematical expression output by the Softmax classifier, which states that if the label y of a sample... i If k = , then we want to maximize the model output:

[0053]

[0054] Among them, model (k) f(x) represents the value of the k-th dimension element of the model output, where f(x) ∈ R. d×1 The representation of sample x is obtained by the feature extractor, where d is the dimension of the output features of the feature extractor. k The weights W = {w1, w2, ..., w} represent the weights of the linear classifier c(·). K}∈R d×K The k-th weight vector. It is easy to see that maximizing the above formula is equivalent to maximizing the sum of the features and weights w. kThe weights of a well-trained classifier are the class prototypes in the feature space, while minimizing the distance between the weights and the other K-1 weights. In other words, the weights of a well-trained classifier are the class prototypes in the feature space.

[0055] Therefore, assuming a well-trained source domain model, if its classifier can also classify target domain features well, it is equivalent to indirectly achieving feature alignment between the source and target domains; that is, the target domain features are embedded in the class prototype space of the source domain. Inspired by this, this embodiment achieves indirect feature alignment by fixing the classifier and optimizing the feature extractor. Under this premise, if the source domain classifier can classify the target domain data well, then the model should meet the following three requirements:

[0056] (1) The target domain features should produce a class-deterministic output after passing through the source domain classifier;

[0057] (2) Under the assumption of class balance in this embodiment, the classification results of the target domain should be as diverse as possible to avoid all predictions from converging to the same category.

[0058] (3) Incorrect alignment between categories should be avoided as much as possible.

[0059] It is worth noting that, in theory, the three objectives are complementary and indispensable. If objective (1) is missing, the model cannot obtain a definite classification result; if objective (2) is missing, the model's prediction results may all collapse to the same category; if objective (3) is missing, the model may produce incorrect alignment.

[0060] 3. Optimize the framework

[0061] As mentioned earlier, in order to achieve good classification of the target domain data by the classifier, this embodiment fixes the classifier and proposes single-class deterministic loss, all-class diversity loss, and nearest neighbor comparison loss according to the above optimization objectives to optimize the feature extractor. The overall framework is as follows: Figure 1 As shown, a detailed introduction follows.

[0062] For objective one, in order to generate a single-category deterministic output, inspired by [5], this embodiment uses the information entropy of the probability output of each category as the loss function. Consider a C classification problem, where the output probability vector of sample x is p∈R C Then the mathematical expression for information entropy loss is:

[0063]

[0064] Where p iLet p be the i-th element, representing the probability that the sample belongs to the i-th class. Since the prediction result of a classification problem can be considered as a discrete information source containing only 0-1, according to the "deterministic" property of discrete information source information entropy, when the probability is 0 or 1, it is a deterministic state, and its information entropy is 0. Therefore, by minimizing the information entropy of the predicted probabilities of each class, so that the vector p contains as little uncertain information as possible, we can obtain the deterministic output of each class of the model, i.e., the form of one-hot encoding.

[0065] For objective two, to ensure the model produces diverse predictions and avoids all target domain samples being predicted as the same category, this embodiment also proposes a class-wide diversity loss based on the information entropy function. Specifically, firstly, the average probability output p of all target domain samples is calculated. avg :

[0066]

[0067] Where, n t p represents the number of all samples in the target domain. i The output representing the classification probability of the i-th sample is a vector consisting of the probabilities corresponding to the C categories;

[0068] Since we don't want the model to predict all samples as belonging to the same category, the average probability output should contain as much information as possible. Therefore, we use negative information entropy as the loss function:

[0069]

[0070] To illustrate with an extreme case, when the average probability output of the C classification model is p avg When the probability distribution is [1 / C, 1 / C, ..., 1 / C], the distribution of predicted probabilities is balanced, resulting in the highest information entropy and the lowest diversity loss. However, when all model outputs belong to the same category, p... avg This is equivalent to one-hot encoding, where the information entropy is 0, and the diversity loss is correspondingly the highest. It's worth noting that this loss relies on the assumption of a balanced distribution of the true classes in the target domain. When the distribution itself is severely imbalanced, this loss is difficult to optimize, or overfitting can lead to incorrect alignment of samples.

[0071] Regarding objective three, while the model produces deterministic and diverse outputs under the combined effect of the two aforementioned losses, this is insufficient to handle inter-domain misalignment. To address this issue, this embodiment proposes nearest neighbor contrastive learning to mine instance-level associations, thereby mitigating potential misalignment.

[0072] In this embodiment, samples with high correlation are brought closer together, while samples with low correlation are pushed away as proxy tasks, and the model is optimized using InfoNCE loss[6]. Specifically, before training begins, the features of each target domain sample are calculated using the source domain feature extractor to construct a complete feature bank.

[0073]

[0074] Then, during the training of each batch, in order to determine the positive and negative samples, the K nearest neighbors are used to calculate the K closest samples to the current sample as positive samples and the K furthest samples as negative samples.

[0075]

[0076] Where q represents the current query sample being trained, and KNN represents the K-nearest neighbor operator. - This represents the K-nearest neighbor inverse operator, used to find the K furthest samples; k + {·} represents the set of positive examples, k - {·} represents the set of negative examples. Since the InfoNCEloss numerator contains only one positive sample, the K-nearest neighbor contrastive loss is rewritten as follows:

[0077]

[0078] Where k i+ ∈k + {·},k j- ∈k - {·}.

[0079] After updating the model using samples from the current batch, the sample features for that batch in the feature library need to be updated. Since the samples in each training batch are randomly selected, the indices in the sample feature library need to be recorded during the implementation. Note that all functions remain unchanged except for the update functionality for the current batch. This queue-based update can lead to an inconsistent feature library because different models compute different features. Therefore, we also use momentum-based updates during model updates to maintain the consistency of the feature library:

[0080]

[0081] in Let K represent the feature extractor for the k-th iteration of the target domain, and α be a hyperparameter. Furthermore, the choice of K is also noteworthy. If its value is small, the model's learning effect is limited, while if its value is high, it will consume more computational resources. In addition, due to the existence of domain differences and the unknown distribution of the target domain, the model may produce unreliable outputs, and higher values ​​may lead to incorrect constraints. Therefore, in this embodiment, the value of K is gradually increased as the model trains, aiming to ensure the accuracy of selecting nearest neighbor samples in the early stages of training.

[0082] K = K0 + λ k ·k (11)

[0083] Where, λ k Let represent the growth rate of the k-th training iteration, where k represents the current training iteration number. Therefore, the total loss of the model training is:

[0084]

[0085] The detailed steps of the method framework in this embodiment are as described in Algorithm 1.

[0086]

[0087] The manufacturer's actual images show five identical wheel models from two subsidiary factories; specific images are as follows. Figure 2 As shown, subgraphs (a) and (b) represent data from two sub-factories, with the corresponding letters representing the same wheel type.

[0088] The experiment sets up two domain adaptation tasks: migration from factory #1 to factory #2 and migration from factory #2 to factory #1, denoted as #1->#2 and #2->#1, respectively. Factory #1 has 100 images per class, and factory #2 has 150 images per class. Each image has a resolution of 1700×1700 pixels and includes RGB channels. The training and test sets are divided in an 8:2 ratio. The methods MMD and ADDA are compared under source domain data participation conditions, with supervised training (target-supervised) and direct transfer (source-only) used as upper and lower bounds for performance comparison.

[0089] The experimental results are shown in Table 1. The method of this invention achieved 100% performance in both tasks, and on industrial data, it met or even surpassed the performance of target-supervised training and active domain data participation training (ADDA) methods. MMD is a relatively weak constraint for large amounts of data, and this disadvantage is more significant when the intra-class variance is large. In addition, the experiment also verified that the method of this invention has low sensitivity and high robustness to inter-domain data imbalance.

[0090] Table 1. Comparative Experiment Results of Industrial Cases

[0091]

[0092] The above embodiments are preferred embodiments of this application. Those skilled in the art can make various changes or improvements based on them. Without departing from the overall concept of this application, these changes or improvements should fall within the scope of protection claimed in this application.

[0093] References:

[0094] [1]X.Wu, S.Zhang, Q.Zhou, Z.Yang, C.Zhao and LJLatecki, "EntropyMinimization Versus Diversity Maximization for DomainAdaptation," in IEEETransactions on Neural Networks and Learning Systems, vol.34, no.6, pp.2896-2907, June 2023,doi:10.1109 / TNNLS.2021.3110109.

[0095] [2] 2023,doi:10.1109 / TNNLS.2021.3110109.

[0096] [3]Huang R, Li J, Liao Y, et al.Deep adversarial capsule network for compound fault diagnosis of machinery toward multidomain generalization task[J]. IEEE Transactions on Instrumentation and Measurement, 2020,70:1-11.

[0097] [4]Chen Z,Liao Y,Li J,et al.Amulti-source weighted deep transfernetwork for open-set fault diagnosis of rotary machinery[J].IEEE Transactionson Cybernetics,2022,53(3):1982-1993.

[0098] [5]Hu W,Miyato T,Tokui S,et al.Learning discrete representations viainformation maximizing self-augmented training[C] / / International Conferenceon Machine Learning.PMLR,2017:1558-1567.

[0099] [6]He,Kaiming,et al."Momentum contrast for unsupervised visualrepresentation learning."Proceedings of the IEEE / CVF conference on computervision and pattern recognition.2020.

Claims

1. An unsupervised model-based domain adaptation method for secure transfer of factory-level knowledge, characterized in that, The target domain factory is used to train the target domain model, including: Step 1: Obtain the source domain model of the source domain factory; wherein, the source domain model is trained using a labeled source domain dataset and includes a feature extractor and a classifier; the source domain model and the target domain model are used to identify the type of wheel produced by their respective factories based on the input image. Step 2: Initialize the feature extractor and classifier of the target domain model using the feature extractor and classifier of the source domain model, respectively. Step 3: Use the feature extractor of the source domain model to calculate the features of each target domain sample and build a complete feature library; Step 4: Fix the parameters of the classifier for the target domain model and train the target domain model in batches using the feature library; The total loss of the target domain model trained in batches for: ； in, Represents single-class deterministic loss. Represents the loss of diversity across all categories. Represents the K-nearest neighbor comparison loss; For the sample The output probability vector, C represents the number of categories; for The i-th element represents the sample. The probability of belonging to the i-th class; q represents the average of the probability outputs for all target domain samples; q represents the query sample currently being trained. Represents the set of positive examples The i-th sample in the series, Represents the set of negative examples The j-th sample in the dataset; K represents the number of positive or negative samples found using the K-nearest neighbor algorithm; Specifically, after updating the model parameters in each batch of training, the features of the current training batch of samples are recalculated using the feature extractor of the current target domain model, the feature library is updated, and then the target domain model is trained using the feature library of the next batch of samples.

2. The unsupervised model domain adaptation method for secure transfer of factory-level knowledge as described in claim 1, characterized in that, The loss for batch training of the target domain model includes single-class deterministic loss. The information entropy loss of each category's probability output is used as the metric: ； in, For the sample The output probability vector, C represents the number of categories; for The j-th element represents the sample The probability of belonging to the j-th class; By minimizing the information entropy of the predicted probabilities of each category, the target domain model can output deterministic values ​​for each category, i.e., in the form of one-hot encoding.

3. The unsupervised model domain adaptation method for secure transfer of factory-level knowledge as described in claim 1, characterized in that, One of the losses for batch training of the target domain model is to use the full-class diversity loss as a measure; First, calculate the average of the probability outputs for all samples in the target domain. : ； in, This represents the number of all samples in the target domain. The output representing the classification probability of the i-th sample is a vector consisting of the probabilities corresponding to the C categories; Then, information entropy is calculated as the diversity loss across all categories. : 。 4. The unsupervised model domain adaptation method for secure transfer of factory-level knowledge according to claim 1, characterized in that, The loss for batch training of the target domain model includes K-nearest neighbor comparison loss. The calculation formula is: ； ； Where q represents the query sample currently being trained. Represents the K-nearest neighbor operator. This represents the K-nearest neighbor inverse operator, used to find the K furthest samples. Represents the set of positive examples. Represents the set of positive examples The i-th sample in the series, Represents the set of negative examples. Represents the set of negative examples The j-th sample in; This refers to the feature library.

5. The unsupervised model domain adaptation method for secure transfer of factory-level knowledge as described in claim 1 or 4, characterized in that, During the process of updating the model parameters after each batch of training, the feature extractor is updated based on momentum: ； in, It's a hyperparameter. This represents the feature extractor obtained during the k-th iteration of training. This represents the feature extractor obtained from the (k-1)th and kth iterations and the update.

6. The unsupervised model domain adaptation method for secure transfer of factory-level knowledge as described in claim 1 or 4, characterized in that, K-nearest neighbor comparison loss The value of K in the model gradually increases with the number of training iterations: ； Where K0 is the initial value, This represents the growth rate of the k-th training iteration, where k represents the current training iteration number.

Images