Small sample diagnostic capability enhancement method based on Bayesian meta-knowledge extraction and embedding

Abstract

The present invention provides a method for enhancing the diagnostic capability of a small sample based on Bayesian meta-knowledge extraction and embedding, and belongs to the technical field of electrical data processing. The method comprises: step 1: extracting the intrinsic structural feature vector of a labeled data set of a target task of an electromechanical product through a feature extraction network; step 2: using the conjugate prior of the mean and covariance matrix of the multivariate normal distribution N through Bayesian meta-learning, selecting the normal-inverse-Wachter distribution as the prior of the mean and covariance matrix, and obtaining the posterior probability of the Bayesian secondary classifier; step 3: using the posterior probability to instantiate the Bayesian secondary classifier and output the final diagnostic model. The present invention introduces a conjugate prior through Bayesian meta-learning, performs posterior inference, and enables the classifier to acquire cross-task knowledge, enhance fitting ability, and avoid overfitting.

Description

Technical Field

[0001] The invention relates to a small sample diagnostic capability enhancement method based on Bayesian meta-knowledge extraction and embedding, and belongs to the technical field of electrical data processing. Background Art

[0002] In the development of fault diagnosis models, monitoring data sets from actual engineering often consist of a large amount of normal state data and a small amount of fault data, and there is also a significant imbalance between fault categories. This non-ideal data condition will have an adverse effect on the feature extraction and classification capabilities of the diagnostic model, making it impossible for most deep learning models to take advantage of data-driven. Under such data conditions, effective knowledge must be introduced from outside the information provided by the current data set in order to achieve a credible improvement in the ability of the diagnostic model. Domain priors and similar task information are two relatively easy sources of knowledge to obtain. Two traditional solutions are to pre-train and fine-tune historical tasks in a supervised learning manner, and to use historical task models for migration. However, these two methods tend to over-optimize the training of feature extraction networks in the direction of distinguishing differences between sample categories in the training data, which is not better for generalization on unseen categories.

[0003] Self-Supervised Learning (SSL) is a learning method that uses unlabeled data to learn effective representations by setting different pretext tasks (also called proxy tasks). It can be used as a feature extraction network for downstream tasks, or trained as an auxiliary task to improve the performance of the main task. At the same time, in order to cope with data scarcity, complex working conditions, and time-consuming modeling, meta-learning methods can be applied to the field of small sample fault diagnosis.

[0004] When self-supervised learning is performed in unlabeled data, imbalanced data categories may cause the representation learned by self-supervision to lack the capture of minority samples, which further affects the downstream diagnosis tasks. The core difference between transductive learning and traditional inductive learning is that transductive learning allows the model to directly access unlabeled test data during model training. This method can obtain additional information from the distribution of unlabeled test data, so it has a natural advantage in small sample classification problems. This setting method does not meet the application scenario of online fault diagnosis, but is suitable for tasks of annotating a large amount of unlabeled historical data. Most transductive learning methods are based on distance metrics and label propagation in various feature spaces, which can be well combined with metric-based meta-learning methods. A simple and efficient transductive meta-learning method is to combine the soft k-means algorithm with the prototypical network to derive class prototypes in a semi-supervised learning manner. However, this method ignores the distribution uncertainty of the data domain between different tasks, resulting in excessive confidence in unlabeled samples in domain-deviated tasks, which reduces the generalization ability of class prototypes.

[0005] Traditional meta-learning usually chooses linear classifiers. Due to the extremely limited number of labeled samples in current diagnostic tasks, classifiers often face scenarios where the sample feature dimension is greater than the number of labeled samples available for training, making it very easy to overfit. However, the wide range of data domain deviations between cross-task small sample diagnostic tasks determines that no matter what method is used, the upper limit of its feature expression ability is definitely lower than the single data domain of large samples, resulting in insufficient fitting ability when using a simple linear classifier, and unable to achieve better accuracy. Summary of the invention

[0006] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a method for enhancing the diagnostic ability of small samples based on Bayesian meta-knowledge extraction and embedding. Through Bayesian meta-learning, conjugate priors are introduced and posterior inference is performed, so that the Bayesian quadratic classifier can acquire cross-task knowledge, enhance the fitting ability, and avoid overfitting.

[0007] To achieve the above-mentioned object of the invention, the present invention provides a method for enhancing the diagnostic capability of small samples based on Bayesian meta-knowledge extraction and embedding, which comprises the following steps:

[0008] Step 1: Through feature extraction network Extracting labeled dataset D for electromechanical product target tasks target The intrinsic structural feature vector ,in, is the optimal parameter of the feature extraction network, ;

[0009] Step 2: Obtain a Bayesian quadratic classifier by learning the conjugate priors of the mean and covariance matrix of the multivariate normal distribution N through Bayesian meta-learning, selecting the normal-inverse-Weichter distribution as the prior of the mean and covariance matrix The posterior probability of the parameter , where They are the four parameters of the normal-inverse-Weichter distribution; and They are the labeled dataset vectors D of the target tasks respectively. target The mean and covariance matrix of ;

[0010] Step 3: Using the Posterior Probability , for the Bayesian quadratic classifier Instantiate and output the final diagnostic model .

[0011] Compared with the prior art, the small sample diagnostic capability enhancement method based on Bayesian meta-knowledge extraction and embedding provided by the present invention has the following beneficial effects: through Bayesian meta-learning, conjugate priors are introduced and posterior inference is performed, so that the Bayesian quadratic classifier can acquire cross-task knowledge, enhance the fitting capability, and avoid overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flow chart of a method for enhancing the diagnostic capability of small samples based on Bayesian meta-knowledge extraction and embedding provided by the present invention; DETAILED DESCRIPTION

[0013] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0014] In the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance.

[0015] Figure 1 is a flow chart of a method for enhancing the diagnostic capability of a small sample based on Bayesian meta-knowledge extraction and embedding provided by the present invention, such as Figure 1 As shown, the small sample diagnostic capability enhancement method based on Bayesian meta-knowledge extraction and embedding provided by the present invention comprises the following steps:

[0016] Step 1: Through feature extraction network Extracting labeled dataset D for electromechanical product target tasks target The intrinsic structural feature vector ,in, is the optimal parameter of the feature extraction network, ;

[0017] Step 2: Obtain a Bayesian quadratic classifier by learning the conjugate priors of the mean and covariance matrix of the multivariate normal distribution N through Bayesian meta-learning, selecting the normal-inverse-Weichter distribution as the prior of the mean and covariance matrix The posterior probability , where They are the four parameters of the normal-inverse-Weichter distribution; and They are the labeled dataset vectors D of the target tasks respectively. target The mean and covariance matrix of ;

[0018] Step 3: Using the Posterior Probability , for the Bayesian quadratic classifier Instantiate and output the final diagnostic model .

[0019] The present invention introduces conjugate priors and performs posterior inference through Bayesian meta-learning, so that the Bayesian quadratic classifier can acquire cross-task knowledge, enhance the fitting ability and avoid overfitting.

[0020] In the present invention, the feature extraction network is trained, and the training process includes:

[0021] S1-1: Initialize feature extraction network and distance encoder , For feature extraction network The initial parameters of For distance encoder The initial parameters of

[0022] S1-2: From the meta-training set D meta-train Get the diagnostic task T i , diagnostic task T i Includes labeled training dataset and unlabeled training dataset , based on the labeled training data set Calculate the class prototype of the cth fault category:

[0023] ,

[0024] In the formula, , represents a pseudo label for 1 when it is, otherwise 0;

[0025] S1-3: Based on the unlabeled training dataset The class prototype of the cth fault category Updated to :

[0026] ,

[0027] In the formula, , C is the total number of fault categories;

[0028] S1-4: From the meta-training set D meta-train Get the test dataset from , , the diagnostic task T is calculated by the following formula i The pseudo-label annotation loss is:

[0029] ,

[0030] In the formula, ;

[0031] ;

[0032] Indicates distance;

[0033] S1-5: Loss through pseudo-labeling Update distance encoder and feature extraction network Parameters:

[0034] ,

[0035] ,

[0036] In the formula, and is the learning rate, For The gradient of For The gradient of

[0037] S1-6: Determine the pseudo-label loss function Is it the smallest? If the pseudo-label loss is not minimized, Assign to ,Will Assign to , and return to S1-1, if the pseudo-label loss When the minimum is reached, the training ends and the distance encoder is obtained. and feature extraction network .

[0038] Through the learnable distance encoder , which can adjust the differences between different data domains, making the update of class prototypes by unlabeled samples in different data domains more reasonable. The use of shallow encoder networks can achieve distance adjustment without hindering the efficiency of metric-based transductive learning.

[0039] In the present invention, the feature extraction network is trained, and the training process also includes:

[0040] S1-7: Unlabeled training dataset Perform two weak perturbations and unlabeled perturbation training datasets , Represents unlabeled data Perform a weak perturbation; Represents unlabeled data Perform a secondary weak perturbation;

[0041] S1-8: Calculate the weak perturbation self-supervision loss according to the following formula:

[0042] ,

[0043] In the formula, is a hyperparameter;

[0044] S1-9: Self-supervised loss via weak perturbations Update feature extraction network Parameters:

[0045] ,

[0046] In the formula, is the learning rate, For The gradient of

[0047] S1-10: Determining weak perturbation self-supervision loss Is it the smallest? If the weak perturbation self-supervision loss If the minimum value is not reached, return to S1-6. When it reaches the minimum, the training ends and the feature extraction network is obtained. .

[0048] The present invention guides the entire feature extraction network to be trained in a direction that is beneficial to the diagnosis task through weak perturbation self-supervision loss.

[0049] In the present invention, the feature extraction network is trained, and the training process also includes:

[0050] S1-11: Unlabeled training dataset Perform two strong perturbations to obtain an unlabeled strong perturbation training dataset , Represents unlabeled data Perform a strong disturbance; Represents unlabeled data Perform a second strong disturbance;

[0051] S1-12: Calculate the strong perturbation self-supervision loss according to the following formula:

[0052] ,

[0053] In the formula, is a hyperparameter;

[0054] S1-13; Screening confidence is greater than or equal to the hyperparameter Pseudo-labeled unlabeled data , re-apply strong and weak perturbations to it, and obtain its loss in a supervised learning manner: , where Indicated in the sample The maximum probability test value on ; Indicates that the maximum probability test value exceeds 1 when it is, otherwise 0; represents cross entropy; Represents unlabeled data In weak disturbance Get the probability vector of pseudo label y below; Represents unlabeled data In strong disturbance Get the probability vector of pseudo label y below;

[0055] The total loss function is obtained according to the following formula:

[0056] ,

[0057] In the formula, is the learning coefficient;

[0058] S1-14: Through the loss function Update feature extraction network Parameters:

[0059] ,

[0060] In the formula, is the learning rate, For The gradient of

[0061] S1-15: Determine the total loss function Is it the minimum? If the total loss function If the total loss function does not reach the minimum, return to S1-11; When it reaches the minimum, the training ends and the feature extraction network is obtained. .

[0062] The present invention guides the entire feature extraction network to be trained in a direction that is beneficial to the diagnosis task through the total loss function.

[0063] In the present invention, the Bayesian quadratic classifier is trained, and the training process includes:

[0064] S2-1: Through feature extraction network Get diagnostic task T k A labeled training dataset The eigenvector of the intrinsic structure , through Bayesian meta-learning of the conjugate priors of the mean and covariance matrix of the multivariate normal distribution N, the normal-inverse-Weichter distribution is selected as the prior of the mean and covariance moment, and the eigenvector is obtained Posterior probability , where They are the four parameters of the normal-inverse-Weichter distribution; and They are respectively the diagnostic tasks T k A labeled training dataset The mean and covariance matrix of ;

[0065] S2-2: According to the four parameters of the normal-inverse-Weichter distribution, namely meta-knowledge Calculating the meta-loss function ;

[0066] S2-3: Through the meta-loss function Compute meta-gradients , through the meta-gradient Update meta-knowledge ;

[0067] S2-4: Judgment Meta-Loss Function Is it the minimum? If the meta-loss function If the minimum is not reached, return to S2-1; if the total element loss function When the minimum is reached, the training ends and the Bayesian quadratic classifier is obtained. .

[0068] In the present invention, ,

[0069] Where d is the eigenvector The dimension of

[0070] .

[0071] The present invention introduces conjugate priors and performs posterior inference by Bayesian meta-learning, so that the Bayesian secondary classifier can obtain cross-task knowledge, enhance the fitting ability and avoid overfitting. The present invention is mainly used in mechanical and electrical product fault diagnosis.

[0072] The present invention also provides a system, which includes a storage medium and one or more processors, wherein the storage medium is used to store a computer program, and the computer program can be called by one or more processors to perform the above method.

[0073] The present invention also provides a computer program product, which is a program code compiled by a computer language according to the above method, and the program code can be called and executed by one or more processors.

[0074] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for enhancing small sample diagnostic capability based on Bayesian meta-knowledge extraction and embedding, characterized in that: The steps include: Step 1: Through feature extraction network Extracting labeled dataset D for electromechanical product target tasks target The intrinsic structural feature vector ,in, is the optimal parameter of the feature extraction network, , Step 2: Obtain a Bayesian quadratic classifier by learning the conjugate priors of the mean and covariance matrix of the multivariate normal distribution N through Bayesian meta-learning, selecting the normal-inverse-Weichter distribution as the prior of the mean and covariance matrix The posterior probability of the parameter , where They are the four parameters of the normal-inverse-Weichter distribution; and They are the labeled dataset vectors D of the target tasks respectively. target The mean and covariance matrix of ; Step 3: Using the Posterior Probability , for Bayesian quadratic classifier Instantiate and output the final diagnostic model ; Train the feature extraction network. The training process includes: S1-1: Initialize feature extraction network and distance encoder , For feature extraction network The initial parameters of For distance encoder The initial parameters of S1-2: From the meta-training set D meta-train Get the diagnostic task T i , diagnostic task T i Includes labeled training dataset and unlabeled training dataset , based on the labeled training data set Calculate the class prototype of the cth fault category: , In the formula, , represents the pseudo label for 1 when it is, otherwise 0; S1-3: Based on the unlabeled training dataset The class prototype of the cth fault category Updated to : ， In the formula, , C is the total number of fault categories; S1-4: From the meta-training set D meta-train Get the test dataset from , , the diagnostic task T is calculated by the following formula i The pseudo-label annotation loss is: ， In the formula, Indicates distance; S1-5: Loss through pseudo-labeling Update distance encoder and feature extraction network Parameters: ， ， In the formula, and is the learning rate, For The gradient of For The gradient of S1-6: Determine the pseudo-label loss function Is it the smallest? If the pseudo-label loss is not minimized, Assign to ,Will Assign to , and return to S1-1, if the pseudo-label loss When the minimum is reached, the training ends and the distance encoder is obtained. and feature extraction network .

2. The method according to claim 1, characterized in that Train the feature extraction network. The training process also includes: S1-7: Unlabeled training dataset Perform two weak perturbations to obtain an unlabeled perturbation training dataset , Represents unlabeled data Perform a weak perturbation; Represents unlabeled data Perform a secondary weak perturbation; S1-7: Calculate the weak perturbation self-supervision loss according to the following formula: ， In the formula, is a hyperparameter; S1-8: Self-supervised loss via weak perturbations Update feature extraction network Parameters: ， In the formula, is the learning rate, For The gradient of S1-9: Determining weak perturbation self-supervision loss Is it the smallest? If the weak perturbation self-supervision loss If the minimum value is not reached, return to step S1-6. When it reaches the minimum, the training ends and the feature extraction network is obtained. .

3. The method according to claim 2, characterized in that Train the feature extraction network. The training process also includes: S1-10: Unlabeled training dataset Perform two strong perturbations to obtain an unlabeled strong perturbation training dataset , Represents unlabeled data Perform a strong disturbance; Represents unlabeled data Perform a secondary strong disturbance; S1-11: Calculate the strong perturbation self-supervision loss according to the following formula: ， In the formula, is a hyperparameter; S1-12; Screening confidence is greater than or equal to the hyperparameter Pseudo-labeled unlabeled data , re-apply strong and weak perturbations to it, and obtain its loss in a supervised learning manner: ， In the formula, Indicated in the sample The maximum probability test value on ; Indicates that the maximum probability test value exceeds 1 when it is, otherwise 0; represents cross entropy; Represents unlabeled data In weak disturbance Get the probability vector of pseudo label y below; Represents unlabeled data In strong disturbance Get the probability vector of pseudo label y below; The total loss function is obtained according to the following formula: , where is the learning coefficient; S1-13: Through the total loss function Update feature extraction network Parameters: ， In the formula, is the learning rate, For The gradient of S1-14: Determine the total loss function Is it the minimum? If the total loss function If the total loss function does not reach the minimum, return to S1-11; When it reaches the minimum, the training ends and the feature extraction network is obtained. .

4. The method according to claim 3, characterized in that It also includes training the Bayesian quadratic classifier, and the training process includes: S2-1: Through feature extraction network Get diagnostic task T k A labeled training dataset The eigenvector of the intrinsic structure , through Bayesian meta-learning of the conjugate priors of the mean and covariance matrix of the multivariate normal distribution N, the normal-inverse-Weichter distribution is selected as the prior of the mean and covariance moment, and the eigenvector is obtained Posterior probability , where They are the four parameters of the normal-inverse-Weichter distribution; and They are respectively the diagnostic tasks T k A labeled training dataset The mean and covariance matrix of ; S2-2: According to the four parameters of the normal-inverse-Weichter distribution, namely meta-knowledge Calculating the meta-loss function ; S2-3: Through the meta-loss function Compute meta-gradients , through the meta-gradient Update meta-knowledge ; S2-4: Judgment Meta-Loss Function Is it the minimum? If the meta-loss function If the minimum is not reached, return to S2-1; if the total element loss function When the minimum is reached, the training ends and the Bayesian quadratic classifier is obtained. .

5. The method for enhancing small sample diagnostic capability based on Bayesian meta-knowledge extraction and embedding according to claim 3 is characterized in that: 。 6. The method according to claim 5, characterized in that 。

Images