A radar transceiver fault diagnosis method based on pseudo-label guided Transformer learning
Through pseudo-label guidance and Kepler optimization algorithm, the Transformer model's feature homogeneity and insufficient label data in radar system are solved, the accuracy and generalization capabilities of fault diagnosis are improved, and the stable operation of the radar system is ensured.
Patent Information
- Application Number
- CN202410932483.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-07-12
AI Technical Summary
The Transformer model faces the problem of feature homogeneity when processing large-scale data. In devices with high reliability requirements such as radar systems, it is expensive and time-consuming to obtain large amounts of labeled data, making it difficult to effectively use unlabeled data to improve the fault recognition rate.
The Transformer learning method based on pseudo-labeling is adopted to expand the training data set by generating pseudo-labels, and the Kepler optimization algorithm is used to optimize the model configuration parameters to improve the accuracy and generalization capabilities of the model.
Under the case of limited labeled samples, the accuracy and generalization capabilities of radar transceiver fault diagnosis are improved, the dependence on a large amount of labeled data is reduced, and the stable operation of the system and the smooth completion of tasks is ensured.
Smart Images

Figure CN118468092B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of fault diagnosis of key components of a radar system, and in particular to a radar transceiver fault diagnosis method based on pseudo-label guided Transformer learning. Background Art
[0002] With the digitization and intelligence of engineering systems, data-driven fault diagnosis methods based on machine learning and deep learning have become a research hotspot in the field of fault diagnosis in recent years. Data-driven methods use intelligent learning technology to build a diagnostic model that can automatically connect the relationship between monitoring data and machine health status. Among many data-driven methods, the Transformer model can effectively capture long-distance dependencies in the data through the self-attention mechanism, showing its advantages in processing sequence data, which is crucial for understanding and predicting the operating status of equipment.
[0003] However, the Transformer model may face the problem of feature homogeneity when processing large-scale data, which may affect the classification performance of the model. In addition, fault diagnosis methods usually assume that sufficient labeled data is available, but in practice, especially in equipment with high reliability requirements (such as radar systems), obtaining a large amount of labeled data is expensive and time-consuming. Therefore, how to effectively extract useful information from monitoring data and make full use of unlabeled data to improve the fault recognition rate remains an urgent problem to be solved. Summary of the invention
[0004] In view of the shortcomings of the prior art, the present invention provides a radar transceiver fault diagnosis method based on pseudo-label guided Transformer learning, which solves the technical problem that traditional methods cannot use unlabeled data to improve the fault recognition rate.
[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a radar transceiver fault diagnosis method based on pseudo-label guided Transformer learning, the method comprising the following steps:
[0006] S1. Build a Transformer model based on the Transformer network;
[0007] S2, use the unlabeled label as the input of the Transformer model to generate pseudo labels for unlabeled samples and calculate the confidence level of the pseudo labels;
[0008] S3, using the Kepler optimization algorithm KOA to optimize the configuration parameters of the Transformer model;
[0009] S4, offline training of the Transformer model with optimized configuration parameters is performed through pseudo-label guidance to obtain an optimized Transformer model;
[0010] S5. Perform fault diagnosis on the radar transceiver online.
[0011] Furthermore, in step S1, the specific process includes the following steps:
[0012] S11. Use sine and cosine functions to define the position information of the sequence, which is obtained by the following formula:
[0013] ;
[0014] in, is the position of the object in the input sequence; is the position in the vector to which it belongs; Represents the dimension of the Transformer model output embedding space;
[0015] S12. Adding position information to the original input Above, the initial input of the multi-head attention that constitutes the Transformer model is obtained by the following formula:
[0016] ;
[0017] in, Refers to Labeled samples are taken as input; Finger input The corresponding position code;
[0018] S13. Use multi-head attention to calculate the relationship between features to obtain the output of each sub-head attention, where the output of a single sub-head attention is obtained by the following formula:
[0019] ;
[0020] in, Refers to the mapping function of the sub-head attention; Refers to The output of the layer encoder; , , is the Transformer model for Three randomly initialized weight matrices of the attention modules of each sub-head of the layer; is the number of encoder stacking layers in the Transformer model;
[0021] S14, the outputs of each sub-head are combined and concatenated, and the result is obtained by the following formula:
[0022] ;
[0023] in, is the mapping function of multi-head attention; is the weight matrix of multi-head attention; For the Attention head;
[0024] S15, after the feedforward layer transformation, the final output of the last encoder layer in the Transformer model is obtained ,Right now:
[0025] ;
[0026] in, is the mapping function of the feedforward layer; Built-in for feed-forward layers The mapping function of the activation function; , is the weight matrix of the linear layer; , is the bias term of the linear layer;
[0027] S16, according to the final output Calculate the raw output of the fully connected layer , the output of the last encoder layer is transformed through a fully connected layer, then the original output of the fully connected layer The calculation is as follows:
[0028] ;
[0029] in, is the number of health status categories; is the weight matrix of the fully connected layer; is the bias term of the fully connected layer;
[0030] S17, use the softmax activation function to convert the original output of the fully connected layer Convert to probability distribution Get the Transformer model, probability distribution It is obtained from the following formula:
[0031] ;
[0032] in, is the probability distribution The probability of each category; is the exponential function acting on the output of the fully connected layer. On each element; The summation term ensures that the sum of all class probabilities is 1.
[0033] Furthermore, in step S2, the specific process includes the following steps:
[0034] S21, use the trained Transformer model to calculate the Unlabeled samples Classify into categories Probability , obtained by the following formula:
[0035] ;
[0036] in, Represents unlabeled samples Possible labels of , i.e. pseudo labels; Refers to the mapping function of the trained Transformer model;
[0037] S22. According to the classification The probability of calculating unlabeled samples The pseudo-label is:
[0038] ;
[0039] Among them, K is the category Total number of;
[0040] S23. Calculate unlabeled samples The confidence level of the pseudo-label , obtained by the following formula:
[0041] ;
[0042] ;
[0043] in, is the maximum entropy value in the state space; Refers to information entropy; is the normalized probability value;
[0044] S24. Retrain the Transformer model using unlabeled samples and labeled samples corresponding to credible pseudo labels.
[0045] Furthermore, in step S3, the specific process includes the following steps:
[0046] S31, initialization includes four parameters The location of the qualified point of the population, that is:
[0047] ;
[0048] in, Refers to the sampler size; Refers to the learning rate; Refers to the hidden layer size; refers to the regularization coefficient; represents the first planets, i.e. candidate solutions; Represents the number of candidate solutions in the search space; Represents the dimension of the problem to be optimized; and Respectively represent The first planet upper and lower bounds for the decision variables; is a randomly generated number between 0 and 1;
[0049] S32, orbital eccentricity and orbital period Initialize, that is:
[0050] ;
[0051] in, is a normally distributed random number;
[0052] S33, celestial body velocity update and planet position update;
[0053] S34. Update the distance between the planet and the sun;
[0054] S35, elite retention strategy, if the fitness of the updated position is better, then the updated position is retained, which is obtained by the following formula:
[0055] ;
[0056] in, is the updated position; This is the position before the update.
[0057] Further, in step S33, the celestial body velocity is updated by the following formula:
[0058] ;
[0059] ;
[0060] in, Representing objects In time speed; Representing objects ; and is a randomly generated number between 0 and 1; and is a vector containing random numbers between 0 and 1; and represents a solution randomly selected from the population; and Respectively and quality; represents the gravitational constant; Indicates at time Time Object The semi-major axis of the elliptical orbit; is a small value used to prevent division by zero errors; Indicates at time Best solution and The distance between objects; yes The normalized value of and Respectively represent The first planet upper and lower bounds for the decision variables;
[0061] The planet position is updated using the following formula:
[0062] ;
[0063] in, is the object in time new location; It is an object the speed required to reach the new location; It is the best position of the sun yet discovered; Used as a flag to change the direction of the search; is a normally distributed random number; and express and The normalized value of It indicates the best position and new location The Euclidean distance between The normalized value of For the The initialization value of the orbital eccentricity of the planet is a random number between 0 and 1. A random number between 0 and 1.
[0064] Furthermore, in step S34, the distance between the planet and the sun is updated, that is:
[0065] ;
[0066] in, It is used to control the time Adaptive factor for the distance between the sun and the current planet at that time; is a normally distributed random number; is a linear decreasing factor from 1 to −2; is defined in the formula for the entire optimization process The loop control parameter gradually decreases from -1 to -2 in each loop; For the deadline Discover the best position of the sun; is a solution randomly selected from the total solution; for of Power; The maximum number of iterations for the entire optimization process; For current time In cycle The result is a number between 0 and 1.
[0067] Furthermore, in step S4, the specific process includes the following steps:
[0068] S41. For a given labeled dataset , unlabeled dataset And the test dataset , using four parameters Initialize the position of each qualified point, where is the labeled sample in the labeled dataset; To mark samples Pseudo labels of is the unlabeled sample in the unlabeled dataset; is the test sample in the test data set; For test samples Pseudo labels of
[0069] S42, the mapping function of the training Transformer model, namely:
[0070] ;
[0071] in, Refers to the mapping function of the initially trained Transformer model; is the number of samples in the labeled dataset; is the number of samples in the unlabeled dataset; Refers to the number of samples in the test dataset;
[0072] S43, construct the first fitness function of the Kepler optimization algorithm KOA by cross-validating the classification error, that is:
[0073] ;
[0074] in, Refers to the 5-fold cross validation function; Refers to the fitness function initially constructed by the cross-validation method; Refers to the sampler size; Refers to the learning rate; Refers to the hidden layer size; refers to the regularization coefficient;
[0075] S44, using the Kepler optimization algorithm KOA algorithm to preliminarily optimize the configuration parameters to obtain the best model configuration parameters, namely:
[0076] ;
[0077] in, Refers to the Transformer model sampler size, learning rate, hidden layer size, and regularization coefficient after KOA's initial optimization;
[0078] S45. Generate unlabeled samples through the Transformer model according to the optimal model configuration parameters Pseudo-labels ,Right now:
[0079] ;
[0080] in, Refers to the initial training update unlabeled samples The number after
[0081] S46. Retraining Transformer model and labeling samples and unlabeled samples The mapping function is:
[0082] ;
[0083] in, Refers to the mapping function of the Transformer model after further training;
[0084] S47, reconstructing the second fitness function of the Kepler optimization algorithm KOA by cross-validating the classification error, that is:
[0085] ;
[0086] in, is the mapping function of the fitness further reconstructed by the cross-validation method, i.e., the second fitness function;
[0087] S48. Use the Kepler optimization algorithm KOA to regain the optimal model configuration parameters of the Transformer model according to the second fitness function, namely:
[0088] ;
[0089] in, The sampler size, learning rate, hidden layer size, and regularization coefficient of the Transformer model after the final optimization of the KOA algorithm.
[0090] Further, in step S5, specifically: receiving the current The signal at time is taken as sample input , and input it into the optimized Transformer model to obtain the health status of the radar transceiver, namely:
[0091] ;
[0092] In the above formula, Refers to the mapping function of the Transformer model after further training, The sampler size, learning rate, hidden layer size, and regularization coefficient of the Transformer model after the KOA algorithm is finally optimized. The health status of the radar transceiver.
[0093] By means of the above technical solution, the present invention provides a radar transceiver fault diagnosis method based on pseudo-label guided Transformer learning, which has at least the following beneficial effects:
[0094] 1. The present invention can generate pseudo labels through the Transformer model under limited labeled samples, thereby expanding the training data set, and improve the accuracy of the radar transceiver fault diagnosis model by optimizing the network configuration parameters of the Transformer model.
[0095] 2. The present invention introduces a pseudo-label guided learning strategy, which enables the Transformer model to make full use of unlabeled data and enhance the generalization ability of the model. At the same time, the application of the Kepler optimization algorithm KOA ensures the optimal configuration of the configuration parameters and further improves the accuracy of fault identification. This provides reliable support for the maintenance decision of the radar system and ensures the stable operation of the system and the smooth completion of the task. BRIEF DESCRIPTION OF THE DRAWINGS
[0096] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0097] Figure 1 It is a framework diagram of the phased array radar transceiver system in the present invention;
[0098] Figure 2 is a flow chart of a radar transceiver fault diagnosis method of the present invention;
[0099] Figure 3 It is a confidence level diagram of some labeled samples corresponding to pseudo labels in the present invention;
[0100] Figure 4 It is a graph showing the optimal fitness value variation of the Kepler optimization algorithm KOA in the present invention;
[0101] Figure 5 This is a fault classification result diagram obtained by the present invention using the test set. DETAILED DESCRIPTION
[0102] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods, so that the implementation process of how the present application uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.
[0103] The transceiver T / R module of a phased array radar system is one of its core components, and its operating performance directly affects the reliability of the entire radar system. In existing radar systems, fault diagnosis methods are mainly divided into model-based methods, knowledge-based methods, and data-driven methods. Model-based methods are committed to establishing mathematical or physical models of actual systems or industrial processes, and developing fault diagnosis methods by monitoring the consistency between the model's predicted output and the actual system output. Knowledge-based methods rely on sufficient expert knowledge of the machine's fault mechanism and achieve fault diagnosis by defining qualitative models or rule models. However, due to the complex structure of modern engineering systems or the diversity of fault causes, it becomes difficult to establish accurate models or expert knowledge bases, and these traditional fault diagnosis methods may not be convenient for industrial applications.
[0104] With the digitization and intelligence of engineering systems, data-driven fault diagnosis methods based on machine learning and deep learning have become a research hotspot in the field of fault diagnosis in recent years. Data-driven methods use intelligent learning techniques to build a diagnostic model that can automatically connect the relationship between monitoring data and machine health status. Among many data-driven methods, the Transformer model can effectively capture long-distance dependencies in the data through the self-attention mechanism, showing advantages in processing sequence data, which is crucial for understanding and predicting the operating status of equipment. However, the Transformer model may face the problem of feature homogeneity when processing large-scale data, which may affect the classification performance of the model. In addition, fault diagnosis methods usually assume that sufficient labeled data is available, but in practice, especially in equipment with high reliability requirements (such as radar systems), it is expensive and time-consuming to obtain a large amount of labeled data. Therefore, how to effectively extract useful information from monitoring data and make full use of unlabeled data to improve the fault recognition rate remains an urgent problem to be solved.
[0105] The application object of the fault diagnosis method proposed in the present invention is a transceiver system of a phased array radar, such as Figure 1 As shown in the figure, it includes three parts: antenna array, T / R module and signal processing system. Sensors are installed inside the T / R module to monitor a total of 26 condition variables, divided into 5 types. The first 8 variables refer to the output power of transmitter amplifiers 1 to 8, while the next 16 variables correspond to the received signals and noise signals from receiving channels 1 to 8 respectively. The last two variables represent the transceiver power supply voltage and current. Fault numbers 1 to 4 are single fault types, while fault number 5 is a composite fault.
[0106] Please refer to Figure 1-Figure 5 This embodiment proposes a radar transceiver fault diagnosis method based on pseudo-label guided Transformer learning, which can generate pseudo-labels through the Transformer model to expand the training data set under limited labeled samples, and improve the accuracy of radar transceiver fault diagnosis by optimizing the configuration parameters of the Transformer model. Figure 2 As shown, the method comprises the following steps:
[0107] S1. Build a Transformer model based on the Transformer network. The specific process includes the following steps:
[0108] S11. Use sine and cosine functions to define the position information of the sequence, which is obtained by the following formula:
[0109] ; (1)
[0110] in, is the position of the object in the input sequence; is the position in the vector to which it belongs; Represents the dimension of the Transformer model output embedding space;
[0111] S12. Adding position information to the original input Above, the initial input of the multi-head attention that constitutes the Transformer model is obtained by the following formula:
[0112] ; (2)
[0113] in, Refers to Labeled samples are taken as input; Finger input The corresponding position code;
[0114] S13. Use multi-head attention to calculate the relationship between features to obtain the output of each sub-head attention, where the output of a single sub-head attention is obtained by the following formula:
[0115] ; (3)
[0116] in, Refers to the mapping function of the sub-head attention; Refers to The output of the layer encoder; , , is the Transformer model for Three randomly initialized weight matrices of the attention modules of each sub-head of the layer; is the number of encoder stacking layers in the Transformer model;
[0117] S14, the outputs of each sub-head are combined and concatenated, and the result is obtained by the following formula:
[0118] ; (4)
[0119] in, is the mapping function of multi-head attention; is the weight matrix of multi-head attention; For the Attention head;
[0120] S15, based on equations (2), (3) and (4), the final output of the last encoder layer in the Transformer model is obtained through feedforward transformation ,Right now:
[0121] ; (5)
[0122] in, is the mapping function of the feedforward layer; Built-in for feed-forward layers The mapping function of the activation function; , is the weight matrix of the linear layer; , is the bias term of the linear layer;
[0123] S16, according to the final output Calculate the raw output of the fully connected layer , the output of the last encoder layer is transformed through a fully connected layer, then the original output of the fully connected layer The calculation is as follows:
[0124] ; (6)
[0125] in, is the number of health status categories; is the weight matrix of the fully connected layer; is the bias term of the fully connected layer;
[0126] S17, use the softmax activation function to convert the original output of the fully connected layer Convert to probability distribution Get the Transformer model, probability distribution It is obtained from the following formula:
[0127] ; (7)
[0128] in, is the probability distribution The probability of each category; is the exponential function acting on the output of the fully connected layer. On each element; The summation term ensures that the sum of all class probabilities is 1.
[0129] In this embodiment, the Transformer network is redefined through the above steps to solve the problem of feature homogeneity. By using the self-attention mechanism, the Transformer network can effectively capture long-distance dependencies in the data. This mechanism helps to identify and distinguish subtle differences between features, thereby solving the problem of feature homogeneity faced by traditional methods when processing large-scale data and improving the classification performance of the model.
[0130] At the same time, the redefinition of the Transformer network in this application enables it to generate pseudo labels, thereby using unlabeled data to expand the training data set. By guiding the learning strategy with pseudo labels, the model can make full use of unlabeled data in the case of limited labeled samples, thereby improving the accuracy and generalization ability of fault identification.
[0131] And reduce the dependence on labeled data. In equipment with high reliability requirements (such as radar systems), it is expensive and time-consuming to obtain a large amount of labeled data. Through the redefinition of the Transformer network and the pseudo-label guided learning strategy, the efficiency and effect of fault diagnosis can be improved in the case of limited labeled samples, reducing the dependence on a large amount of labeled data.
[0132] S2. Use the unlabeled labels as the input of the Transformer model to generate pseudo labels for the unlabeled samples, and calculate the confidence level of the pseudo labels. In practical applications, especially in equipment with high reliability requirements such as radar systems, obtaining a large amount of labeled data is expensive and time-consuming, and traditional fault diagnosis methods often cannot make full use of unlabeled data, resulting in low data utilization.
[0133] In order to solve this technical problem, in step S2, this embodiment can effectively utilize unlabeled data by generating pseudo labels, reduce the dependence on a large amount of labeled data, and solve the problem of insufficient labeled data. At the same time, by generating pseudo labels, unlabeled data can be effectively utilized, reducing the dependence on a large amount of labeled data, and solving the problem of insufficient labeled data. Specifically, the detailed process includes the following steps:
[0134] S21, use the trained Transformer model to calculate the Unlabeled samples Classify into categories Probability , obtained by the following formula:
[0135] ; (8)
[0136] in, Represents unlabeled samples Possible labels of , i.e. pseudo labels; Refers to the mapping function of the trained Transformer model;
[0137] S22. According to the classification The probability of calculating unlabeled samples The pseudo-label is:
[0138] ; (9)
[0139] Among them, K is the category Total number of;
[0140] S23. Calculate unlabeled samples The confidence level of the pseudo-label ,like Figure 3 As shown, it is obtained by the following formula:
[0141] ; (10)
[0142] ; (11)
[0143] in, is the maximum entropy value in the state space; Refers to information entropy; is the normalized probability value;
[0144] S24, retrain the Transformer model using unlabeled samples and labeled samples corresponding to credible pseudo labels. Here, in this embodiment, unlabeled samples with a confidence level greater than or equal to 0.9 are selected to be added to the training set in order to retrain the network.
[0145] In this step, generating pseudo labels and calculating their confidence levels have the following technical effects, namely:
[0146] Expanding the training dataset: The generated pseudo labels allow unlabeled data to be converted into labeled data, significantly expanding the size of the training dataset. This is crucial to improving the learning ability and accuracy of the Transformer model, especially when labeled data is limited.
[0147] Enhance the generalization ability of the model: By using pseudo labels, the Transformer model can learn from more data, effectively improving the generalization ability of the model. This means that the Transformer model can maintain high accuracy and stability when facing new, unseen data.
[0148] Optimize the performance of the Transformer model: The introduction of pseudo labels and the calculation of confidence levels allow the Transformer model to prioritize learning of high-confidence data, thereby improving the performance and reliability of the Transformer model. The Transformer model can automatically select the most credible samples during training, improving training efficiency and effectiveness.
[0149] S3. Use the Kepler optimization algorithm KOA to optimize the configuration parameters of the Transformer model. The specific process includes the following steps:
[0150] S31, initialization includes four parameters The location of the qualified point of the population, that is:
[0151] ; (12)
[0152] in, Refers to the sampler size; Refers to the learning rate; Refers to the hidden layer size; refers to the regularization coefficient; represents the first planets, i.e. candidate solutions; Represents the number of candidate solutions in the search space; Represents the dimension of the problem to be optimized, here ; and Respectively represent The first planet upper and lower bounds for the decision variables; is a randomly generated number between 0 and 1;
[0153] S32, orbital eccentricity and orbital period Initialize, that is:
[0154] ; (13)
[0155] in, is a normally distributed random number;
[0156] S33, celestial body velocity update and planet position update;
[0157] Among them, the celestial body speed is updated through the following formula:
[0158] ; (14)
[0159] ; (15)
[0160] in, Representing objects In time speed; Representing objects ; and is a randomly generated number between 0 and 1; and is a vector containing random numbers between 0 and 1; and represents a solution randomly selected from the population; and Respectively and quality; represents the gravitational constant; Indicates at time Time Object The semi-major axis of the elliptical orbit; is a small value used to prevent division by zero errors; Indicates at time Best solution and The distance between objects; yes The normalized value of and Respectively represent The first planet upper and lower bounds for the decision variables;
[0161] The planet position is updated using the following formula:
[0162] ; (16)
[0163] in, is the object in time new location; It is an object the speed required to reach the new location; It is the best position of the sun yet discovered; Used as a flag to change the direction of the search; is a normally distributed random number; and express and The normalized value of It indicates the best position and new location The Euclidean distance between The normalized value of For the The initialization value of the orbital eccentricity of the planet is a random number between 0 and 1. A random number between 0 and 1 to give more variation to the gravity value during optimization;
[0164] S34. Update the distance between the planet and the sun, that is:
[0165] ; (17)
[0166] in, It is used to control the time Adaptive factor for the distance between the sun and the current planet at that time; is a normally distributed random number; is a linear decreasing factor from 1 to −2; is defined in the formula for the entire optimization process The loop control parameter gradually decreases from -1 to -2 in each loop; is a conditional value, which is given in formula (15); For the deadline Discover the best position of the sun; is a solution randomly selected from the total solution; for of Power; The maximum number of iterations for the entire optimization process; is the total number of cycles, is a cycle length, which means that the maximum number of iterations is divided equally into cycles; It is a modulo operation (taking the remainder); For current time In cycle The result is a number between 0 and 1.
[0167] S35, elite retention strategy, if the fitness of the updated position is better, then the updated position is retained, which is obtained by the following formula:
[0168] ; (18)
[0169] in, is the updated position; This is the position before the update.
[0170] In this embodiment, the optimization model configuration parameters play the following roles:
[0171] Avoid the difficulty of manual setting: Manually adjusting parameters is time-consuming and error-prone. KOA solves the difficulty of Transformer model parameter setting by simulating the mechanism of planetary motion and automatically finding the optimal parameter combination.
[0172] Improved robustness: KOA can dynamically adjust parameters to adapt to different data distributions and characteristics by simulating the mechanism of planetary motion. This adaptive capability enables the model to maintain good performance in different application scenarios.
[0173] Accelerate convergence: Reasonable configuration parameters can accelerate the convergence of Transformer model training and reduce training time. The optimized parameter settings can enable the Transformer model to achieve optimal performance in a shorter time and improve training efficiency.
[0174] Avoid local optimum: Kepler optimization algorithm has a strong global search capability and can find the global optimal solution in a complex parameter space to avoid falling into the local optimum. Compared with traditional optimization methods, KOA has more advantages in dealing with high-dimensional and nonlinear problems.
[0175] In this embodiment, the following specific problems may occur due to the non-optimization of configuration parameters:
[0176] 1. Sampler size is not optimized
[0177] Unstable gradient descent: Too small a sampler size leads to unstable gradient updates, and model parameters fluctuate greatly during the update process, making it difficult to converge to the optimal solution;
[0178] High memory requirements: A large sampler size requires a lot of memory, which may cause insufficient memory and affect the training process, especially when processing large-scale data.
[0179] 2. Problem of unoptimized learning rate
[0180] Convergence difficulties: Too large a learning rate makes the parameter update step too large, which may skip the optimal solution and cause the model to fail to converge;
[0181] Slow convergence speed: Too small a learning rate makes the parameter update step too small, the model converges very slowly, and more iterations are required to achieve the expected effect.
[0182] 3. The problem of hidden layer size not being optimized
[0183] Underfitting: The hidden layer size is too small, the model has insufficient capacity, and cannot learn the complex patterns and features in the data, resulting in poor model performance;
[0184] Overfitting: The hidden layer size is too large, the capacity of the model is too large, and it is easy to remember the noise in the training data, resulting in poor generalization ability of the model on new data.
[0185] 4. The problem of unoptimized regularization coefficient
[0186] Overfitting: The regularization coefficient is too small, the model complexity is too high, and it is easy to remember the noise in the training data, which manifests as overfitting;
[0187] Underfitting: The regularization coefficient is too large, the model complexity is limited, and it is unable to fully learn the features and patterns in the data, resulting in the model being underfitting.
[0188] In the unoptimized parameters There are the above four direct defects based on the above, so in this embodiment, the Transformer model is trained offline in a guided manner to obtain the optimal model configuration parameters , thereby improving the performance of the Transformer model. The specific implementation method is as follows:
[0189] S4. Offline training of the Transformer model with optimized configuration parameters is performed through pseudo-label guidance to obtain an optimized Transformer model. The specific process includes the following steps:
[0190] S41. For a given labeled dataset , unlabeled dataset And the test dataset , using four parameters Initialize the position of each qualified point, where is the labeled sample in the labeled dataset; To mark samples Pseudo labels of is the unlabeled sample in the unlabeled dataset; is the test sample in the test data set; For test samples Pseudo labels of
[0191] S42, the mapping function of the training Transformer model, namely:
[0192] ; (19)
[0193] in, Refers to the mapping function of the initially trained Transformer model; is the number of samples in the labeled dataset; is the number of samples in the unlabeled dataset; Refers to the number of samples in the test dataset;
[0194] S43, construct the first fitness function of the Kepler optimization algorithm KOA by cross-validating the classification error, that is:
[0195] ; (20)
[0196] in, Refers to the 5-fold cross validation function; Refers to the fitness function initially constructed by the cross-validation method; Refers to the sampler size; Refers to the learning rate; Refers to the hidden layer size; refers to the regularization coefficient;
[0197] S44, using the Kepler optimization algorithm KOA algorithm to preliminarily optimize the configuration parameters to obtain the best model configuration parameters, namely:
[0198] ;(twenty one)
[0199] in, Refers to the Transformer model sampler size, learning rate, hidden layer size, and regularization coefficient after KOA's initial optimization;
[0200] S45. Generate unlabeled samples through the Transformer model according to the optimal model configuration parameters Pseudo-labels ,Right now:
[0201] ;(twenty two)
[0202] in, Refers to the initial training update unlabeled samples The number after
[0203] S46. Retraining Transformer model and labeling samples and unlabeled samples The mapping function is:
[0204] ;(twenty three)
[0205] in, Refers to the mapping function of the Transformer model after further training;
[0206] S47, reconstructing the second fitness function of the Kepler optimization algorithm KOA by cross-validating the classification error, that is:
[0207] ;(twenty four)
[0208] in, is the mapping function of the fitness further reconstructed by the cross-validation method, i.e., the second fitness function;
[0209] S48. Use the Kepler optimization algorithm KOA to regain the optimal model configuration parameters of the Transformer model according to the second fitness function, namely:
[0210] ; (25)
[0211] in, The sampler size, learning rate, hidden layer size, and regularization coefficient of the Transformer model after the final optimization of the KOA algorithm.
[0212] In this embodiment, the direct effect of pseudo-label guidance is:
[0213] Make full use of unlabeled data: Pseudo-labels enable unlabeled data to be converted into part of the training data, and by expanding the size of the training set, the model's learning ability is enhanced. This can improve the model's training effect based on limited labeled data;
[0214] Improve the generalization ability of the model: Through pseudo-label guidance, the model can learn more data patterns and features during the training process, thereby improving the generalization ability of the model. This means that the model can still maintain high accuracy and stability when facing new and unseen data;
[0215] Improve the robustness of the model: The pseudo-label-guided training method enables the model to process more diverse data, improves the model's adaptability to different input data, and enhances its robustness. In this way, the model's ability to handle noise and abnormal data in practical applications is also improved;
[0216] Speed up model training: Under the guidance of pseudo labels, the model can quickly extract useful information from a large amount of unlabeled data, speeding up model training. The model with optimized configuration parameters can be trained under the guidance of pseudo labels and converge to the global optimum more effectively.
[0217] Optimal model configuration parameters The role is:
[0218] Improve model accuracy: Optimizing sampler size, learning rate, hidden layer size, and regularization coefficient can improve the model's ability to fit and generalize data, thereby improving the accuracy of fault diagnosis;
[0219] Improve training efficiency: Reasonable sampler size and learning rate settings can speed up model training and make it converge to the optimal solution more quickly;
[0220] Enhanced model robustness: Optimized parameter settings make the model more robust to different types of input data and able to maintain stable performance in different application scenarios;
[0221] Prevent overfitting and underfitting: By properly setting the hidden layer size and regularization coefficient, overfitting and underfitting can be effectively prevented, and the performance consistency of the model on the training set and the test set can be improved.
[0222] In summary, optimizing the sampler size, learning rate, hidden layer size, and regularization coefficient can significantly improve the training effect and performance of the Transformer model, solve various problems caused by unoptimized parameters, and provide reliable technical support for radar transceiver fault diagnosis.
[0223] S5. Online fault diagnosis of the radar transceiver is performed, specifically: receiving the signal at the current time t as a sample input, and inputting it into the optimized Transformer model to obtain the health status of the radar transceiver, that is:
[0224] ; (26)
[0225] In the above formula, Refers to the mapping function of the Transformer model after further training, The sampler size, learning rate, hidden layer size, and regularization coefficient of the Transformer model after the KOA algorithm is finally optimized. The health status of the radar transceiver.
[0226] In order to verify the performance of a radar transceiver fault diagnosis method based on pseudo-label guided Transformer learning proposed in the present invention, in this embodiment, a total of 11,643 samples were collected for the 26 conditional variables monitored, of which 1,800 were labeled and 9,843 were unlabeled. Therefore, this dataset is suitable for developing a pseudo-label guided Transformer learning method for fault diagnosis of key radar components. In addition, the labeled samples have 6 health conditions, including 1 normal condition and 5 fault conditions. Under each health condition, 250 samples are randomly selected as the training set and 50 samples are selected as the test set.
[0227] like Figure 4 As shown, the configuration parameters of the Transformer model Determined by the KOA algorithm, the number of qualified points and the maximum number of iterations are set to 20 and 100 respectively. , The solution space range is [1,100], and The range of is [0,1]. Finally, after the 37th iteration, the optimal solution for the configuration parameters is found, namely , , and .
[0228] Calculate the confidence of the pseudo-label sample, such as Figure 3 is the confidence of some pseudo-labeled samples. At the same time, in order to determine the threshold of the confidence level, some validation tests were conducted and it was found that the classification error was the smallest when the confidence level was 0.9. Therefore, unlabeled samples with a confidence level greater than or equal to 0.9 will be added to the training set in order to retrain the network. After that, the network configuration parameters need to be re-optimized in the above manner.
[0229] Perform identification diagnosis on the test samples, such as Figure 5 As shown, the recognition rates of the proposed method for fault categories 1-6 are 94%, 92%, 100%, 98%, 90% and 98% respectively, all of which achieve recognition rates higher than 90%. This shows that the radar transceiver fault diagnosis method proposed in the present invention is effective and can ensure the safe and reliable operation of the radar transceiver.
[0230] The reason why the radar transceiver fault diagnosis method proposed in the present invention is so effective is mainly due to the fact that the pseudo-label guided Transformer learning method for radar transceiver fault diagnosis can effectively learn the intrinsic relationship between monitoring data and the health status of radar components. By introducing the pseudo-label guided learning strategy, the Transformer model can make full use of unlabeled data and enhance the generalization ability of the model. At the same time, the application of the Kepler optimization algorithm KOA ensures the optimal configuration of the configuration parameters and further improves the accuracy of fault identification. The integration of these technologies makes fault diagnosis more accurate, thereby providing reliable support for the maintenance decision of the radar system and ensuring the stable operation of the system and the smooth completion of the task.
[0231] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, so the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0232] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the above embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0233] The above implementation methods have been described in detail. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A radar transceiver fault diagnosis method based on pseudo-label guided Transformer learning, characterized in that: The method comprises the following steps: S1. Build a Transformer model based on the Transformer network. The specific process includes the following steps: S11. Use sine and cosine functions to define the position information of the sequence, which is obtained by the following formula: Where t is the position of the object in the input sequence; u is the position in the belonging vector; d represents the dimension of the output embedding space of the Transformer model; S12. Adding position information to the original input Above, the initial input of the multi-head attention that constitutes the Transformer model is obtained by the following formula: in, Refers to the i-th labeled sample as input; PE (i) Finger input The corresponding position code; S13. Use multi-head attention to calculate the relationship between features to obtain the output of each sub-head attention, where the output of a single sub-head attention is obtained by the following formula: Among them, Attention(·) refers to the mapping function of the sub-head attention; H l-1 Refers to the output of the l-1th layer encoder; are the three randomly initialized weight matrices for each sub-head attention module in the lth layer in the Transformer model; L is the number of encoder stacking layers in the Transformer model; S14, the outputs of each sub-head are combined and concatenated, and the result is obtained by the following formula: MultiHead(H l-1 )=[head1;head2;···;head h ]W O Among them, MultiHead(·) is the mapping function of multi-head attention; W O is the weight matrix of multi-head attention; head h is the hth attention head; S15, after the feedforward layer transformation, the final output H of the last encoder layer in the Transformer model is obtained L ,Right now: H L =FFN(MultiHead(H L-1 ))=ReLU(MultiHead(H L-1 )W1+b1)W2+b2 Among them, FFN(·) is the mapping function of the feedforward layer; ReLU(·) is the mapping function of the built-in ReLU activation function of the feedforward layer; W1, W2 are the weight matrices of the linear layer; b1, b2 are the bias terms of the linear layer; S16, according to the final output H L Calculate the original output o of the fully connected layer. The output of the last encoder layer is transformed through a fully connected layer. The original output o of the fully connected layer is calculated as follows: <h2 style=";text-align:left;direction:ltr">o=W3H<h2 style=";text-align:left;direction:ltr"> L <h2 style=";text-align:left;direction:ltr"> +b3 Among them, K is the number of health status categories; W3 is the weight matrix of the fully connected layer; b3 is the bias term of the fully connected layer; S17. Use the softmax activation function to convert the original output o of the fully connected layer into a probability distribution Get the Transformer model, probability distribution It is obtained from the following formula: Among them, K is the number of health status categories; is the probability of the i-th category in the probability distribution; e (o)i is the exponential function acting on the i-th element of the fully connected layer output; The summation term ensures that the sum of the probabilities of all categories is 1; S2, use the unlabeled label as the input of the Transformer model to generate pseudo labels for unlabeled samples and calculate the confidence level of the pseudo labels; S3, using the Kepler optimization algorithm KOA to optimize the configuration parameters of the Transformer model; S4, offline training the Transformer model with optimized configuration parameters is performed through pseudo-label guidance to obtain an optimized Transformer model; S5. Perform fault diagnosis on the radar transceiver online.
2. The radar transceiver fault diagnosis method according to claim 1, characterized in that: In step S2, the specific process includes the following steps: S21, use the trained Transformer model to calculate the i-th unlabeled sample The probability of classification into category k (k = 1, 2, ..., K) It is obtained from the following formula: in, Represents unlabeled samples possible labels, namely pseudo labels; f transformer (·) refers to the mapping function of the trained Transformer model; S22. Calculate the probability of unlabeled samples according to the classification into category k (k = 1, 2, ..., K) The pseudo-label is: Where K is the total number of category k; S23. Calculate unlabeled samples The confidence level w of the pseudo label i , obtained by the following formula: Where log(K) is the maximum entropy value in the state space; H i Refers to information entropy; is the normalized probability value; S24. Retrain the Transformer model using unlabeled samples and labeled samples corresponding to credible pseudo labels.
3. The radar transceiver fault diagnosis method according to claim 1, characterized in that: In step S3, the specific process includes the following steps: S31. Initialize the positions of qualified points of the population containing four parameters (m, τ, s, λ), namely: Among them, m refers to the sampler size; τ refers to the learning rate; s refers to the hidden layer size; λ refers to the regularization coefficient; X i represents the i-th planet in the search space, i.e., the candidate solution; N represents the number of candidate solutions in the search space; D represents the dimension of the problem to be optimized; and denote the upper and lower bounds of the jth decision variable of the ith planet respectively; rand [0,1] is a randomly generated number between 0 and 1; S32, orbital eccentricity e i and orbital period T i Initialize, that is: Among them, r is a normally distributed random number; S33, celestial body velocity update and planet position update; S34. Update the distance between the planet and the sun; S35, elite retention strategy, if the fitness of the updated position is better, then the updated position is retained, which is obtained by the following formula: in, is the updated position; This is the position before the update.
4. The radar transceiver fault diagnosis method according to claim 3, characterized in that: In step S33, the celestial body velocity is updated by the following formula: Among them, V i (t) represents the velocity of object i at time t; represents object i; r3 and r4 are randomly generated numbers between 0 and 1; and is a vector containing random numbers between 0 and 1; and represents a solution randomly selected from the population; M s and m i Represents X and X respectively i The mass of; μ(t) represents the gravitational constant; a i(t) represents the semi-major axis of the elliptical orbit of object i at time t; ε is a small value used to prevent division by zero errors; R i (t) represents the best solution X at time t s and object X i The distance between i-norm (t) is R i Normalized value of (t); and denote the upper and lower bounds of the j-th decision variable of the ith planet respectively; The planet position is updated using the following formula: in, is the new position of the object at time t+1; is the speed required for object i to reach its new position; It is the best position of the sun yet discovered; Used as a sign to change the search direction; r is a normally distributed random number; and Indicates M s and m i The normalized value of Is the best position X S and the new position X i The Euclidean distance between The normalized value of i is the initialization value of the orbital eccentricity of the i-th planet, which is a random number between 0 and 1, and r1 is a random number between 0 and 1.
5. The radar transceiver fault diagnosis method according to claim 4, characterized in that: In step S34, the distance between the planet and the sun is updated, that is: Where h is an adaptive factor used to control the distance between the sun and the current planet at time t; r is a normally distributed random number; η is a linear decreasing factor from 1 to -2; a2 is defined in the formula for the entire optimization process The loop control parameter gradually decreases from -1 to -2 in each loop; is the best position of the sun found up to time t; is a solution randomly selected from the total solution; e ηr is e to the power of ηr; T max The maximum number of iterations for the entire optimization process; is the current time t in the cycle The result is a number between 0 and 1.
6. The radar transceiver fault diagnosis method according to claim 1, characterized in that: In step S4, the specific process includes the following steps: S41. For a given labeled dataset Unlabeled dataset and test dataset The position of each qualified point is initialized using four parameters (m, τ, s, λ), where is the labeled sample in the labeled dataset; To mark samples Pseudo labels of is the unlabeled sample in the unlabeled dataset; is the test sample in the test data set; For test samples Pseudo labels of S42, the mapping function of the training Transformer model, namely: in, Refers to the mapping function of the initially trained Transformer model; N la is the number of samples in the labeled dataset; N ul is the number of samples in the unlabeled dataset; N t Refers to the number of samples in the test dataset; S43, construct the first fitness function of the Kepler optimization algorithm KOA by cross-validating the classification error, that is: Among them, CV5(·) refers to the 5-fold cross validation function; fitness (1) (·) refers to the fitness function initially constructed by the cross-validation method; m refers to the sampler size; τ refers to the learning rate; s refers to the hidden layer size; λ refers to the regularization coefficient; S44, using the Kepler optimization algorithm KOA algorithm to preliminarily optimize the configuration parameters to obtain the best model configuration parameters, namely: (m * ,t * ,s * ,l * )←argminfitness (1) Among them, (m * ,τ * ,s * ,λ * ) refers to the sampler size, learning rate, hidden layer size, and regularization coefficient of the Transformer model after preliminary optimization by KOA; S45. Generate unlabeled samples through the Transformer model according to the optimal model configuration parameters Pseudo labels Right now: in, Refers to the initial training update unlabeled samples The number after S46. Retraining Transformer model and labeling samples and unlabeled samples The mapping function is: in, Refers to the mapping function of the Transformer model after further training; S47, reconstructing the second fitness function of the Kepler optimization algorithm KOA by cross-validating the classification error, that is: Among them, fitness (2) (·) is the mapping function of the fitness further reconstructed by the cross-validation method, i.e., the second fitness function; S48. Use the Kepler optimization algorithm KOA to regain the optimal model configuration parameters of the Transformer model according to the second fitness function, namely: (m ** ,t ** ,s ** ,l ** )←argminfitness (2) Among them, (m ** ,τ ** ,s ** ,λ ** ) is the sampler size, learning rate, hidden layer size, and regularization coefficient of the Transformer model after the final optimization of the KOA algorithm.
7. The radar transceiver fault diagnosis method according to claim 1, characterized in that: In step S5, specifically: receiving the signal at the current time t as a sample input And input it into the optimized Transformer model to obtain the health status of the radar transceiver, that is: In the above formula, Refers to the mapping function of the Transformer model after further training; (m ** ,τ ** ,s ** ,λ ** ) is the sampler size, learning rate, hidden layer size and regularization coefficient of the Transformer model after the final optimization of the KOA algorithm; The health status of the radar transceiver.
