Large-aperture antenna system health state monitoring and evaluation method based on multi-modal data and deep learning

Abstract

The invention provides a large-aperture antenna system health state monitoring and evaluation method based on multi-modal data and deep learning, and the method carries out the comprehensive investigation of the multi-modal data of the front and back correlation and mutual coupling of a large-aperture antenna system, and employs the deep learning technology to design a network module with a decoupling effect. Features of various types of data are independently extracted from the first several layers and then fused, so that high-precision detection quality is realized; and through influence weight analysis of related factors, problems are searched and judged in an auxiliary manner, and intelligent debugging is realized. According to the method, the working efficiency of a large-aperture antenna installation and adjustment field is improved, the fault hidden danger is reduced at the initial stage of implementation, a later stable operation and maintenance foundation is laid, and an effective means is provided.

Description

Technical Field

[0001] The present invention relates to the technical field of large-aperture antenna system health status monitoring, and in particular to a large-aperture antenna system health status monitoring and evaluation method based on multimodal data and deep learning. Background Art

[0002] Large-aperture antenna systems are mainly used in astronomical observation and deep space exploration, and their core task is to accurately track deep space targets. However, as the need to explore more distant targets and obtain more information is raised, the caliber and volume of antennas are getting larger and larger, which makes the impact of originally weak system failures suddenly magnified. This requires that the servo control system data generated during the antenna installation and debugging process be processed and analyzed as early as possible through the antenna health management module during the on-site implementation stage to identify existing faults and reduce the occurrence of serious faults in the later stage.

[0003] The "intermittent disorder" fault phenomenon of antenna servo control is manifested as intermittent loss of control and deviation from the normal track during the operation of the antenna. With the amplification of multi-stage transmission, it will cause serious damage to the accuracy of the mechanism. However, this fault occurs when the antenna initially completes the primary rotation device, which is not intuitive. Given the debugging speed command, the antenna can rotate normally in both the forward and reverse directions. Even if senior experts monitor the collected data manually, it takes a lot of time to process multiple data separately and conduct joint comparison and analysis to make judgments. In order to improve efficiency, some studies have applied machine learning methods to fault diagnosis in similar scenarios, but the data that can be processed by such methods is of a single form, and these methods do not consider the coupling between related data and the correlation between previous and subsequent factors, which limits their application. Summary of the invention

[0004] In view of the problems that the existing technology has a single form of processable data and does not consider the coupling between data and the correlation between previous and subsequent factors, the present invention proposes a large-caliber antenna system health status monitoring and evaluation method based on multimodal data and deep learning, including the following steps:

[0005] Step 1: Start the antenna that has completed preliminary debugging to rotate at a constant speed, and collect multi-modal data of the antenna servo control system; perform fault calibration on the collected multi-modal data and establish a database;

[0006] Step 2: Weight fusion and alignment of the collected multimodal data, including the following steps:

[0007] Step 2.1: After completing the acquisition of each modal data in the Δt time period, the acquired modal data are spliced ​​into a data matrix with consistent row and column alignment;

[0008] Step 2.2: Determine the number of channels before fusion of each modality data, where the number of channels of the i-th modality data is

[0009] C i =f(h i ,w i ,γ i ,μ i )

[0010] Among them, C i is the number of channels, which is positively correlated with the four input information, which is the number of rows h of the i-th modal data matrix i , dimension w i , linear correlation coefficient between single-mode data and faults γ i And the importance index μ of the i-th modal data is set i ;

[0011] Step 2.3: Batch collect the modal data within N Δt time periods, and for each modal data x i The j-dimensional data x ij Perform normalization operation; when normalizing, count the x at all N moments ij The maximum value of and minimum value After normalization In subsequent new sample operations, if there is an The value of the range is first constrained to the range Normalize again;

[0012] Step 2.4: Create a mapping relationship to complete data alignment;

[0013] Step 3: Establish a fault detection model; the fault detection model includes a base layer single-modal data processing link, a middle layer multi-modal data fusion link and a top-level fault detection link;

[0014] The basic layer single-modal data processing link includes the basic backbone network and the attention module; the basic layer single-modal data processing link is used to extract the features of each single-modal data;

[0015] The middle-layer multimodal data fusion link includes the middle-layer backbone network and the attention module; the fused features obtained by splicing the feature channels through the middle-layer multimodal data fusion link are subjected to feature extraction;

[0016] The top-level fault detection link completes fault detection within the Δt time period through the Head network; the Head network output is a one-hot vector of dimension D+1, where D is the number of fault categories and 1 corresponds to the output dimension of no fault;

[0017] Step 4: Use sample data to train the fault detection model established in step 3;

[0018] Step 5: Perform network pruning and knowledge distillation on the fault detection model trained in step 4 to obtain a streamlined network structure;

[0019] Step 6: Use the fault detection model that has been streamlined in step 5 to monitor and evaluate the health status of the large-aperture antenna system. During the monitoring and evaluation process, refer to the structure of the time series data processing network to update the time series status and further optimize the monitoring and evaluation results.

[0020] Furthermore, in step 1, the collected multimodal data include the current of the antenna rotation drive motor, the antenna rotation speed, the antenna shaft angle, and the source code coarse value and source code fine value of the antenna rotation system encoder.

[0021] Furthermore, in step 2.2, the linear correlation coefficient γ between single-mode data and fault i Obtained through the following process:

[0022] For a certain modal data, a simple linear model between the single modal data and the fault label is established, the model is trained and the detection accuracy is evaluated, and the detection accuracy is used as the linear correlation coefficient γ between the single modal data and the fault i .

[0023] Furthermore, the mapping relationship in step 2.4 adopts a projection function, a fully connected neural network or an interpolation scaling function.

[0024] Furthermore, the specific process of step 2.4 is as follows:

[0025] Assume that the height and width of all modal data matrices after alignment are h and w, then each modal data x i The mapping relationship is

[0026]

[0027] Among them, H and W are both learnable mapping matrices, and their dimensions are h×h respectively. i and w i ×w;

[0028] Then the modal data Using C i A 1×1 convolution is performed to increase the number of channels from 1 to C while keeping the rows and columns aligned. i The expansion of is used to obtain the three-dimensional feature data of each modality after weight fusion and alignment.

[0029] Furthermore, during the training process in step 4, both the input data and the intermediate layer features are subjected to noise processing.

[0030] Furthermore, in step 5, during the knowledge distillation process, the network loss function setting process is:

[0031] According to the network conditions of each layer in the basic Backbone module and the intermediate Backbone module, they are split into R and K serial split modules according to the network layer combination, and the loss function between the corresponding modules of the unpruned model and the pruned model is defined, where the total loss function is defined as:

[0032] L total =L base +L middle +L head

[0033] Among them, L base and L middle To split the module loss, KL divergence is used as the loss function:

[0034]

[0035] L head It is the loss function between the head network output and the true value.

[0036] Furthermore, in step 6, the attention module of the middle-layer multimodal data fusion link is used as the temporal state update module, and in each time period Δt i+1 The timing status update module runs independently and only its output is updated in timing:

[0037] G i+1 (θ) = λg i+1 (θ)+(1-λ)G i (θ)

[0038] Among them, G i+1 (θ) is the updated temporal state update module output, G i (θ) is the output of the timing state update module in the previous time period before the update, g i+1 (θ) is the current Δt i+1 The output of the temporal state update module within a single time period of the time period, and λ is a weight coefficient between 0 and 1.

[0039] Beneficial Effects

[0040] The present invention conducts a comprehensive investigation of the multi-modal data of the front-to-back correlation and mutual coupling of the large-aperture antenna system, and uses deep learning technology to design a network module with a decoupling effect, that is, first independently extracting the features of each type of data in the first few layers, and then integrating them to achieve high-precision detection quality; and by analyzing the influence weights of related factors, assisting in problem finding and judgment, and realizing intelligent debugging. The present invention improves the efficiency of on-site work for large-aperture antenna installation and adjustment, reduces potential faults in the initial stage of implementation, and lays the foundation for stable operation and maintenance in the later stage, providing an effective means.

[0041] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0043] Figure 1 : Schematic diagram of fault detection model;

[0044] Figure 2 : Schematic diagram of network pruning for fault detection model;

[0045] Figure 3 : Sequential state update process structure diagram; DETAILED DESCRIPTION

[0046] Embodiments of the present invention are described in detail below. The embodiments are exemplary and intended to be used to explain the present invention, but should not be construed as limiting the present invention.

[0047] This embodiment includes the following steps:

[0048] Step 1: Start the antenna that has completed preliminary debugging to rotate at a uniform speed and collect multi-modal data of the antenna servo control system; perform fault calibration on the collected multi-modal data and establish a database.

[0049] For the antenna that has completed the preliminary installation and debugging, in the single motor mode, give a uniform rotation command, turn on the data recording function of the antenna servo control system, and collect the time, the current of the antenna rotation drive motor, the antenna speed, the antenna shaft angle, the source code coarse value and the source code fine value of the antenna rotation system encoder. Among them, the speed range of the antenna rotation drive motor is 0-2000r / min, the current feedback unit is %, the motor rated current range is 0-100%, the shaft angle value range is 0°~360°, if it turns to 360°, it will become 0° and start again. The value range of the source code coarse value and the source code fine value are both 0~65536, using hexadecimal numbers, the coarse code value rotates one circle (the value from 0 to 65536 is 1 circle), and the fine code value will rotate 16 circles. The shaft angle is the source code coarse value and the source code fine value converted to a real number after the hexadecimal operation is performed and then multiplied by the coefficient. The data collected in the previous project are manually calibrated to establish an initial database. Through manual calibration, the data can be classified into two categories, normal and abnormal; or multiple categories, normal and multiple abnormalities (locating the specific data source of the problem). The data collection and recording frequency is optional from 50ms to 1m.

[0050] Step 2: Perform weight fusion and alignment on the collected multimodal data.

[0051] Since the acquisition frequency of multimodal data is not completely consistent, in order to facilitate the use as effective input data in the subsequent deep learning process, this step splices, fuses and aligns the multimodal monitoring data with weights. In the data splicing stage, the single-channel matrix dimensions of each modal data collected per unit time are unified, and then the number of channels to be expanded for each modal data is calculated. After completing the channel expansion of each modal data, splice it according to the channel dimension to form multimodal fusion data.

[0052] Assume that the multimodal data collected within the task cycle time period Δt is x1, x2,…, x q , q is the modal number, for example, x1 represents the first modal data collected within the time period of Δt, for example, the current data of the antenna rotation drive motor.

[0053] After completing the acquisition of each modal data in the Δt time period, the acquired modal data are spliced ​​into a data matrix with consistent row and column alignment. The number of groups of each modal data acquisition and the dimension of the data itself are the number of rows and columns of the data matrix. For example, the acquired h i The i-th modal data of the group is arranged into h i Row, dimension w of the i-th modality data i is the number of matrix columns.

[0054] Then, the dimensional information of each modal data before fusion, that is, the number of channels, is determined based on the estimated weights. Since the weights of each modal data are different, the number of channels is also different. Weight estimation is mainly calculated based on three types of information: the data matrix dimension h of each modal data itself i and w i , the correlation between single modal data and faults estimated by simple linear model, and the importance index μ of each modal data set manually i For a certain modal data, a simple linear model between the single modal data and the fault label is established, the model is trained and the detection accuracy is evaluated, and the detection accuracy is used as the linear correlation coefficient γ between the single modal data and the fault i In this way, the number of channels of each modal data is constructed based on these three types of information:

[0055] C i =f(h i ,w i ,γ i ,μ i )

[0056] Among them, C i is the number of channels, which is positively correlated with the four input information and can adopt linear, logarithmic or exponential correlation.

[0057] Here is a specific example:

[0058] For the i-th modal data, according to the single modal data x i - Linear relationship between faults y, constructing linear functions

[0059] linear(x ij )=ω ij ·x ij

[0060] Collect batch samples corresponding to the i-th modal data Where j represents the order in which samples are collected, m i is the total number of batch samples corresponding to the i-th modal data. j Represents x ij The corresponding true output value. The optimal linear weight coefficient ω is calculated using the least squares method. ij , and judge its linear correlation through the mean square error function L:

[0061]

[0062] Define the linear correlation value as a function that is inversely proportional to the mean square error value:

[0063]

[0064] After obtaining the correlation coefficient and importance index, the number of channels after the i-th feature alignment can be defined as:

[0065] C i =f(h i ,w i ,γ i ,μ i )=g(h i ,w i )·η i

[0066] Where g(h i ,w i ) is the data matrix dimension h of the modal data itself i and w i Positively correlated functions, for example:

[0067]

[0068] By setting the number of channels, the number of channels corresponding to the modal data with higher importance and larger number of samples can be increased, thus enhancing its role in the subsequent prediction process. After confirming the number of channels for each modal data, perform the normalization operation on the collected data:

[0069] By batch collecting each modal data within N Δt time periods, and for each modal data x i The j-dimensional data x ij Perform a normalization operation, in which the x values ​​at all N moments are counted ij The maximum value of and minimum value After normalization In subsequent new sample operations, if there is an The value of the range is first constrained to the range Normalize again. After the normalization operation, the anti-overfitting effect during network training is better.

[0070] Then construct the mapping relationship (projection function, fully connected neural network or interpolation scaling function) to complete the data alignment.

[0071] Here, we take the projection function as an example, and set the height and width of all modal data matrices after alignment to h and w, then each modal data x i The mapping relationship is

[0072]

[0073] Among them, H and W are both learnable mapping matrices, and their dimensions are h×h respectively. i and w i ×w.

[0074] Given that 1×1 convolution has the property of flexibly changing the number of channels when fusing features at the same position between channels, 1×1 convolution is used here to complete the channel expansion of each modality data (from single channel to C i ), that is, for modal data Using C i A 1×1 convolution is performed to increase the number of channels from 1 to C while keeping the rows and columns aligned. i The expansion of is used to obtain the three-dimensional feature data of each modality after weight fusion and alignment.

[0075] Step 3: Build a fault detection model:

[0076] For each modal data after expansion and alignment, the next step is to perform feature extraction and fault monitoring evaluation through the fault detection model to finally obtain the evaluation results.

[0077] In the fault detection model, the preset basic backbone network and attention module are used to extract their respective features, and then the fused features are extracted based on the channel splicing of the middle-layer backbone network and attention module. Finally, the fault detection within the Δt time period is completed through the top-level head network.

[0078] Specifically, the overall structure of the fault detection model includes three processing steps:

[0079] (1) Basic layer single-modal data processing link, namely Figure 1 All modules before the feature channel splicing of the fault detection model: the basic Backbone module and the unimodal attention module corresponding to each modal data.

[0080] The role of this link is to use the basic Backbone module to extract the underlying feature information of each modal input data, and use the unimodal attention module to extract the saliency information of the underlying features. The basic Backbone module is used to extract the shallow feature fusion information of each local area of ​​each type of modal input data. In this embodiment, according to the data situation, a Backbone module with relatively more layers and parameters is used. The classic structures that can be used include ResNet-101 / 152 or Swin-L as the Backbone model. The unimodal attention module can use classic attention mechanism modules or networks such as basic Transformer or ViT to highlight the saliency information of a single module.

[0081] (2) The middle-layer multimodal data fusion link includes the intermediate module after the base-layer single-modal data processing link and before the top-layer Head network: the feature channels are spliced ​​to obtain the fused features, and then processed by the middle-layer Backbone module and attention module.

[0082] After completing the feature extraction of single modal data, the data features of each modality are spliced ​​in the channel direction to form fused features. On this basis, the features are fused at a deeper level to explore the correlation between different modal data and the potential impact and effect of fused features on fault conditions. The network structure in the middle layer is used to further extract features from multi-category fusion features, using Backbone modules with relatively fewer layers and parameters, such as ResNet-18 / 34 / 50 or Swin-B. The attention module uses the same attention module as the single modal data processing link.

[0083] (3) Top-level fault detection link, that is, the top-level Head network performs fault diagnosis.

[0084] The Head network is a functional output network used to predict various types of faults. The final output is D+1, where D is the number of fault categories and 1 is the output dimension corresponding to no fault. Since the final output is the probability value output by the Softmax function, the ideal state in the case of no fault is that the output is close to 1, and the output of other states is 0.

[0085] The true value corresponding to the fault data is a one-hot vector of dimension D+1. The network structure of this link can use 2-4 layers of fully connected network layers, or 1-2 layers of global average pooling network layers. Add a Softmax function after the network layer to convert the absolute value of the network output into a probability value, which is convenient for confirming whether there is a possibility of fault.

[0086] Furthermore, the fault classification can be matched with multi-source data, so that it is intuitive to determine which sensor has failed, and then confirm the cause of the fault through technical analysis and manual troubleshooting of the sensor. In this case, the number of fault categories D and the number of modes q of multi-source input are consistent. In this way, there is a one-to-one correspondence between the first D outputs of the model and the multi-source input. Through the multi-source feature fusion analysis of the network, the correlation between the features of different data sources can be analyzed, and through the comprehensive analysis of different source data, the source of abnormal data can be better located.

[0087] The above top-level fault detection link can also be weakened into a two-class problem, that is, normal and abnormal, when there is no need to analyze which data source has a problem. In this way, the dimension of the final network output is reduced to 2, which is only used to warn of faults, reduce the difficulty of network functions, and simultaneously improve the prediction accuracy.

[0088] Step 4: Use sample data to train the fault detection model established in step 3:

[0089] The loss function of the network training process is the Head network output L head The loss function between the true value and the true value can be a cross entropy loss function. The training process uses the stochastic gradient descent method for iterative training. When the loss function converges to an acceptable range and no longer decreases, the training is terminated and the final network parameters are obtained.

[0090] In the training phase of the network, due to the accuracy of the equipment itself and the influence of various external factors, there is a certain amount of true value disturbance noise in the training sample data. Therefore, a certain noise addition strategy can be adopted in the training phase to improve the model's ability to resist noise disturbance. The noise addition strategy here is a conventional method itself, but in this embodiment, not only is noise added at the data end, but also the features of the middle layer are appropriately noised, thereby improving the network's ability to resist disturbance.

[0091] For example, in the corresponding network model in step 3, in addition to adding appropriate noise to the input samples, there is also a corresponding noise addition operation on the middle layer. In each round of training, the output features of the concatenated feature channels of the middle layer and the input features of the top-level Head network can be noised according to the set rules. The specific operation plan is designed as follows:

[0092] This embodiment constrains the upper and lower extreme outlier thresholds for the collected training samples according to the possible value range of each dimension of the samples, and normalizes them to the range of [0,1]. Then, in different training rounds, according to a certain selection strategy, different amplitudes of Gaussian noise, uniform noise, and salt and pepper noise are added to improve the network's ability to resist various types of noise.

[0093] For the middle layer of the network structure, appropriate noise is added according to the variance of the features of each layer, and range constraints are added to the features after adding noise to ensure that the data after adding noise is not abnormal. For example, taking Gaussian noise as an example, for a certain layer of features, the mean of the added Gaussian noise is 0. Let the numerical variance of the features of this layer be σ, and the variance of the features of this layer be multiplied by a scaling factor γ, then for the i-th row and j-th column element x of the feature of the k-th channel ijk , the value after adding noise becomes x ijk ←x ijk +γσ. By adding noise in this intermediate layer, the network's anti-disturbance ability is further improved.

[0094] Step 5: Perform network pruning and knowledge distillation on the fault detection model trained in step 4 to obtain a streamlined network structure.

[0095] In order to make the detection method run smoothly on the front-end device, after the above network is trained to work normally, the basic Backbone module and the intermediate layer Backbone module are first pruned through the post-pruning algorithm. The number of layers of the network is not changed here, and the number of convolutions in each layer is reduced only when the test accuracy allows. First, fully train the unpruned network as a guiding teacher model. After pruning, the unpruned network is used to perform knowledge distillation on the pruned model, and the effect of the pruned network is improved through secondary learning. The distillation learning method adopted in the present invention is mainly to compare the KL divergence between the teacher model and the student model through the features extracted from multiple intermediate links while ensuring that the output of the student model is consistent with the true value. During the knowledge distillation process, the loss function setting process of the network is:

[0096] According to the network conditions of each layer in the basic Backbone module and the intermediate Backbone module, they are split into R and K serial split modules according to the network layer combination, and the loss function between the corresponding modules of the unpruned model and the pruned model is defined. The total loss function is defined as:

[0097] L total =L base +L middle +L head

[0098] Among them, L base and L middle To split the module loss, KL divergence (the feature difference between the corresponding layer features of the unabridged full teacher model and the simplified student model) is used. full ,feature simple ) as the loss function:

[0099]

[0100] L head It is the loss function between the head network output and the true value, and the cross entropy loss function can be used.

[0101] Step 6: Use the fault detection model that has been streamlined in step 5 to monitor and evaluate the health status of the large-aperture antenna system. During the monitoring and evaluation process, refer to the structure of the time series data processing network to update the time series status and further optimize the monitoring and evaluation results.

[0102] Here, the attention module (close to the top-level Head function module) of the second stage (intermediate multimodal data fusion link) can be used as a time series state update module. Through the state data update in the attention module, the time series data is inherited and gradually updated.

[0103] The time series state update module uses the attention module before the top-level Head network. This module is gradually updated during the process of data input in time series, and gradually updates and adjusts while maintaining the information contained in the time series.

[0104] For the timing state update module, in each time period Δt i+1 The timing status update module runs independently and only its output is updated in timing:

[0105] G i+1 (θ) = λg i+1 (θ)+(1-λ)G i (θ)

[0106] Among them, G i+1 (θ) is the updated state update module output, G i (θ) is the output of the state update module in the previous time period before the update, g i+1 (θ) is the current Δt i+1 The state update module output in a single time period of the time period, λ is the weight coefficient. The structure of the time series state update process of this scheme is as follows Figure 3 shown.

[0107] Finally, the above fault detection scheme is used to detect the operating status of the machinery in real time, find problems in time and perform maintenance to ensure the normal operation of the equipment.

[0108] The above overall process is summarized as follows:

[0109] First, the multimodal data in the initial time period Δt1 is weighted and aligned according to step 2 for the multimodal monitoring data of the large-aperture antenna system. Then, the processed data is feature fused through the single-period data processing network in step 3 until the fault judgment result output by the top-level network is obtained (there is no time series state fusion in the first time period).

[0110] In the subsequent time period Δt i In (i=2,3,…), the multimodal data collected each time is similar to the initial time period Δt1. According to step 2, the multimodal monitoring data of the large-aperture antenna system is weighted and aligned. Then, the processed data is feature fused through the single-period data processing network in step 3. The output of the timing update module is obtained by calculation, and the output result of the timing module and the previous time Δt are updated through the state update scheme in step 6. i-1 The fusion update of the state variables and output of the current time Δt iThe top-level Head network fault is targeted at the result output, and the proportion and distribution of the abnormal points are found to locate the potential fault data source. Through this solution, the state of time-series progressive transmission is used to complete the moment-by-moment result analysis of continuous time periods, and the working status of each data source module inside the antenna servo control system is monitored in real time. It provides an effective means to improve the efficiency of large-caliber antenna installation and adjustment on-site work, reduce fault risks in the early stage of implementation, and lay the foundation for stable operation and maintenance in the later stage.

[0111] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention without departing from the principles and intent of the present invention.

Claims

1. A method for monitoring and evaluating the health status of a large-caliber antenna system based on multimodal data and deep learning, characterized in that: The following steps are involved: Step 1: Start the antenna that has completed preliminary debugging to rotate at a constant speed and collect multi-modal data of the antenna servo control system; And perform fault calibration on the collected multi-modal data and establish a database; Step 2: Weight fusion and alignment of the collected multimodal data, including the following steps: Step 2.1: After completing the acquisition of each modal data in the Δt time period, the acquired modal data are spliced ​​into a data matrix with consistent row and column alignment; Step 2.2: Determine the number of channels before fusion of each modality data, where the number of channels of the i-th modality data is C i =f(h i ,w i ,c i ,m i ) Among them, C i is the number of channels, which is positively correlated with the four input information, which is the number of rows h of the i-th modal data matrix i , dimension w i , linear correlation coefficient between single-mode data and faults γ i And the importance index μ of the i-th modal data is set i ; Step 2.3: Batch collect the modal data within N Δt time periods, and for each modal data x i The j-dimensional data x ij Perform normalization operation; when normalizing, count the x at all N moments ij The maximum value of and minimum value After normalization In subsequent new sample operations, if there is an The value of the range is first constrained to the range Normalize again; Step 2.4: Create a mapping relationship to complete data alignment; Step 3: Establish a fault detection model; the fault detection model includes a base layer single-modal data processing link, a middle layer multi-modal data fusion link and a top-level fault detection link; The basic layer single-modal data processing link includes the basic backbone network and the attention module; the basic layer single-modal data processing link is used to extract the features of each single-modal data; The middle-layer multimodal data fusion link includes the middle-layer backbone network and the attention module; the fused features obtained by splicing the feature channels through the middle-layer multimodal data fusion link are subjected to feature extraction; The top-level fault detection link completes fault detection within the Δt time period through the Head network; the Head network output is a one-hot vector of dimension D+1, where D is the number of fault categories and 1 corresponds to the output dimension of no fault; Step 4: Use sample data to train the fault detection model established in step 3; Step 5: Perform network pruning and knowledge distillation on the fault detection model trained in step 4 to obtain a streamlined network structure; Step 6: Use the fault detection model that has been streamlined in step 5 to monitor and evaluate the health status of the large-aperture antenna system. During the monitoring and evaluation process, refer to the structure of the time series data processing network to update the time series status and further optimize the monitoring and evaluation results.

2. According to claim 1, a large-caliber antenna system health status monitoring and evaluation method based on multimodal data and deep learning is characterized by: In step 1, the collected multimodal data include the current of the antenna rotation drive motor, the antenna rotation speed, the antenna shaft angle, and the source code coarse value and source code fine value of the antenna rotation system encoder.

3. According to claim 1, a large-caliber antenna system health status monitoring and evaluation method based on multimodal data and deep learning is characterized by: In step 2.2, the linear correlation coefficient γ between single-mode data and fault i Obtained through the following process: For a certain modal data, a simple linear model between the single modal data and the fault label is established, the model is trained and the detection accuracy is evaluated, and the detection accuracy is used as the linear correlation coefficient γ between the single modal data and the fault i .

4. According to claim 1, a large-caliber antenna system health status monitoring and evaluation method based on multimodal data and deep learning is characterized by: The mapping relationship in step 2.4 uses a projection function, a fully connected neural network or an interpolation scaling function.

5. According to claim 4, a large-caliber antenna system health status monitoring and evaluation method based on multimodal data and deep learning is characterized by: The specific process of step 2.4 is as follows: Assume that the height and width of all modal data matrices after alignment are h and w, then each modal data x i The mapping relationship is Among them, H and W are both learnable mapping matrices, and their dimensions are h×h respectively. i and w i ×w; Then the modal data Using C i A 1×1 convolution is performed to increase the number of channels from 1 to C while keeping the rows and columns aligned. i The expansion of is used to obtain the three-dimensional feature data of each modality after weight fusion and alignment.

6. According to claim 1, a large-caliber antenna system health status monitoring and evaluation method based on multimodal data and deep learning is characterized by: During the training process in step 4, both the input data and the intermediate layer features are subjected to noise processing.

7. According to claim 1, a large-caliber antenna system health status monitoring and evaluation method based on multimodal data and deep learning is characterized by: In step 5, during the knowledge distillation process, the network loss function setting process is: According to the network conditions of each layer in the basic Backbone module and the intermediate Backbone module, they are split into R and K serial split modules according to the network layer combination, and the loss function between the corresponding modules of the unpruned model and the pruned model is defined, where the total loss function is defined as: L total =L base +L middle +L head Among them, L base and L middle To split the module loss, KL divergence is used as the loss function: L head It is the loss function between the head network output and the true value.

8. According to claim 1, a large-caliber antenna system health status monitoring and evaluation method based on multimodal data and deep learning is characterized by: In step 6, the attention module of the middle-layer multimodal data fusion link is used as the temporal state update module. i+1 The timing status update module runs independently and only its output is updated in timing: G i+1 (θ)=λg i+1 (θ)+(1-λ)G i (i) Among them, G i+1 (θ) is the updated temporal state update module output, G i (θ) is the output of the timing state update module in the previous time period before the update, g i+1 (θ) is the current Δt i+1 The output of the temporal state update module within a single time period of the time period, and λ is a weight coefficient between 0 and 1.

Images