Structural health monitoring abnormal data diagnosis method based on multi-task hybrid expert vision Transform

Abstract

The invention discloses a structural health monitoring abnormal data diagnosis method based on a multi-task mixed expert vision Transform. The method comprises the following steps: converting a one-dimensional monitoring time sequence of a plurality of bridges into a two-dimensional image, obtaining a Token sequence through patch embedding, and superposing position codes; an MT-MoE ViT shared trunk is constructed, a task awareness dynamic gating route is introduced into a hybrid expert attention module, Token representation and a bridge task are embedded and fused, Gumbel-Softmax and Top-k sparse activation are adopted to realize expert dynamic allocation, and expert feature extraction and weighted fusion are completed; in the training stage, a joint target including classification loss, load balancing loss and Z-Loss is constructed, and GradNorm is adopted to adaptively balance multi-task gradient conflicts and training imbalance; and in the reasoning stage, an abnormal category judgment result of the corresponding bridge is output through the task specific classification head. According to the method, multi-bridge collaborative anomaly diagnosis is realized under a single model framework, cross-bridge sharing and bridge differences are considered, the multi-task training stability and the expert load balancing capability are improved, the engineering deployment and maintenance cost is reduced, and the method has good engineering applicability.

Description

Technical Field

[0001] This invention belongs to the fields of deep learning, abnormal data diagnosis, and structural health monitoring technology, and specifically relates to a structural health monitoring and abnormal data diagnosis method based on a multi-task hybrid expert vision Transformer. Background Technology

[0002] With the rapid development of transportation infrastructure construction, centralized management of regional or city-level bridge clusters has become an important trend in the field of structural health monitoring. Monitoring centers often need to monitor dozens or even hundreds of bridges simultaneously to ensure the safe operation of the transportation network. Structural health monitoring systems diagnose and assess abnormal bridge conditions by collecting long-term sensor data such as acceleration, displacement, and strain. However, existing methods for diagnosing abnormal data face significant challenges when dealing with collaborative monitoring scenarios involving large-scale bridge clusters.

[0003] During the long-term service of civil engineering infrastructure such as large bridges, monitoring systems typically need to continuously collect multi-source response data to support tasks such as structural condition assessment, damage early warning, and long-term performance evaluation. However, due to factors such as sensor aging and drift, acquisition equipment failure, communication link packet loss and jitter, external electromagnetic interference, and the strong time-varying nature of environmental and traffic loads, noise pollution, abnormal segments, distribution drift, and label confusion inevitably occur in the monitoring sequence, thereby weakening the reliability of subsequent diagnostic conclusions.

[0004] To address the problem of diagnosing abnormal data in structural health monitoring, existing technologies typically employ threshold discrimination, statistical feature analysis, and traditional machine learning classification methods. Some studies have also introduced deep learning models to transform one-dimensional time-series signals into two-dimensional images for end-to-end feature learning and anomaly identification. While these methods are effective for single-bridge or relatively stable data distribution scenarios, in the context of collaborative monitoring of bridge groups, significant differences exist between bridges in their structural systems, dynamic characteristics, sensor deployment, and operating conditions, leading to marked heterogeneity in monitoring data distribution and anomaly patterns. Furthermore, the anomaly category spaces of each bridge are often inconsistent, making it difficult to directly share a unified output space. Training an independent model for each bridge would result in a large number of models, high training and maintenance costs, and complex engineering deployment. Using transfer learning or a unified single model for forced generalization can easily lead to unstable transfer or negative transfer phenomena, making it difficult to maintain stable recognition performance across multiple bridges simultaneously.

[0005] Multi-task learning can reveal the sharing and differences between modeling tasks, extract common features of bridges through shared representations, and retain bridge differences with the help of task-specific modules. However, key bottlenecks still exist in multi-bridge anomaly diagnosis: First, the sample size, category distribution, and difficulty of each task vary significantly, and direct joint optimization can easily lead to gradient conflicts and training imbalance, causing the model to favor tasks with larger data volumes; Second, the introduction of dynamic routing mechanisms such as hybrid experts may result in uneven expert load and routing collapse, causing model capacity waste and performance degradation. Therefore, there is an urgent need for an anomaly data diagnosis method for collaborative monitoring of bridge groups, which can take into account both the sharing and individual differences of bridges within a unified framework, adapt to inconsistent category spaces, and improve engineering usability and deployment efficiency. Summary of the Invention

[0006] The purpose of this invention is to provide a structural health monitoring anomaly data diagnosis method based on multi-task hybrid expert vision Transformer, which can solve the problems of existing single-bridge / single-task anomaly diagnosis models that are difficult to take into account the structural heterogeneity of multiple bridges, the spatial inconsistency of anomaly types, and the high deployment cost of cross-bridges. The aim is to achieve collaborative learning and high-precision identification of anomaly patterns of multiple bridges under a unified network framework, while improving the stability of multi-task training and the efficiency of engineering deployment.

[0007] This invention is achieved through the following technical solution: A method for diagnosing structural health monitoring anomaly data based on a multi-task hybrid expert vision Transformer, the method comprising the following steps: Step 1: Obtain one-dimensional structural health monitoring time series data of multiple bridges and convert the time series data into two-dimensional image samples; using "bridge" as the task granularity, define each bridge as an independent subtask and construct a multi-task dataset; use the image embedding module to divide the two-dimensional image into non-overlapping local blocks and map them into token sequences, while superimposing learnable positional codes to obtain the Transformer input sequence. Step 2: Construct a shared backbone network based on visual Transformer. Introduce a hybrid expert attention module and a feedforward neural network module into the Transformer block to form an MT-MoEViT network, which is used to extract common and differential features across bridging tasks in the shared feature space. Step 3: In the hybrid expert attention module introduced in Step 2, the token representation and the bridge task number embedding vector are fused, the expert routing score is calculated through a gating network, and the Gumbel-Softmax strategy is used to achieve differentiable sparse sampling to obtain the expert selection probability distribution. Step 4: Based on the probability distribution of expert selection in Step 3, the Top-k strategy is adopted to activate only the top k most relevant expert subnetworks to generate query vectors, while the key vector and value vector are shared among all experts; the sparse selection probability is used to perform weighted fusion of the multiple expert query outputs, and feature interaction and extraction are completed through a multi-head attention mechanism; Step 5: Construct a joint optimization objective function that includes classification loss, load balancing loss and auxiliary regularization loss Z-Loss, and introduce the GradNorm algorithm to dynamically adjust the loss weight of each bridge task to balance the optimization rate between different tasks. At the same time, use the load balancing loss to constrain the expert activation frequency to prevent expert collapse. Step 6: Set up a task-specific classification head for each bridge subtask, input the features extracted by the deep network into the task-specific classification head that matches the corresponding bridge task, output the anomaly category prediction probability of the bridge task and give the anomaly category judgment result, so as to realize multi-bridge collaborative anomaly diagnosis.

[0008] Furthermore, the image embedding module in step one satisfies the following definition: Represent the input image as a tensor Using patch size Get the number of patches:

[0009] No. Each patch is expanded to Its token is embedded as follows:

[0010] And superimposed with learnable positional encoding The input Token is:

[0011] In the formula, Image height, Image width, This refers to the number of image channels; This represents the number of patches. It is a linear projection matrix. For bias terms; For the first The position encoding vector corresponding to each patch; This is the token representation after adding position encoding.

[0012] Furthermore, the task-aware gated routing in step three satisfies the following definition: Gated networks for the first Tokens Output the expert score vector, along with the task number. The task bias vector is obtained through the task embedding network, and the two are fused to obtain the routing score:

[0013] The expert selection probability is obtained using Gumbel-Softmax:

[0014] In the formula, For the first Expert routing logits for each token; A gating mapping from tokens to expert scores; For the first Task embedding vectors for each task; For task bias mapping; For the first The token is selected from the first... The probability of an expert; For the number of experts; Random noise sampled from a standard Gumbel distribution; This is the temperature coefficient.

[0015] Furthermore, the attention calculation in step four, which features "independent expert queries and shared key / value pairs," satisfies the following: No. The query generated by the expert is:

[0016] The shared key and value are:

[0017] In the formula, The input is a matrix of token sequences; For the first Each expert has an independent query mapping matrix; Map weights to the Key, which is shared by all experts; Value mapping weights are shared by all experts; For the number of experts.

[0018] Furthermore, the Top-k sparse activation and weighted fusion in step four satisfy the following definition: For the first Of the tokens, only the ones with the highest probability are selected. An expert constitutes the activation set And normalize the probabilities within the set to The merged query is:

[0019] The attention weights and outputs are calculated using multi-head attention.

[0020] In the formula, For the first The expert on the first The query vector generated by each token; It is a matrix composed of queries from various tokens; The feature dimension of a single attention head; This is a transpose operation; This is the attention weight matrix; For attention output.

[0021] Furthermore, the multi-task joint loss function in step five satisfies the following definition: No. The cross-entropy classification loss for each task is:

[0022] GradNorm calculates the... Weighted gradient norm for each task:

[0023] The load balancing loss is:

[0024] Z-Loss is:

[0025] The total loss is:

[0026] In the formula, For the first Number of samples per task; For the first Number of exception categories for each task; For genuine one-hot tags; To predict probabilities; For a shared set of parameters; Dynamic task weights; It is an L2 norm; For the first The expected activation frequency of each expert; For the first The actual activation frequency of each expert; and These are the hyperparameter weighting coefficients.

[0027] Furthermore, the task-specific classification head in step six satisfies: The classification prediction output for each bridge task is:

[0028] and with

[0029] Obtain the anomaly category determination result; In the formula, For the first The task category header; The task feature vector is input to the classification head; For the first The probability distribution of each task category; For the predicted category.

[0030] A structural health monitoring anomaly data diagnosis system based on a multi-task hybrid expert vision Transformer is disclosed. The system employs the aforementioned structural health monitoring anomaly data diagnosis method based on a multi-task hybrid expert vision Transformer and comprises the following six steps: Data preprocessing and sequence embedding module: acquire one-dimensional structural health monitoring time series data of multiple bridges, convert the time series data into two-dimensional image samples; define each bridge as an independent subtask with "bridge" as the task granularity, and construct a multi-task dataset; use the image embedding module to divide the two-dimensional image into non-overlapping local blocks and map them into token sequences, and superimpose learnable position codes to obtain the Transformer input sequence. Multi-task hybrid expert network architecture building module: Construct a shared backbone network based on visual Transformer, and introduce a hybrid expert attention module and a feedforward neural network module into the Transformer block to form an MT-MoEViT network, which is used to extract common and differential features across bridging tasks in the shared feature space; Task-aware dynamic gating routing module: In the hybrid expert attention module introduced in step two, the token representation and the bridge task number are fused into the embedding vector. The expert routing score is calculated through the gating network, and the Gumbel-Softmax strategy is used to achieve differentiable sparse sampling to obtain the expert selection probability distribution. Expert Feature Extraction and Weighted Fusion Module: Based on the probability distribution of expert selection in step three, the Top-k strategy is adopted to activate only the top k most relevant expert sub-networks to generate query vectors, while the key vector and value vector are shared among all experts; the sparse selection probability is used to perform weighted fusion of the multiple expert query outputs, and feature interaction and extraction are completed through a multi-head attention mechanism; Multi-task collaborative optimization training module: Construct a joint optimization objective function that includes classification loss, load balancing loss and auxiliary regularization loss Z-Loss, and introduce the GradNorm algorithm to dynamically adjust the loss weights of each bridge task to balance the optimization rates between different tasks. At the same time, the load balancing loss is used to constrain the expert activation frequency to prevent expert collapse. Anomaly diagnosis decision output module: Set a task-specific classification head for each bridge subtask, input the features extracted by the deep network into the task-specific classification head that matches the corresponding bridge task, output the anomaly category prediction probability of the bridge task and give the anomaly category judgment result, so as to realize multi-bridge collaborative anomaly diagnosis.

[0031] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method described above.

[0032] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0033] The beneficial effects of this invention are as follows: This invention addresses the problem of negative transfer and optimization conflict between multiple tasks: It proposes a shared and specialized parallel architecture based on MT-MoE ViT. Compared with simple joint training, through a task-aware hybrid expert mechanism, the model can extract common features of bridges while using specific experts to handle the structural differences and working conditions of different bridges, effectively avoiding interference and negative transfer effects between heterogeneous tasks.

[0034] This invention improves the system's scalability and resource utilization: by employing a single backbone network to collaboratively monitor multiple bridges, compared to the traditional method of deploying models independently for each bridge, it significantly reduces computational resource redundancy and maintenance costs. The sparse activation characteristics of the MoE mechanism allow for increased model capacity without significantly increasing inference computation, making it suitable for large-scale bridge group engineering deployments.

[0035] This invention enhances the robustness of the model under imbalanced data: Addressing the extreme imbalance in sample size and identification difficulty among bridge monitoring data, this invention introduces the GradNorm dynamic weight adjustment strategy and load balancing loss. This ensures the relative balance of gradients for each task during training, preventing simple tasks from dominating gradient updates or long-tail tasks from being forgotten, significantly improving the model's diagnostic accuracy for minority class anomalies and complex operating conditions.

[0036] This invention achieves interpretability and hierarchical feature learning in expert selection: by combining Gumbel-Softmax and Top-k routing strategies, the model can automatically learn the "task-expert" mapping relationship. The shallow network explores multi-scale features through high-entropy distribution, while the deep network locks in specific experts for decision-making through low-entropy distribution. This hierarchical expert collaboration mode not only improves generalization performance but also endows the model with explicit physical interpretability in multi-bridge monitoring scenarios. Attached Figure Description

[0037] Figure 1 This is a flowchart of the method of the present invention.

[0038] Figure 2 This is a diagram of the MT-MoE ViT network architecture of the present invention.

[0039] Figure 3 This is the MoE attention layer of the present invention.

[0040] Figure 4 This is a schematic diagram of the task-aware dynamic gating routing mechanism of the present invention.

[0041] Figure 5 A schematic diagram A shows the confusion matrix of the diagnostic results of abnormal bridge data in this invention, where (a) is the result for bridge A, (b) is the result for bridge B, (c) is the result for bridge C, (d) is the result for bridge D, (e) is the result for bridge E, and (f) is the result for bridge F. Figure 6 This is a schematic diagram B of the confusion matrix of the diagnostic results of abnormal data of each bridge in this invention, where (g) is the result of bridge G, (h) is the result of bridge H, and (i) is the result of bridge I. Detailed Implementation

[0042] To make the technical solution and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail and completely below with reference to the accompanying drawings.

[0043] Specific Implementation Method 1: This implementation method provides a structural health monitoring and abnormal data diagnosis method based on a multi-task hybrid expert vision Transformer. For example... Figure 1As shown, this method mainly includes six stages: data preprocessing and sequence embedding, MT-MoE ViT network construction, task-aware dynamic routing, expert attention calculation, multi-task collaborative optimization training, and anomaly diagnosis output. The specific implementation steps are as follows.

[0044] Step 1: Data Preprocessing and Sequence Embedding. One-dimensional structural health monitoring time-series data from multiple bridges are acquired and converted into two-dimensional image format. Using "bridge" as the task granularity, each bridge is defined as an independent subtask, forming a multi-task training dataset. The image embedding module is used to divide the two-dimensional images into non-overlapping local blocks, mapping them to high-dimensional token sequences, and superimposing learnable positional codes to preserve spatial structural information.

[0045] Step 2: Construct a multi-task hybrid expert network architecture: Build a backbone network based on the Visual Transformer (ViT), and introduce a hybrid expert (MoE) mechanism into the Transformer block to design a multi-task hybrid expert vision Transformer (MT-MoE ViT) network. This network consists of stacked hybrid expert attention modules and feedforward neural network modules, used to dynamically extract common and specific features across bridges in a shared feature space.

[0046] Step 3: Design Task-Aware Dynamic Gated Routing: In the hybrid expert attention module, a dynamic routing mechanism based on joint modeling of token representation and task conditions is designed. The image token features are fused with the corresponding bridge task number embedding vector. The expert selection score is calculated through a gating network, and a Gumbel-Softmax strategy is introduced for differentiable sparse sampling to achieve dynamic expert allocation based on task semantics.

[0047] Step 4: Expert Feature Extraction and Weighted Fusion: Based on the probability distribution generated by the routing mechanism, a Top-k strategy is adopted to activate only the top k most relevant expert sub-networks to generate query representations, while the key and value are shared among all experts. The expert outputs are weighted and fused using sparse selection probability, and feature interaction and extraction are completed through a multi-head attention mechanism.

[0048] Step 5: Multi-task collaborative optimization training: Construct a joint optimization objective function that includes classification loss, load balancing loss, and auxiliary regularization loss (Z-Loss). During training, the GradNorm algorithm is introduced to dynamically calculate and adjust the gradient weights of each bridge task, balancing the optimization rates among different tasks. At the same time, the load balancing loss is used to constrain the expert activation frequency to prevent expert collapse.

[0049] Step 6, Anomaly Diagnosis Decision Output: Input the feature sequence extracted by the deep network into the classification head designed independently for each bridge, and output the anomaly category judgment result of the monitoring data corresponding to each bridge to complete the multi-bridge collaborative anomaly diagnosis.

[0050] like Figure 2 As shown, further, step one specifically includes: The data preprocessing and sequence embedding include converting the one-dimensional structural health monitoring time series of multiple bridges into two-dimensional images, and further generating a token sequence that can be input into a visual Transformer.

[0051] 1. Two-dimensional image construction Samples are obtained by slicing the one-dimensional monitoring sequence according to time windows. And convert it into a two-dimensional image according to a preset mapping rule. .in, For the image height and width, This represents the number of channels.

[0052] 2. Patch Embedding Input image Divide into non-overlapping patches, with a patch size of The number of patches is . No. The patch flattening vector is The token embedding is obtained through linear projection.

[0053] In the formula, For trainable projection matrices, For bias terms, For the embedded dimension.

[0054] 3. Position Embedding Introducing Learnable Position Embeddings The final input sequence is obtained by adding each element of the token to the input sequence:

[0055] This leads to the Transformer input matrix. .

[0056] 4. Task Identifier (Bridge ID) Using "bridge" as the task granularity, and setting the total number of tasks to be... , No. The bridge corresponds to the task number Its task embedding vector is denoted as This is used for subsequent task-aware routing.

[0057] Furthermore, step two specifically involves: The MT-MoE ViT network consists of multiple stacked Transformer Blocks, each containing a hybrid expert attention module and a feedforward neural network module, and employs layer normalization and residual connections. Let the... Layer input is The output of the hybrid expert attention sublayer is:

[0058] The output of the feedforward neural network (FFN) sublayer is:

[0059] In the formula, For layer normalization; Discarded for random paths; It is a hybrid expert attention module; the FFN adopts two fully connected layers and GELU activation, and its per-token calculation form is as follows:

[0060] In the formula, For a trainable parameter matrix, This is a bias term.

[0061] like Figure 3 and Figure 4 As shown, further, steps three and four are as follows: The core of the hybrid expert attention module lies in the expert-oriented extension of the query projection. It determines the selection of experts by jointly modeling token representations and task conditions. The specific calculation steps are as follows: 1. Independent expert queries, shared key / value pairs. Let the number of experts be... , for input ,structure:

[0062]

[0063] In the formula, For the first Each expert has an independent query mapping matrix; A key-value mapping matrix shared by all experts.

[0064] 2. Task-aware routing score fusion For the first Token characteristics With task embedding Gated network output converged routing score:

[0065] In the formula, For the first token pairs The routing logits of an expert; A gated mapping for generating expert scores from token features; For task bias mapping.

[0066] 3. Gumbel-Softmax enables differentially sparse sampling. right Inject Gumbel noise The expert selection probability is obtained by applying softmax:

[0067] In the formula, For the first The token selection is the first one. The probability of an expert; This is the temperature coefficient.

[0068] 4. Fusion of Top-k Sparse Activation and Query Weighted Activation Only activate each token before An expert, let the activation set be... and normalize the probability to (Only in) (Unified) The final merged query is:

[0069] In the formula, For the first The expert on the first The query vector generated by each token.

[0070] 5. Multi-head attention calculation Taking a single head as an example, the attention weights and output are as follows:

[0071] In the formula, For a single attention head dimension; It is a matrix composed of queries from various tokens.

[0072] Furthermore, step five specifically includes: 1. Task classification loss Let the first The number of samples for each task is The number of anomaly categories is The true one-hot label is and the predicted probability is . Then the cross-entropy loss is:

[0073] 2. GradNorm Dynamic Task Weights For shared parameter sets , No. The norm of the weighted loss gradient for each task is:

[0074] And dynamically adjust by minimizing the objective function. :

[0075] In the formula, For the first Dynamic weights for each task; The mean of the gradient norm for each task; The task-relative training rate factor; This is the sensitivity adjustment parameter.

[0076] 3. Load Balancing Loss Record No. The expected activation frequency for each expert is:

[0077] The actual activation frequency (given by Top-k selection) is:

[0078] The load balancing loss can then be defined in the form of frequency mean square error:

[0079] 4. Z-Loss Auxiliary Regular Expression To stabilize the range of gated logits values, a Z-Loss is applied to the routing logits:

[0080] 5. Joint losses from multiple tasks Combining the above loss terms, the total loss function is:

[0081] In the formula, The load balancing loss weighting coefficient; The Z-Loss weighting coefficients.

[0082] Furthermore, step six specifically involves: To accommodate the inconsistent anomaly category spaces across different bridge-related tasks, a task-specific classification head is implemented. Let the classification feature vector output by the deep network be... , No. The task category header is The predicted probability is:

[0083] and with

[0084] Output the anomaly category determination result; In the formula, For the first The task category header; The task feature vector is input to the classification head; For the first The probability distribution of each task category; For the predicted category. Specific implementation examples: This embodiment applies the structural health monitoring anomaly data diagnosis method based on multi-task hybrid expert vision Transformer proposed in this invention to the real health monitoring system of nine long-span cable-stayed bridges to verify the engineering applicability and robustness of the method in the bridge group scenario.

[0086] Monitoring data from nine typical long-span bridges in Guizhou Province (covering various structural types such as suspension bridges, cable-stayed bridges, and continuous rigid frame bridges) were selected as the research object. The dataset includes multi-dimensional monitoring channels such as acceleration, deflection, temperature, and humidity. The samples cover various anomaly patterns, including normal, missing, outlier, drift, abrupt change, and trend, and have significant structural heterogeneity, multi-source sensor, and uneven distribution of anomaly categories.

[0087] Experimental results are as follows Figure 5 As shown, the method of this invention exhibits excellent diagnostic performance in anomaly identification tasks for all nine bridges. The accuracies on the test sets for the nine bridges are 92.1%, 93.1%, 93.9%, 94.1%, 91.6%, 92.2%, 94.4%, 95.9%, and 92.1%, respectively; the overall average accuracy reaches 93.0%, and the average F1 score is 91.1%. Notably, despite significant differences in structural form and data distribution among the different bridges, the performance fluctuation of the model across tasks remains within 4%, and no significant performance degradation occurs in a single task, demonstrating that the model has extremely strong generalization ability and robustness across bridges under multi-source heterogeneous data conditions.

[0088] In summary, the method of the present invention is compatible with the physical characteristics and monitoring data features of different bridge structures, and achieves stable, consistent and high-precision multi-bridge collaborative diagnosis under complex working conditions in real engineering scenarios.

[0089] Implementation Method Two: This embodiment provides a structural health monitoring anomaly data diagnosis system based on a multi-task hybrid expert vision Transformer. The diagnosis system uses the structural health monitoring anomaly data diagnosis method based on a multi-task hybrid expert vision Transformer as described in Embodiment 1. The diagnosis system includes the following six steps: Data preprocessing and sequence embedding module: acquire one-dimensional structural health monitoring time series data of multiple bridges, convert the time series data into two-dimensional image samples; define each bridge as an independent subtask with "bridge" as the task granularity, and construct a multi-task dataset; use the image embedding module to divide the two-dimensional image into non-overlapping local blocks and map them into token sequences, and superimpose learnable position codes to obtain the Transformer input sequence. Multi-task hybrid expert network architecture building module: Construct a shared backbone network based on visual Transformer, and introduce a hybrid expert attention module and a feedforward neural network module into the Transformer block to form an MT-MoEViT network, which is used to extract common and differential features across bridging tasks in the shared feature space; Task-aware dynamic gating routing module: In the hybrid expert attention module introduced in step two, the token representation and the bridge task number are fused into the embedding vector. The expert routing score is calculated through the gating network, and the Gumbel-Softmax strategy is used to achieve differentiable sparse sampling to obtain the expert selection probability distribution. Expert Feature Extraction and Weighted Fusion Module: Based on the probability distribution of expert selection in step three, the Top-k strategy is adopted to activate only the top k most relevant expert sub-networks to generate query vectors, while the key vector and value vector are shared among all experts; the sparse selection probability is used to perform weighted fusion of the multiple expert query outputs, and feature interaction and extraction are completed through a multi-head attention mechanism; Multi-task collaborative optimization training module: Construct a joint optimization objective function that includes classification loss, load balancing loss and auxiliary regularization loss Z-Loss, and introduce the GradNorm algorithm to dynamically adjust the loss weights of each bridge task to balance the optimization rates between different tasks. At the same time, the load balancing loss is used to constrain the expert activation frequency to prevent expert collapse. Anomaly diagnosis decision output module: Set a task-specific classification head for each bridge subtask, input the features extracted by the deep network into the task-specific classification head that matches the corresponding bridge task, output the anomaly category prediction probability of the bridge task and give the anomaly category judgment result, so as to realize multi-bridge collaborative anomaly diagnosis.

[0090] Implementation Method 3: This invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. The memory stores software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory and processor are connected via a bus. Specifically, the processor implements any step in Embodiment 1 by running the computer program stored in the memory.

[0091] It should be understood that, in the embodiments of the present invention, the processor may be a Central Processing Unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0092] Memory may include read-only memory, flash memory, and random access memory, and provides instructions and data to the processor. Some or all of the memory may also include non-volatile random access memory.

[0093] It should be understood that if the integrated modules / units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods described above can also be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0094] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0095] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0096] It should be noted that the methods and detailed examples provided in the above embodiments can be incorporated into the apparatus and devices provided in the embodiments for mutual reference, and will not be repeated here.

[0097] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0098] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units described above is merely a logical functional division, and in actual implementation, it can be divided in other ways. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0099] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for diagnosing structural health monitoring anomaly data based on a multi-task hybrid expert vision Transformer, characterized in that, The diagnostic method includes the following steps: Step 1: Obtain one-dimensional structural health monitoring time series data of multiple bridges and convert the time series data into two-dimensional image samples; using "bridge" as the task granularity, define each bridge as an independent subtask and construct a multi-task dataset; use the image embedding module to divide the two-dimensional image into non-overlapping local blocks and map them into token sequences, while superimposing learnable positional codes to obtain the Transformer input sequence. Step 2: Construct a shared backbone network based on visual Transformer. Introduce a hybrid expert attention module and a feedforward neural network module into the Transformer block to form an MT-MoEViT network, which is used to extract common and differential features across bridging tasks in the shared feature space. Step 3: In the hybrid expert attention module introduced in Step 2, the token representation and the bridge task number embedding vector are fused, the expert routing score is calculated through a gating network, and the Gumbel-Softmax strategy is used to achieve differentiable sparse sampling to obtain the expert selection probability distribution. Step 4: Based on the probability distribution of expert selection in Step 3, the Top-k strategy is adopted to activate only the top k most relevant expert subnetworks to generate query vectors, while the key vector and value vector are shared among all experts; the sparse selection probability is used to perform weighted fusion of the multiple expert query outputs, and feature interaction and extraction are completed through a multi-head attention mechanism; Step 5: Construct a joint optimization objective function that includes classification loss, load balancing loss and auxiliary regularization loss Z-Loss, and introduce the GradNorm algorithm to dynamically adjust the loss weight of each bridge task to balance the optimization rate between different tasks. At the same time, use the load balancing loss to constrain the expert activation frequency to prevent expert collapse. Step 6: Set up a task-specific classification head for each bridge subtask, input the features extracted by the deep network into the task-specific classification head that matches the corresponding bridge task, output the anomaly category prediction probability of the bridge task and give the anomaly category judgment result, so as to realize multi-bridge collaborative anomaly diagnosis.

2. The structural health monitoring and abnormal data diagnosis method based on multi-task hybrid expert vision Transformer according to claim 1, characterized in that, The image embedding module in step one satisfies the following definition: Represent the input image as a tensor Using patch size Get the number of patches: No. Each patch is expanded to Its token is embedded as follows: And superimposed with learnable positional encoding The input Token is: In the formula, Image height, Image width, This refers to the number of image channels; This represents the number of patches. It is a linear projection matrix. For bias terms; For the first The position encoding vector corresponding to each patch; This is the token representation after adding position encoding.

3. The structural health monitoring and abnormal data diagnosis method based on a multi-task hybrid expert vision Transformer according to claim 1, characterized in that, The task-aware gated routing in step three satisfies the following definition: Gated networks for the first Tokens Output the expert score vector, along with the task number. The task bias vector is obtained through the task embedding network, and the two are fused to obtain the routing score. The expert selection probability was obtained using Gumbel-Softmax. In the formula, For the first Expert routing logits for each token; A gating mapping from tokens to expert scores; For the first Task embedding vectors for each task; For task bias mapping; For the first The token is selected from the first... The probability of an expert; For the number of experts; Random noise sampled from a standard Gumbel distribution; This is the temperature coefficient.

4. The structural health monitoring and abnormal data diagnosis method based on a multi-task hybrid expert vision Transformer according to claim 1, characterized in that, The attention calculation in step four, which features "independent expert queries and shared key / value pairs," satisfies the following: No. The query generated by the expert is: The shared key and value are: In the formula, The input is a matrix of token sequences; For the first Each expert has an independent query mapping matrix; Map weights to the Key shared by all experts; Value mapping weights are shared by all experts; For the number of experts.

5. The structural health monitoring and abnormal data diagnosis method based on a multi-task hybrid expert vision Transformer according to claim 4, characterized in that, The Top-k sparse activation and weighted fusion in step four satisfy the following definition: For the Of the tokens, only the ones with the highest probability are selected. An expert group constitutes the activation set. And normalize the probabilities within the set to The merged query is: The attention weights and outputs are calculated using multi-head attention. In the formula, For the first The expert on the first The query vector generated by each token; It is a matrix composed of queries from various tokens; The feature dimension of a single attention head; This is a transpose operation; This is the attention weight matrix; For attention output.

6. The structural health monitoring and abnormal data diagnosis method based on a multi-task hybrid expert vision Transformer according to claim 1, characterized in that, The multi-task joint loss function in step five satisfies the following definition: No. The cross-entropy classification loss for each task is: GradNorm calculates the... Weighted gradient norm for each task: The load balancing loss is: Z-Loss is: The total loss is: In the formula, For the first Number of samples per task; For the first Number of exception categories for each task; For genuine one-hot tags; To predict probabilities; For a shared set of parameters; Dynamic task weights; It is an L2 norm; For the first Expected activation frequency of an expert; For the first The actual activation frequency of each expert; and These are the hyperparameter weighting coefficients.

7. The structural health monitoring and abnormal data diagnosis method based on a multi-task hybrid expert vision Transformer according to claim 1, characterized in that, The task-specific classification head in step six satisfies: the... The classification prediction output for each bridge task is: and with Obtain the anomaly category determination result; In the formula, For the first The task category header; The task feature vector is input to the classification head; For the first The probability distribution of each task category; For the predicted category.

8. A structural health monitoring and abnormal data diagnosis system based on a multi-task hybrid expert vision Transformer, characterized in that, The diagnostic system uses the structural health monitoring anomaly data diagnostic method based on multi-task hybrid expert vision Transformer as described in any one of claims 1-7, and the diagnostic system includes the following six steps: Data preprocessing and sequence embedding module: acquire one-dimensional structural health monitoring time series data of multiple bridges, convert the time series data into two-dimensional image samples; define each bridge as an independent subtask with "bridge" as the task granularity, and construct a multi-task dataset; use the image embedding module to divide the two-dimensional image into non-overlapping local blocks and map them into token sequences, and superimpose learnable position codes to obtain the Transformer input sequence. Multi-task hybrid expert network architecture building module: Construct a shared backbone network based on visual Transformer, and introduce a hybrid expert attention module and a feedforward neural network module into the Transformer block to form an MT-MoEViT network, which is used to extract common and differential features across bridging tasks in the shared feature space; Task-aware dynamic gating routing module: In the hybrid expert attention module introduced in step two, the token representation and the bridge task number are fused into the embedding vector. The expert routing score is calculated through the gating network, and the Gumbel-Softmax strategy is used to achieve differentiable sparse sampling to obtain the expert selection probability distribution. Expert Feature Extraction and Weighted Fusion Module: Based on the probability distribution of expert selection in step three, the Top-k strategy is adopted to activate only the top k most relevant expert sub-networks to generate query vectors, while the key vector and value vector are shared among all experts; the sparse selection probability is used to perform weighted fusion of the multiple expert query outputs, and feature interaction and extraction are completed through a multi-head attention mechanism; Multi-task collaborative optimization training module: Construct a joint optimization objective function that includes classification loss, load balancing loss and auxiliary regularization loss Z-Loss, and introduce the GradNorm algorithm to dynamically adjust the loss weights of each bridge task to balance the optimization rates between different tasks. At the same time, the load balancing loss is used to constrain the expert activation frequency to prevent expert collapse. Anomaly diagnosis decision output module: Set a task-specific classification head for each bridge subtask, input the features extracted by the deep network into the task-specific classification head that matches the corresponding bridge task, output the anomaly category prediction probability of the bridge task and give the anomaly category judgment result, so as to realize multi-bridge collaborative anomaly diagnosis.

9. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-7.

Images