Industrial data analysis system and method based on MoE-Transform

Abstract

According to the industrial data analysis system and method, an input layer carries out preprocessing on multi-modal original industrial data, a MoE-Transform core layer carries out feature extraction on the preprocessed data to obtain multi-modal feature vectors, on the basis of a hierarchical entropy gating mechanism, experts are distributed to process the feature vectors of all the modals, and the multi-modal feature vectors of all the modals are obtained; wherein a K value in the Top-K strategy is determined through an expert load weight, the expert load weight reflects an expert resource consumption level in a predetermined time period, then an expert is matched to a corresponding node for processing according to a hardware state to obtain a result feature, and the intelligent output layer fuses to obtain an analysis result based on the result feature and a global expert parameter. Therefore, the experts are dynamically determined according to the expert resource consumption level, the nodes distributed by the experts are dynamically determined according to the hardware state, the experts and hardware resources can be effectively utilized in a balanced mode, the model efficiency is improved, and application of the model in industrial data analysis is promoted.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an industrial data analysis system and method based on MoE-Transformer. Background Art

[0002] In industrial quality inspection scenarios, the traditional hybrid expert MoE-Transformer model is usually used to process multimodal data such as real-time text work orders and captured images to obtain the required quality inspection results.

[0003] Traditional quality inspection and analysis mainly have the following technical defects: 1. The existing routing mechanism causes some experts to be overloaded. For example, the call rate of text experts exceeds 80%, while other experts are idle. Statistics show that in the MoE-Transformer model, the variance of expert utilization can be as high as 45%.

[0004] 2. The dense feedforward network (FFN) in the Transformer model fully calculates irrelevant modal features, and approximately 70% of the parameters in the model do not contribute to the current task.

[0005] The above technical defects make it impossible for the traditional MoE-Transformer model to effectively and evenly utilize resources such as experts when processing industrial data analysis tasks, resulting in the inability to effectively improve the model's performance and restricting the model's application in industrial data analysis. Summary of the Invention

[0006] This application proposes an industrial data analysis system and method based on MoE-Transformer, which can solve one of the problems existing in the background technology.

[0007] To achieve the above objectives, this application adopts the following technical solutions: In a first aspect, an industrial data analysis system based on MoE-Transformer is provided, the analysis system comprising: An input layer, configured to obtain multimodal raw industrial data from raw data generation nodes and preprocess the multimodal raw industrial data; The MoE-Transformer core layer is configured to extract features from the pre-processed multimodal industrial data to obtain a multimodal feature vector; assign an expert to each modal feature vector based on a hierarchical entropy gating mechanism, wherein the K value in the Top-K strategy is adjusted according to the expert load weight, where the expert load weight is determined by a normalized parameter of the expert resource consumption level within a predetermined time period; and assign the expert to the corresponding processing node based on the obtained real-time hardware status of the processing node; and The intelligent output layer is used to obtain the result features obtained by each expert processing and the global expert parameters, and weight each result feature and the global expert parameters to obtain the required analysis results.

[0008] Based on the above technical solution, the input layer preprocesses the multimodal raw industrial data, and the MoE-Transformer core layer extracts features from the preprocessed data to obtain multimodal feature vectors. Based on the hierarchical entropy gating mechanism, experts are assigned to process each modal feature vector. The K value in the Top-K strategy is determined by the expert load weight, which reflects the expert resource consumption level within a predetermined time period. Then, based on the hardware status, the experts are matched to the corresponding nodes for processing to obtain the result features. The intelligent output layer fuses the result features and global expert parameters to obtain the analysis results. In this way, experts are dynamically determined based on the expert resource consumption level and the nodes to which they are assigned based on the hardware status. This can effectively and evenly utilize expert and hardware resources, improve model performance, and promote the application of models in industrial data analysis.

[0009] In a possible design of the first aspect, the MoE-Transformer core layer is specifically configured to: Statistically obtaining a normalized weight of requests processed by each expert within a predetermined period of time as the expert load weight; and When the expert load weight exceeds a preset threshold, the K value is determined to be 1.

[0010] In a possible design of the first aspect, the MoE-Transformer core layer is specifically configured to: According to the set upper limit of expert processing requests, the requests exceeding the limit are redistributed according to the expert load weight.

[0011] In a possible design of the first aspect, the MoE-Transformer core layer is specifically configured to: Using a block sparse algorithm, perform group matrix multiplication on the FFNN layers corresponding to the activated experts; and The output of the inactivated experts is compensated by the shared experts.

[0012] Based on the above technical solution, the block sparse algorithm can speed up the calculation by at least three times compared with dense calculation, thereby improving the overall processing efficiency of the model.

[0013] In a possible design of the first aspect, the analysis system further includes: The federated parameter aggregation layer is used to aggregate the local expert parameters of multiple nodes to generate the global expert parameters.

[0014] In a possible design manner of the first aspect, the federated parameter aggregation layer is specifically configured to: Determine the aggregation weight of local expert parameters based on the expert activation frequency; Obtaining a weighted aggregation parameter by using the local expert parameter and its corresponding aggregation weight through homomorphic encryption; and Gaussian noise is added to the weighted aggregation parameters to form the global expert parameters.

[0015] Based on the above technical solution, Gaussian noise is added to the weighted aggregation parameters to meet the differential privacy constraints and achieve privacy protection.

[0016] In a possible design manner of the first aspect, the federated parameter aggregation layer is further configured to: The aggregation weight is dynamically modified based on the expert activation distribution and the real-time hardware status.

[0017] In a possible design method of the first aspect, the data generation node includes: an equipment operation terminal, a thermal imager, an acceleration sensor and a controller; the multimodal industrial data includes: text logs, infrared images, vibration signals and controller parameters; the preprocessing includes: data spatiotemporal synchronization and data cleaning; the hardware status includes: GPU memory occupancy and cross-node communication delay, and the node is used for distributed deployment of experts; the analysis results include: a comprehensive health score obtained by weighted fusion of each of the result features and the global expert parameters, a health assessment result determined based on the comprehensive health score and a preset threshold, and an explainable decision made based on the health assessment result.

[0018] In a possible design of the first aspect, feature extraction is performed on the preprocessed multimodal industrial data, specifically including: Using the BERT architecture, we extract primary semantic features from text logs. Using word embedding and positional encoding, we generate secondary semantic features based on these primary semantic features. We then process these secondary semantic features using a dimensionality reduction matrix to obtain a text semantic vector. Using the trained ResNet-50, we extract primary image features from infrared images, focus on defect areas through the non-local mean attention module, and map the primary image features into visual feature vectors using a cross-modal projection matrix. Extracting primary spectral features from the vibration signal using FFT, weighting designated frequency bands in the primary spectral features to obtain secondary spectral features, capturing temporal dependencies of the secondary spectral features using a bidirectional LSTM, and performing dimensionality compression on the secondary spectral features to obtain a frequency domain feature vector; and The controller parameters are crossed with the structured parameters to generate interactive features, which are then mapped into working condition embedding vectors through the fully connected layer. The text semantic vector, the visual feature vector, the frequency domain feature vector and the working condition embedding vector have a unified feature dimension.

[0019] In a second aspect, an industrial data analysis method based on MoE-Transformer is provided. The analysis method is based on the above-mentioned analysis system and includes: Obtaining multimodal raw industrial data from a raw data generating node, and preprocessing the multimodal raw industrial data; performing feature extraction on the preprocessed multimodal industrial data to obtain a multimodal feature vector; assigning an expert to each modal feature vector based on a hierarchical entropy gating mechanism, wherein a K value in a Top-K strategy is adjusted according to an expert load weight, wherein the expert load weight is determined by a normalized parameter of an expert resource consumption level within a predetermined time period; and assigning the expert to a corresponding processing node based on the obtained real-time hardware status of the processing node; and Obtain the result features and global expert parameters obtained by each expert processing, and weight each result feature and the global expert parameter to obtain the required analysis results.

[0020] In a third aspect, an electronic device is provided, comprising: a processor, and a memory coupled to the processor, the memory being used to store a computer program; and the processor being used to execute the computer program stored in the memory, so that the electronic device performs the analysis method described in the second aspect.

[0021] In a fourth aspect, a computer-readable storage medium is provided, comprising a computer program or instructions, which, when executed on a computer, causes the computer to execute the analysis method described in the second aspect.

[0022] In a fifth aspect, a computer program product is provided, comprising: a computer program or instructions, which, when executed on a computer, causes the computer to execute the analysis method described in the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0024] Figure 1This is a schematic diagram of the structure of the industrial data analysis system provided in Example 1 of the present application; Figure 2 This is a schematic diagram of the industrial multimodal semantic fusion architecture based on MoE-Transformer provided in Example 2 of this application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0026] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0028] Example 1 This embodiment provides an industrial data analysis system 100, which includes: The input layer 101 is used to obtain multimodal original industrial data from the original data generation node and preprocess the multimodal original industrial data; The MoE-Transformer core layer 102 is configured to perform feature extraction on the pre-processed multimodal industrial data to obtain a multimodal feature vector; assign an expert to each modal feature vector based on a hierarchical entropy gating mechanism, wherein the K value in the Top-K strategy is adjusted according to the expert load weight, wherein the expert load weight is determined by a normalized parameter of the expert resource consumption level within a predetermined time period; and assign the expert to the corresponding processing node based on the obtained real-time hardware status of the processing node; and The intelligent output layer 103 is used to obtain the result features obtained by each expert processing and the global expert parameters, and weight each result feature and the global expert parameters to obtain the required analysis results.

[0029] Specifically, in an industrial production environment, data generation nodes can include equipment operating terminals, thermal imagers or cameras, accelerometers configured on equipment, and equipment controllers. Controllers can be programmable logic controllers (PLCs) or field programmable gate arrays (FPGAs). Accordingly, equipment operating terminals can generate text logs, thermal imagers can generate infrared images, cameras can generate RGB images, accelerometers can generate equipment vibration signals, and controllers can generate controller parameters such as speed and load factor. Text logs, images, vibration signals, and controller parameters constitute multimodal industrial data.

[0030] The preprocessing of multimodal industrial data may include aspects such as data spatiotemporal synchronization and data cleaning. The main purpose is to enable multimodal industrial data to be aligned in time and space, and to perform corresponding noise reduction and completion, etc., to facilitate subsequent model processing.

[0031] Regarding data spatiotemporal synchronization, hardware-level time synchronization utilizes the IEEE 1588 PTPv2 protocol to establish a master-slave clock system, employing a round-trip delay compensation algorithm to adjust clock skew. For asynchronous data, time alignment is achieved by calculating the optimal alignment path, using cubic spline interpolation to compensate for missing segments.

[0032] For data cleaning, a built-in rule library can be used to convert non-standard terminology. A gated recurrent unit can be used to detect and correct unit conflicts. A pixel search window can be used to calculate similarity weights, combined with the Sobel operator to protect defect edges. An improved Kalman filter algorithm can be used to optimize the process noise covariance matrix, suppress power frequency interference, and preserve the fault frequency band.

[0033] The MoE-Transformer is a Transformer-based mixture of experts model. It typically replaces the Multilayer Perceptron (MLP) layer in a Transformer block. A MoE layer primarily consists of experts and routers. Each MoE layer contains many experts, typically 16 or 64. In actual reasoning, only a few are used. Each expert learns certain lexical features of sentences during training, such as whether one expert is better at processing punctuation marks, while another is better at processing verbs. The router, also known as the gating network, determines which expert to assign the model's basic unit, a token. Both the expert and the router are essentially feed-forward neural networks (FFNNs). The router's output undergoes a softmax operation, resulting in a one-dimensional vector representing each expert's confidence in the token. A higher value indicates a more competent expert. The expert's processing result is multiplied by the confidence score to produce the output of that expert's layer.

[0034] The MoE-Transformer model usually sets a Top-K strategy to determine the number of experts to select, where K refers to the number of experts activated at each token.

[0035] After preprocessing, feature extraction is performed on multimodal industrial data, and the extracted features of each mode need to be standardized to eliminate heterogeneity between modes.

[0036] Specifically, the BERT architecture is used to extract primary semantic features from text logs. Secondary semantic features are generated based on the primary semantic features through word embedding and position encoding. The secondary semantic features are then processed using a dimensionality reduction matrix to obtain text semantic vectors.

[0037] The trained ResNet-50 is used to extract primary image features from infrared images, and the defect area is focused on by the non-local mean attention module. The primary image features are then mapped into visual feature vectors through the cross-modal projection matrix.

[0038] The primary spectral features of the vibration signal are extracted using FFT, and the specified frequency bands in the primary spectral features are weighted to obtain the secondary spectral features. The temporal dependency of the secondary spectral features is captured through a bidirectional LSTM, and the secondary spectral features are dimensional compressed to obtain the frequency domain feature vector.

[0039] The controller parameters undergo feature crossover of the structured parameters to generate interaction features, which are then mapped into working condition embedding vectors through a fully connected layer.

[0040] The text semantic vector, visual feature vector, frequency domain feature vector and working condition embedding vector are standardized so that all the final vectors have a unified feature dimension.

[0041] The Hierarchical Entropy-Gated mechanism combined with the Mixture of Experts (MoE) means that the expert selection process is optimized through a multi-layer gating mechanism, which mainly involves: a global balancing stage, a cluster-level pruning stage, and a final expert allocation stage.

[0042] In the global balancing phase, tokens are routed to an initial pool of N experts using softmax affinity scores.

[0043] In the cluster-level pruning stage, in the selected initial pool, a secondary gating mechanism prunes the experts according to the entropy constraint.

[0044] In the final expert assignment stage, the top K experts are selected using an adjusted probability function that incorporates an entropy-sensitive penalty term.

[0045] Expert load weights are used to measure the level of expert resource consumption within a predetermined time period. In practice, the normalized weights of tokens processed by each expert within the predetermined time period can be calculated and used as the expert load weight. For example, within a predetermined time period T, expert 1 processes X1 tokens, and expert 2 processes X2 tokens. Normalizing X1 and X2 yields corresponding normalized weights Y1 and Y2. Therefore, expert 1's expert load weight is Y1, and expert 2's expert load weight is Y2.

[0046] Adjust the K value in the Top-K strategy based on the expert load weight. Specifically, the K value can be determined when the expert load weight exceeds a preset threshold. For example, when the expert load weight Y1 exceeds 80%, the K value is set to 1.

[0047] When assigning experts, you can also redistribute tokens that exceed the limit according to the expert's load weight based on the set expert processing request limit. For example, if the expert processing request limit is 200 tokens / second, tokens that exceed this limit will be redistributed to other experts according to the expert's load weight.

[0048] Experts are assigned to appropriate processing nodes based on real-time hardware status such as GPU memory usage and cross-node communication latency, for example, the processing node with the lowest GPU memory usage or the lowest cross-node communication latency.

[0049] In the process of expert calculation to obtain inference results, a block sparse algorithm is used to perform group matrix multiplication on the FFNN layer corresponding to the activated expert, while the output of the inactivated expert is compensated by the shared expert.

[0050] Block-sparse algorithms primarily divide data into blocks. Each block has a certain degree of sparsity: only a few elements are non-zero, while the majority are zero or close to zero. Block-sparse algorithms optimize the objective function to minimize the number of non-zero elements while maintaining the overall structure of the data.

[0051] The Grouped GEMM algorithm mainly divides the calculation task of a large matrix into smaller subtasks, and then speeds up the entire process through parallel computing.

[0052] In other optional implementations, the above analysis system may further be configured with a federal parameter aggregation layer to aggregate local expert parameters of multiple nodes to form global expert parameters.

[0053] Specifically, the aggregation algorithm involves: first, determining the aggregation weights of local expert parameters based on the expert activation frequency; second, using homomorphic encryption, using the local expert parameters and their corresponding aggregation weights to obtain a weighted aggregation parameter; and finally, adding Gaussian noise to the weighted aggregation parameter to form a global expert parameter. Homomorphic encryption allows computation to be performed directly on encrypted data without decrypting it. After the computation is completed, the result remains encrypted, and only users with the decryption key can see the true value of the result.

[0054] The federated parameter aggregation layer can also dynamically modify the aggregation weights based on expert activation distribution and real-time hardware status.

[0055] This embodiment further provides an industrial data analysis method, which is based on the above-mentioned analysis system and includes: Obtaining multimodal raw industrial data from a raw data generating node, and preprocessing the multimodal raw industrial data; performing feature extraction on the preprocessed multimodal industrial data to obtain a multimodal feature vector; assigning an expert to each modal feature vector based on a hierarchical entropy gating mechanism, wherein a K value in a Top-K strategy is adjusted according to an expert load weight, wherein the expert load weight is determined by a normalized parameter of an expert resource consumption level within a predetermined time period; and assigning the expert to a corresponding processing node based on the obtained real-time hardware status of the processing node; and Obtain the result features and global expert parameters obtained by each expert processing, and weight each result feature and the global expert parameter to obtain the required analysis results.

[0056] The content of the analysis method is similar to that of the above-mentioned analysis system and will not be repeated here.

[0057] Example 2 We propose an industrial multimodal semantic fusion architecture based on MoE-Transformer. This architecture achieves closed-loop intelligence from perception to decision-making of multi-source data under the constraints of industrial-grade latency (<200ms) and accuracy (recall rate >98%) through a multimodal dynamic routing mechanism (to solve modal heterogeneity), residual compensation of mechanism constraints (to ensure physical interpretability), and a hierarchical federated learning framework (to break through resource limitations). Figure 2 shown.

[0058] A brief description of the architectural layers: Based on the architecture described above, we implement the logic of this embodiment using the following steps: Step 1: Multimodal data synchronization and cleaning This step corresponds to the input layer in the structure. Its purpose is to eliminate the time deviation, spatial misalignment and noise interference of multimodal industrial data, generate a clean data stream with strict time and space alignment and a satisfactory signal-to-noise ratio, and provide a basis for subsequent cross-modal feature fusion.

[0059] The main input data for this step are as follows: The data is processed as follows: 1. Multimodal data synchronization Regarding time synchronization (ensuring that distributed nodes such as robots, sensors, and controllers execute operations according to a unified timing), hardware-level synchronization is first implemented. A master-slave clock system is established using the IEEE 1588 PTPv2 protocol. Round-trip delay measurement is employed to reduce sensor and thermal imager clock deviations from ±50ms to ±1μs. Dynamic time adjustment is then implemented.

[0060] For the synchronization of asynchronous data such as text logs, the optimal alignment path is calculated with a window width of 10ms and a path cost function: D(i,j)=min(D(i−1,j),D(i,j−1),D(i−1,j−1))+‖x i −x j ‖, where D(i, j) represents the minimum cost of the alignment path from the starting point to the position (i, j), and x i and x j The data points are represented by Euclidean distance, and the similarity between them is measured. The missing segments are compensated by cubic spline interpolation (error < 0.5%).

[0061] 2. Cleaning of multimodal data 2.1 About text cleaning A built-in rule library of over 300 rules (e.g., / [A-Za-z_]+ / → ISO standard terms) allows for the conversion of non-standard terms (e.g., "Bearing_vibr_over" → "Bearing vibration exceeds standard"). A gated recurrent unit (32 hidden nodes) detects unit conflicts (e.g., "mm" mistakenly written as "cm") with an error rate of <0.1%.

[0062] 2.2 About Image Noise Reduction 21×21 pixel search window, similarity weight calculation: , where i, j represents the target pixel position, p i represents the local image block centered at pixel i, p j represents the local image block centered at pixel j, h is the attenuation control parameter, and the Sobel operator is combined to protect the defect edge (smoothing is prohibited when the gradient change rate is > 0.8).

[0063] 2.3 About signal filtering Improved Kalman filtering, optimized process noise covariance matrix Q, suppressed power frequency interference <6kHz (frequency domain attenuation slope >40dB / dec), and retained the 6-10kHz fault frequency band (energy retention rate >95%).

[0064] After the above processing, we get the following output data: Standardized text: ISO terminology, work order logs with aligned timestamps (±5ms deviation) (e.g., "Bearing vibration exceeds standard: 0.02mm@2023-09-15 15:30:05.123").

[0065] Denoised image: 1024×768 pixel matrix, physical coordinate mapping completed (e.g., pixel (320, 480) → device coordinate system (x=1.2m, y=0.8m, z=0.3m)).

[0066] Filtered signal: 1kHz vibration data, energy in the 6-10kHz frequency band accounts for >85%.

[0067] Complete PLC parameters: continuous data flow without interruptions (packet loss rate <0.01%).

[0068] Step 2: Cross-modal feature encoding The purpose of this step is to map multimodal data (text, images, signals) into a unified semantic space, eliminate heterogeneity between modalities, and provide comparable feature vectors for subsequent MoE gating networks.

[0069] The input for this step is the data obtained in the previous step: normalized text, noise-reduced image, filtered signal, and complete PLC parameters: These data are processed as follows: 1. Standardize the text and use the BERT architecture to extract semantic features (12-layer Transformer encoder). Generate a 768-dimensional vector through word embedding (Word2Vec) and position encoding. text ∈R 768×128 Project it into the 128-dimensional semantic space to obtain the text semantic vector.

[0070] Denoised infrared image, image features extracted by pre-trained ResNet-50 (2048 dimensions), and then processed by non-local mean attention module (weight calculation: , where the query vector q i is the linear projection feature of the target position i, the key vector k j is the linear projection feature of the reference position j, T is the transpose) focus on the defect area, and then through the cross-modal projection matrix W image ∈R 2048 ×128 Mapping to obtain the visual feature vector.

[0071] The vibration signal is filtered and the spectrum features (512 dimensions) are extracted through 1024-point FFT. The 6-10kHz frequency band is weighted (energy share > 85%). The timing dependency is captured through bidirectional LSTM. Finally, the dimensionality reduction matrix W is used. signal ∈R 512×128 Compress the dimension and obtain the frequency domain feature vector.

[0072] The complete PLC parameters are transformed into interactive features through feature crossover (such as speed × load rate) through structured parameters (speed, load rate), and mapped to 128 dimensions through a fully connected layer (activation function ReLU) to obtain the working condition embedding vector.

[0073] Cross-modal alignment, calling the Generative Adversarial Network (GAN), constrains the distribution consistency between modalities in the latent space (loss function: taking text and image as an example, L adv =‖E text (x)−E image (y)‖2), where E text (x) is the feature vector output by the text modality encoder, E image (y) is the feature vector output by the image modality encoder, and x,y are sample pairs of text and image. Semantic similarity is measured using Euclidean distance, and the cosine similarity of text, image, and signal features in the 128-dimensional space is required to be greater than 0.85.

[0074] After the above processing, four 128-dimensional output data are obtained: Text semantic vector: represents the semantic information of the fault description.

[0075] Visual feature vector: Encodes the location and temperature distribution of bearing hot spots.

[0076] Frequency domain eigenvector: reflects the energy spectrum of the 6-10kHz fault characteristic frequency band.

[0077] Working condition embedding vector: integrating the real-time working condition parameters of the equipment.

[0078] Step 3: Dynamic Routing and Expert Computing The purpose of this step is to dynamically allocate expert resources and perform sparse computing based on multimodal features and real-time hardware status, achieving load balancing and computing power optimization while ensuring inference accuracy.

[0079] The input data for this step are: The logic of data processing is as follows: 1. Feature vector, reinforcement learning routing, based on the hierarchical entropy gating mechanism (HE-MoE), the feature vector h at each time t t Calculate the fitness score of expert i: s i,t = Softmax(W g h t +ε), ε∼N(0, 0.1), where W g is the routing weight matrix, ε is Gaussian noise with a standard deviation of 0.1, i is the expert index value, t is the current time, N is the normal distribution, softmax is the normalized exponential function, and the noise ε is injected to prevent routing from solidifying. During Top-K selection, the K value is adjusted according to the expert load weight. For example, when the expert load weight is greater than 80%, K = 1, that is, when overloaded, it degenerates to the single-expert mode, and the routing decision matrix is ​​obtained.

[0080] 2. Routing decisions are based on this logic to activate experts: experts are assigned to the device node with the lowest load based on GPU memory usage and communication latency. An upper limit is set for expert processing (e.g., a single expert can process a maximum of 200 tokens / second), and requests exceeding this limit are redistributed based on load weight.

[0081] 3. Expert calculation to obtain inference results: Using block sparse calculation optimization, Grouped GEMM is performed on the activated expert FFN layer, which is 3.2 times faster than dense calculation; the output of inactivated experts is compensated by shared experts , where h outis the output feature vector generated by the weighted combination of activation experts and shared experts, S is the activation expert set, g k,t is the gating weight, SharedExpert is the universal expert for all modalities, and Expert() is the expert output. The more inactive experts there are, the greater the contribution of the shared experts. λ is the weight that is dynamically adjusted based on the number of inactive experts.

[0082] Shared expert compensation is used primarily to maintain baseline performance when expert activation rates are low. When the system activates only a small number of experts (e.g., K = 1) due to load balancing or hardware limitations, the features of the inactive experts are forcibly disabled, resulting in reduced feature expressiveness for some modalities or scenarios. In this case, the shared experts perform compensation calculations on all input features to cover the functional blind spots of the inactive experts.

[0083] After the above processing, we get the following output data: Expert activation distribution: an expert-device mapping table in JSON format (e.g., {"Expert3": "GPU1", "Expert5": "GPU2"}), containing the number of tokens processed by each expert and their weight.

[0084] Inference results: 128-dimensional enhanced feature vector (example: [0.85, -0.33, ..., 1.02]), with computation time and hardware utilization labels.

[0085] Demotion strategy parameters: Demotion instructions generated when GPU load > 90% (e.g. {"mode": "K=1", "experts": [3,7]}).

[0086] Step 4: Federated Parameter Aggregation The purpose of this step is to aggregate the local expert parameters of multiple edge nodes, generate a global expert model and support dynamic expansion, so as to realize distributed knowledge fusion and model evolution under privacy protection.

[0087] The input data for this step is as follows: Data processing logic: 1. The "local expert parameters" are securely aggregated to obtain the "weighted aggregate parameters": the improved FedAvg algorithm is used to perform weighted averaging on the encryption parameters of each edge node. , where α k is the aggregation weight, is the global model parameter after weighted aggregation, which is composed of the local expert parameters of all participating nodes Weighted. , n k is the amount of data at node k, f kActivate frequency for experts and achieve parameter aggregation through homomorphic encryption.

[0088] 2. Perform resource-aware adjustments to the expert activation distribution to obtain dynamic adaptation weights: The aggregate weights are dynamically modified based on the GPU utilization and communication latency in the metadata. Dynamically modify the aggregation weights based on the GPU utilization and communication latency in the metadata; If the inter-node delay is >50μs, the same device expert parameters are preferred; Specifically, dynamic correction of aggregation weights may involve the following process: Nodes with high GPU utilization (e.g. >80%) may suffer from insufficient computing resources, resulting in a decrease in the quality of local training parameters. It is necessary to reduce their aggregation weights to prevent low-quality parameters from contaminating the global model. When the cross-node delay is >50μs, synchronous aggregation will significantly slow down the federated learning cycle. Parameters of nodes on the same device or with low latency need to be prioritized to reduce communication overhead. By introducing the load factor λ load , so that the higher the GPU utilization, the lower the weight of the node participating in the aggregation. You can also introduce a delay factor λ latency , so that the higher the delay, the more significant the node weight decay.

[0089] The original aggregation weight is α k , and dynamically correct it to get: 3. Obtaining "incremental expert parameters" from "dynamic expansion requests": When a new sensor triggers an expansion, the new adapter layer is fine-tuned based on the feature encoder in step 2. Meta-learning (MAML) is used to initialize the adapter parameters to minimize the KL divergence between the new modality data and the existing semantic space. Secure multi-party computation (SMPC) is used to verify parameter compliance to prevent malicious node injection.

[0090] 4. "Global parameter distribution" → "federated verification" is mainly about differential privacy protection: Gaussian mechanism is a common method to achieve (ϵ,δ)-differential privacy. Add multidimensional Gaussian noise to (I is the unit matrix, ensuring that noise is injected independently into each parameter dimension.) The global parameters after adding noise are obtained , where the noise variance σ 2 Dynamically adjust according to privacy budget.

[0091] Specifically, the noise variance σ2 is determined by the privacy budget And the gradient clipping threshold L determines: .satisfy , defense member inference attack. Budget for privacy and control the strength of privacy protection. is the relaxation probability, representing the probability of allowing privacy protection to fail. L is the gradient clipping threshold, which constrains the upper bound of the L2 norm of the locally updated parameters. As long as the added Gaussian noise meets certain variance requirements (based on the gradient clipping threshold L, the privacy budget ε, and the probability δ), the algorithm can be mathematically proven to satisfy (ε, δ)-differential privacy.

[0092] After the above processing, we get the following output data: Updated global expert parameters: decrypted 128-dimensional adapter weight matrix (example: [0.72, -0.15,..., 1.03]) with privacy noise labels.

[0093] Incremental Expert Configuration: Added adapter initialization parameters corresponding to new sensors (example: {"infrared_Expert8":[0.12, 0.88, ...]}).

[0094] Aggregate verification report: JSON log containing metrics such as resource utilization and privacy budget consumption.

[0095] Step 5: Decision Making and Feedback The purpose of this step is to integrate expert reasoning results with global parameter update information, generate equipment health diagnosis decisions and control instructions, and optimize the system's adaptive capabilities through a feedback mechanism.

[0096] The input for this step is as follows: Data processing logic: 1. Multimodal fusion ("Inference result" + "Global parameter" → "Health assessment matrix"): The 128-dimensional feature vector is weighted and fused with the global expert parameter (the weight is determined by the expert activation frequency) to calculate the comprehensive health score of the device. , where λ is the residual compensation coefficient (default 0.3), Activate weights for experts, To contribute expert output, It is the shared expert output used to balance the contribution of inactivated experts.

[0097] 2. "Health Assessment" → "Status Classification and Decision": Set multiple thresholds (healthy / warning / fault) based on historical baselines (for example: healthy: S ≥ 90; 70 ≤ S < 90 → trigger preventive maintenance recommendations; fault: S < 70 → force shutdown instructions).

[0098] 3. "Decision Confidence Verification" → "Control Instruction Optimization": Confidence weighting: If the confidence of the inference result is <0.8, a conservative strategy is adopted (such as extending the detection cycle); if the confidence is >0.95, a high-priority instruction is directly issued.

[0099] Instruction encryption: The integrity of control instructions is ensured through digital signatures (RSA-2048).

[0100] 4. Closed-loop quality monitoring ("Feedback Link Construction" → "System Adaptation"): Abnormal data (such as false positives) is marked and fed back to the data cleaning module in step 1 to optimize data preprocessing rules.

[0101] After the above processing, we get the following output data: Equipment health report: diagnostic results in JSON format (example: {"Health Status":"Warning", "Score":75, "Major Abnormality":"Bearing vibration exceeds standard (0.02mm)"}).

[0102] Control instruction set: includes maintenance actions, parameter adjustments, shutdown instructions, etc. (Example: {"Instruction type":"Lubrication maintenance", "Execution time":"2025-04-29 14:30", "Priority":2}).

[0103] Routing optimization parameters: expert weight adjustment table generated based on decision confidence (for example: {"Expert3":0.85→0.78, "Expert7":0.12→0.25}).

[0104] The industrial large-scale model monitoring system, constructed through five core steps, has formed an innovative architecture with industry barriers: 1. Verification of Technology Uniqueness 1. Collaborative Architecture of Multimodal Dynamic Routing and Federated Learning Uniqueness: Deeply coupling a Mixture of Experts (MoE) dynamic routing mechanism with privacy-preserving federated aggregation resolves the conflict between model generalization and data privacy in traditional industrial monitoring. For example, when new sensors (such as infrared thermal imagers) are added, the federated module achieves incremental parameter expansion through meta-learning, while traditional methods require retraining the entire model.

[0105] Technology comparison: Compared with protocol anomaly detection (single-dimensional feature analysis) or graph neural network spillage monitoring (specific scenario modeling), this application's cross-modal feature alignment accuracy (cosine similarity > 0.85) and real-time inference efficiency (latency < 200ms) have reached industry-leading levels.

[0106] 2. Embedded integration of industrial mechanisms and large models Uniqueness: Encodes equipment physical constraints (such as vibration severity thresholds) as prior knowledge of large models, enabling data-driven and mechanism-driven dual-engine decision-making.

[0107] Technology comparison: Although the process monitoring system supports data analysis, its rule engine requires manual configuration. This application automatically balances the expert output deviation through a residual compensation mechanism, reducing the need for manual intervention by 70%.

[0108] 2. Strengthening Business Value 1. Accident prevention economy By identifying anomalies early (such as signs of bearing wear), unplanned downtime is reduced by 60%, lowering annual maintenance costs for a single production line by 2 million yuan. Compared to an alarm response mechanism (which only handles the issue afterward), this application's progressive risk quantification system (a three-level response system of health, warning, and fault) can shorten the incident response window by 3-8 hours.

[0109] 2. Reusable monitoring base for the entire industry Support for cross-enterprise knowledge sharing (differential privacy aggregation) enables small and medium-sized manufacturers to share fault models from leading companies, reducing the industry average accident rate by 35%. However, vertical scenario solutions cannot achieve cross-domain migration.

[0110] 3. Compliance and standardization barriers Generates digitally signed, traceable alarm logs that meet standards like ISO 45001, whereas conventional protocol anomaly detection lacks audit tracking capabilities. Dynamic expansion modules can adapt to new compliance indicators (such as the EU's new industrial dust emission standards) within an hour, whereas traditional systems would take months to adapt.

[0111] Summarize This application utilizes a multimodal industrial large-scale model architecture and mechanism-data dual-driven decision-making to construct a monitoring platform that combines high precision, strong generalization, and easy scalability. Its uniqueness lies in: (1) the collaborative innovation of dynamic routing and federated learning; (2) the embedded integration of physical rules and large-scale models; and (3) the adaptive evolution capability driven by closed-loop feedback. Compared with similar patents, this technical solution covers a more complete value chain (from data collection to decision execution) and forms a significant barrier in terms of accident prevention economics, cross-industry reusability, and compliance agility.

[0112] An embodiment of the present application also provides an electronic device, comprising: a processor, and a memory coupled to the processor, wherein the memory is used to store a computer program; and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method described in any one of the above embodiments.

[0113] The electronic device may be a computing device such as a desktop computer, a notebook computer, a PDA, a cloud server, etc. The electronic device may include, but is not limited to, a processor and a memory.

[0114] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting various parts of the entire device using various interfaces and lines.

[0115] The memory may be used to store the computer program, and the processor implements various functions of the electronic device by running or executing the computer program stored in the memory and calling the data stored in the memory.

[0116] The memory may primarily include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function, and the like; the data storage area may store data generated based on the use of the mobile phone. Furthermore, the memory may include high-speed random access memory (RAM) and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0117] The embodiment of the present application also provides a storage medium, which is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. The computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.

[0118] An embodiment of the present application further provides a computer program product, including: a computer program or instructions, which, when executed on a computer, causes the computer to execute any of the above-mentioned possible implementation methods.

[0119] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications are also considered to be within the scope of protection of the present application.

Claims

1. An industrial data analysis system based on MoE-Transformer, characterized in that: The analysis system comprises: An input layer, configured to obtain multimodal raw industrial data from raw data generation nodes and preprocess the multimodal raw industrial data; The MoE-Transformer core layer is configured to extract features from the pre-processed multimodal industrial data to obtain a multimodal feature vector; assign an expert to each modal feature vector based on a hierarchical entropy gating mechanism, wherein the K value in the Top-K strategy is adjusted according to the expert load weight, where the expert load weight is determined by a normalized parameter of the expert resource consumption level within a predetermined time period; and assign the expert to the corresponding processing node based on the obtained real-time hardware status of the processing node; and The intelligent output layer is used to obtain the result features obtained by each expert processing and the global expert parameters, and weight each result feature and the global expert parameters to obtain the required analysis results.

2. The analysis system according to claim 1, wherein The MoE-Transformer core layer is specifically used to: Statistically obtaining a normalized weight of requests processed by each expert within a predetermined period of time as the expert load weight; and When the expert load weight exceeds a preset threshold, the K value is determined to be 1.

3. The analysis system according to claim 1, wherein: The MoE-Transformer core layer is specifically used to: According to the set upper limit of expert processing requests, the requests exceeding the limit are redistributed according to the expert load weight.

4. The analysis system according to claim 1, wherein: The MoE-Transformer core layer is specifically used to: Using a block sparse algorithm, perform group matrix multiplication on the FFNN layers corresponding to the activated experts; and The output of the inactivated experts is compensated by the shared experts.

5. The analysis system according to claim 1, wherein: The analysis system further comprises: The federated parameter aggregation layer is used to aggregate the local expert parameters of multiple nodes to generate the global expert parameters.

6. The analysis system according to claim 5, wherein: The federated parameter aggregation layer is specifically used for: Determine the aggregation weight of local expert parameters based on the expert activation frequency; Obtaining a weighted aggregation parameter by using the local expert parameter and its corresponding aggregation weight through homomorphic encryption; as well as Gaussian noise is added to the weighted aggregation parameters to form the global expert parameters.

7. The analysis system according to claim 6, wherein: The federated parameter aggregation layer is further configured to: The aggregation weight is dynamically modified based on the expert activation distribution and the real-time hardware status.

8. The analysis system according to claim 1, wherein: The data generation nodes include: equipment operation terminals, thermal imagers, acceleration sensors and controllers; multimodal industrial data include: text logs, infrared images, vibration signals and controller parameters; the preprocessing includes: data spatiotemporal synchronization and data cleaning; the hardware status includes: GPU memory occupancy and cross-node communication delay, and the nodes are used for distributed deployment of experts; the analysis results include: a comprehensive health score obtained by weighted fusion of each of the result features and the global expert parameters, a health assessment result determined based on the comprehensive health score and a preset threshold, and an explainable decision made based on the health assessment result.

9. The analysis system according to claim 3, wherein: Performing feature extraction on the preprocessed multimodal industrial data, specifically including: Using the BERT architecture, we extract primary semantic features from text logs. Using word embedding and positional encoding, we generate secondary semantic features based on these primary semantic features. We then process these secondary semantic features using a dimensionality reduction matrix to obtain a text semantic vector. Using the trained ResNet-50, we extract primary image features from infrared images, focus on defect areas through the non-local mean attention module, and map the primary image features into visual feature vectors using a cross-modal projection matrix. Extracting primary spectral features from the vibration signal using FFT, weighting designated frequency bands in the primary spectral features to obtain secondary spectral features, capturing temporal dependencies of the secondary spectral features using a bidirectional LSTM, and performing dimensionality compression on the secondary spectral features to obtain a frequency domain feature vector; and The controller parameters are crossed with the structured parameters to generate interactive features, which are then mapped into working condition embedding vectors through the fully connected layer. The text semantic vector, the visual feature vector, the frequency domain feature vector and the working condition embedding vector have a unified feature dimension.

10. An industrial data analysis method based on MoE-Transformer, characterized in that: The analysis method is based on the analysis system according to any one of claims 1 to 9, and the analysis method comprises: Obtaining multimodal raw industrial data from a raw data generating node, and preprocessing the multimodal raw industrial data; performing feature extraction on the preprocessed multimodal industrial data to obtain a multimodal feature vector; assigning an expert to each modal feature vector based on a hierarchical entropy gating mechanism, wherein a K value in a Top-K strategy is adjusted according to an expert load weight, wherein the expert load weight is determined by a normalized parameter of an expert resource consumption level within a predetermined time period; and assigning the expert to a corresponding processing node based on the obtained real-time hardware status of the processing node; and Obtain the result features and global expert parameters obtained by each expert processing, and weight each result feature and the global expert parameter to obtain the required analysis results.

Images