Industrial multi-modal model adaptive routing method based on task complexity perception

Abstract

This invention provides an adaptive routing method for industrial multimodal models based on task complexity awareness. The method includes acquiring industrial multimodal data streams from the industrial site and preprocessing them into a standardized multimodal dataset; decomposing spatiotemporal features through a physical feature extraction network to extract feature vectors and physical law features, while simultaneously identifying the set of physical constraints for the current operating condition; matching feature vectors with a model capability profile matrix to obtain a matching score; verifying candidate models through physical constraints to obtain a safety score; fusing the matching score, safety score, and resource consumption indicators for multi-objective optimization to determine the target routing model; calling the model to execute inference and outputting the results. This method achieves adaptive decision-making for industrial multimodal model routing, combines physical constraints to complete safety verification, comprehensively optimizes model selection using multi-dimensional indicators, effectively improves inference stability and safety, reduces model selection error rate, adapts to dynamic industrial conditions, and meets industrial real-time requirements.

Description

Technical Field

[0001] This invention relates to the field of industrial artificial intelligence technology, and in particular to an adaptive routing method for industrial multimodal models based on task complexity awareness. Background Technology

[0002] In practical applications of industrial multimodal large models, existing model routing technologies have significant technical shortcomings when processing heterogeneous multimodal data streams in industrial settings, and cannot meet the stringent requirements of industrial scenarios.

[0003] Existing technologies are unable to effectively extract physical features and decompose spatiotemporal features from industrial multimodal data, nor can they accurately identify the physical constraints corresponding to the working conditions. They rely solely on a single indicator for model matching and lack a safety verification process for candidate models. Furthermore, when determining the routing model, they fail to comprehensively consider multiple dimensions such as the model's matching degree with the working conditions, model safety performance, and resource consumption, making only a simple single-objective selection. This results in a mismatch between the selected routing model and the dynamic working conditions in the industrial field, leading to low reliability of inference results and poor resource utilization efficiency. In addition, existing technologies lack supporting feedback and strategy optimization mechanisms for the inference process, resulting in insufficient system adaptability. Ultimately, this leads to large fluctuations in inference accuracy and high safety risks, failing to meet the core requirements of industrial production for real-time performance, stability, and safety.

[0004] Therefore, how to achieve adaptive, intelligent, and secure routing decisions in industrial multimodal models, effectively reduce fluctuations in inference accuracy, and improve the rationality of model selection are technical problems that urgently need to be solved by those skilled in the art. Summary of the Invention

[0005] This invention provides an adaptive routing method for industrial multimodal models based on task complexity awareness, which enables adaptive, intelligent, and secure routing decisions for industrial multimodal models, effectively reducing fluctuations in inference accuracy and improving the rationality of model selection.

[0006] On one hand, this invention provides an adaptive routing method for industrial multimodal models based on task complexity awareness, which includes: Acquire industrial multimodal data streams from industrial sites, clean and normalize the industrial multimodal data streams to obtain standardized multimodal datasets; The standardized multimodal dataset is decomposed into spatiotemporal features using a physical feature extraction network to extract feature vectors and physical law features, and to identify the set of physical constraints corresponding to the current working condition. The matching calculation is performed based on the feature vector and the preset model capability profile matrix to obtain the matching degree score between each candidate model and the current working condition; The physical constraint set is used to verify the safety of each candidate model. The safety score of each candidate model is obtained by calculating the deviation between the predicted output of each candidate model and the constraint function in the physical constraint set. By combining the matching score, the security score, and the resource consumption index of each candidate model, a multi-objective optimization calculation is performed to determine the target routing model; The target routing model is invoked to perform the inference task and the inference results are output.

[0007] On the other hand, the present invention also provides an industrial multimodal model adaptive routing system based on task complexity awareness, which includes: The acquisition module is used to acquire industrial multimodal data streams from industrial sites, clean and normalize the industrial multimodal data streams to obtain standardized multimodal datasets. The extraction module is used to perform spatiotemporal feature decomposition on the standardized multimodal dataset using a physical feature extraction network, extract feature vectors and physical law features, and identify the set of physical constraints corresponding to the current working condition. The matching module is used to perform matching calculations based on the feature vector and the preset model capability profile matrix to obtain the matching degree score between each candidate model and the current working condition. The verification module is used to perform security verification on each candidate model using the physical constraint set. By calculating the deviation between the predicted output of each candidate model and the constraint function in the physical constraint set, the security score of each candidate model is obtained. The optimization module is used to perform multi-objective optimization calculations by combining the matching score, the security score, and the resource consumption indicators of each candidate model to determine the target routing model; The inference module is used to call the target routing model to perform inference tasks and output inference results.

[0008] This invention provides an adaptive routing method for industrial multimodal models based on task complexity awareness. By constructing a dedicated physical feature extraction network, it preprocesses and decomposes the spatiotemporal features of industrial multimodal data streams, achieving accurate identification of industrial physical law features and the physical constraint set of the current operating condition. Through a pre-defined multi-dimensional model capability profile matrix, it calculates the matching degree between feature vectors and model capabilities. Simultaneously, it uses the physical constraint set to perform safety verification on candidate models, obtaining safety scores for each model. Based on this, it integrates the matching degree score, safety score, and resource consumption index for multi-objective optimization calculation, accurately determining the target routing model suitable for the current operating condition and invoking it for inference. Furthermore, this application dynamically updates the model capability profile by collecting real-time measured performance data of model execution, dynamically adjusts the weight coefficients of multi-objective optimization based on operating condition complexity and risk index, and constructs a three-level safety degradation strategy and an online optimization mechanism for routing strategies, achieving closed-loop optimization of routing decisions. This application realizes the adaptation, intelligence and security of routing decision-making for industrial multimodal models, effectively reduces the fluctuation of inference accuracy, improves the rationality of model selection, takes into account both resource utilization efficiency and industrial production safety, adapts to the dynamic working conditions of industrial sites, and meets the stringent requirements of industrial scenarios for real-time performance and stability. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating the adaptive routing method for industrial multimodal models based on task complexity awareness provided in this embodiment of the invention. Figure 2 This is a schematic diagram of the structure of the industrial multimodal model adaptive routing system based on task complexity awareness provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0012] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0013] The following are explanations of some of the words in the text: Physical Feature Extraction Network: A network architecture specifically designed for industrial multimodal data processing. It features a dual-channel encoder structure and can decompose the spatiotemporal features of industrial multimodal data to extract the physical characteristics of the data. It is the core network for feature extraction of industrial multimodal data.

[0014] Model Capability Profile Matrix: A multi-dimensional matrix used to quantitatively evaluate the comprehensive capabilities of industrial multimodal candidate models. It includes standardized scores for multiple dimensions such as basic capabilities and industry-specific capabilities, and serves as the core basis for calculating the model-operating condition matching degree.

[0015] Spatiotemporal feature decomposition: the process of separating and extracting features of industrial multimodal data according to the time and space dimensions, which can yield a time-varying feature field containing features such as time series and frequency domain and a spatial feature field containing features such as geometry and topology.

[0016] Physical constraint set: A set of quantitative constraints extracted from industrial physical laws and safety regulations based on the current working conditions of the industrial site. It is the core basis for the safety verification of candidate models.

[0017] Barrier synchronization mechanism: a progress monitoring mechanism used in multi-model parallel inference. It monitors the inference progress of each model through atomic counter locking to ensure that the results are fused only after all models have completed inference.

[0018] Online learning algorithm: An algorithm that can dynamically calculate the policy gradient and update model parameters / routing policy parameters based on real-time collected performance data, thereby achieving real-time closed-loop optimization of the routing policy.

[0019] Comprehensive benefit function: This function is used to evaluate the effectiveness of routing strategy execution. It is calculated by comparing actual performance indicators with preset target values ​​and is the core indicator for determining whether a routing strategy needs to be optimized.

[0020] Figure 1 This is a flowchart illustrating the adaptive routing method for industrial multimodal models based on task complexity awareness provided in this embodiment of the invention.

[0021] like Figure 1 As shown in the figure, the adaptive routing method for industrial multimodal models based on task complexity awareness provided in this embodiment of the invention mainly includes the following steps: 101. Acquire industrial multimodal data streams from industrial sites, clean and normalize the industrial multimodal data streams to obtain standardized multimodal datasets; In a specific implementation process, industrial multimodal data streams such as vibration, temperature, and oil chromatography collected by field vibration sensors, infrared thermal imagers, and oil chromatography analyzers can be acquired through an industrial IoT gateway. Outlier removal and missing value completion are performed on the industrial multimodal data streams. Then, normalization processing is used to map data sources with different dimensions to a unified numerical range to obtain a standardized multimodal dataset.

[0022] Specifically, receiving industrial multimodal data streams ,in For vibration data, For thermal imaging data, For process parameters, The signal is an acoustic signal. Data cleaning, outlier removal, and normalization are performed using a standardized processing unit. in The mean, Standard deviation, To prevent small constants from being divided by zero.

[0023] 102. The standardized multimodal dataset is decomposed into spatiotemporal features using a physical feature extraction network to extract feature vectors and physical law features, and the set of physical constraints corresponding to the current working condition is identified. In a specific implementation, the multimodal dataset can be input into a pre-built physical feature extraction network. This network extracts feature vectors and physical laws that characterize the transformer's operating state from the data through dedicated spatiotemporal feature decomposition logic. At the same time, it identifies the set of physical constraints such as voltage, temperature, and vibration amplitude under the current transformer load conditions by combining power equipment safety standards.

[0024] Specifically, the standardized multimodal dataset can be decomposed into a time-varying feature field and a spatial feature field using a dual-channel encoder structure in the physical feature extraction network. The time-varying feature field includes temporal dynamic features, frequency domain features, and trend features; the spatial feature field includes geometric features, topological features, and the physical law features. The physical feature extraction network employs a dual-channel parallel structure. The time channel consists of a sequentially connected one-dimensional convolutional layer, an LSTM layer, and a fully connected layer, used to extract temporal dynamic features, frequency domain features, and trend features. The spatial channel includes three parallel branches: a convolutional neural network branch extracts geometric features, a graph convolutional network branch extracts topological features, and a physical information neural network branch extracts physical law features. The outputs of these three branches are concatenated and then dimensionality-reduced by a fully connected layer. The time-varying feature field and spatial feature field outputs from the dual channels are weighted and fused using an attention mechanism, i.e., the correlation weights of the two feature fields are calculated and then summed to obtain a unified multimodal feature representation.

[0025] In detail, when performing spatiotemporal feature decomposition on standardized multimodal datasets, the processing is carried out based on a specially designed dual-channel encoder structure in the physical feature extraction network. This structure includes two core channels: a time-dimensional encoder and a spatial-dimensional encoder, which correspond to the decomposition and extraction of time-varying feature fields and spatial feature fields, respectively.

[0026] The standardized multimodal dataset is input into the time-dimensional encoder. The encoder extracts from the data the time-series dynamic features that characterize the changes of industrial objects over time, the frequency domain features that reflect the frequency distribution of the data, and the trend features that reflect the long-term trend of the data through the core logic of time-series analysis, frequency domain transformation and trend fitting. The above three types of features together constitute the time-varying feature field.

[0027] Simultaneously, the standardized multimodal dataset is input into the spatial dimension encoder. This encoder extracts geometric features that characterize the spatial form of industrial objects, topological features that reflect the connection relationships between industrial equipment, and physical law features that conform to the laws of industrial physics from the data through the core logic of geometric morphology analysis, topological relationship recognition, and physical law mining. The above three types of features together constitute the spatial feature field, completing the entire spatiotemporal feature decomposition process.

[0028] In a specific implementation process, it is possible to identify whether the current data conforms to the industrial physical laws function: ; in Let p represent the p-th physical constraint, such as the energy conservation constraint, mass balance constraint, second law of thermodynamics constraint, etc. Each constraint is defined as: For constraint functions, This is the allowable deviation threshold.

[0029] Furthermore, the feature variance and feature entropy values ​​of each feature dimension in the time-varying feature field and the spatial feature field can be calculated; normalization calculations are performed based on the feature variance and feature entropy values ​​to obtain a feature weight vector; the feature weight vector, the time-varying feature field, the spatial feature field, and the physical constraint set are combined to generate the feature vector. Specifically, a standardized feature vector is generated. ,in Let C be the feature weight vector, and let C be the set of physical constraints, which serves as the core input for routing decisions.

[0030] In detail, after completing the decomposition of the time-varying feature field and the spatial feature field, the feature variance and feature entropy value are calculated for each feature dimension in the two feature fields. The feature variance is used to characterize the degree of dispersion of the feature of that dimension, and the feature entropy value is used to characterize the information richness of the feature of that dimension. Together, they reflect the importance of the feature dimension to the current working condition.

[0031] Subsequently, the feature variance and feature entropy values ​​of all feature dimensions are normalized to eliminate dimensional differences. Based on the calculation results, a corresponding weight value is assigned to each feature dimension. All weight values ​​together constitute the feature weight vector. The higher the weight value, the stronger the importance of the corresponding feature dimension in the current working condition.

[0032] Finally, the generated feature weight vector is structurally combined with the obtained time-varying feature field, spatial feature field, and identified physical constraint set, and integrated into a complete feature vector according to the preset feature splicing rules. This vector will serve as the core input for subsequent model matching degree calculation.

[0033] Based on spatiotemporal feature decomposition, this embodiment quantifies the importance of feature dimensions through feature variance and feature entropy, making the allocation of feature weights more scientific. By generating feature weight vectors, the role of key features is highlighted, avoiding interference from irrelevant features on model matching. At the same time, multiple types of core information are combined to generate feature vectors, enabling the feature vectors to comprehensively and accurately represent the characteristics of the current industrial conditions, laying the foundation for accurate matching between the model and the conditions, and effectively improving the accuracy of matching degree calculation.

[0034] In a specific implementation, IPFS-Net employs a dual-channel encoder structure to decompose the input data into a time-varying feature field and a spatial feature field: Time-varying characteristic field: ; Temporal dynamic features are extracted using one-dimensional convolution and LSTM networks; Frequency domain features are extracted using Fast Fourier Transform (FFT) and wavelet transform. Trend characteristics are extracted through moving averages and difference analysis; : The original multimodal data D after being processed by the standardized processing unit.

[0035] Spatial characteristic field: ; Geometric features are extracted using a convolutional neural network (CNN). Topological features are extracted using a Graph Convolutional Network (GCN). Physical laws and characteristics are extracted using a Physics-Informed Neural Network (PINN).

[0036] In a specific implementation process, an industrial feature importance scoring function can be designed to quantify the importance of each feature dimension under the current operating conditions: ; in, For the first The weights of each feature For the first The characteristic variance of each feature For the first The feature entropy value of each feature. For the first The characteristic variance of each feature For the first The feature entropy value of each feature. and To adjust parameters, dynamic adjustments are made based on the industrial scenario.

[0037] 103. Based on the feature vector and the preset model capability profile matrix, a matching calculation is performed to obtain the matching degree score between each candidate model and the current working condition; In a specific implementation, the model capability profile matrix includes standardized scores for the following dimensions: Dimension 1: This includes fundamental capabilities such as accuracy, response time, resource consumption, and robustness. In a specific implementation, the basic performance metrics for evaluating the model include: Accuracy: ; TP (True Positive): The number of true positive samples. This refers to the number of fault / abnormal samples correctly predicted by the model.

[0038] TN (True Negative): The number of true negative samples. This refers to the number of normal samples correctly identified by the model.

[0039] FP (False Positive): Number of false positive samples. This refers to the number of normal samples that the model mistakenly reports as faults (false positives).

[0040] FN (False Negative): Number of false negative samples. This refers to the number of faulty samples missed by the model (false negatives).

[0041] Response time: ; Represents the response end time. This represents the start time of the response.

[0042] Resource consumption: ; : Normalized GPU utilization (value 0-1), such as the ratio of current video memory usage to total video memory.

[0043] System memory usage (value 0-1).

[0044] The normalized power consumption index (value 0-1) is calculated based on the device's TDP (Thermal Design Power).

[0045] , , Resource weight coefficient.

[0046] robustness: ; The inference error rate of the model on a test set after applying noise of a specific intensity (such as Gaussian white noise or impulse noise).

[0047] The raw inference error rate of the model on a clean, noise-free benchmark dataset.

[0048] Dimension 2: Industry-specific capability dimensions including physical law compliance, temporal consistency, and multimodal fusion capabilities; Among them, the degree of conformity with physical laws: ; For the first The model predicts the output vector for each sample. The physical law function; N is the total number of samples within the evaluation window; This is an indicator function. It takes the value 1 when the condition within the parentheses is met, and 0 otherwise.

[0049] Timing consistency: ,in Representative at The actual observed value at that moment, Representative at Predicted value at time, The first difference representing the true value, The first difference represents the predicted value.

[0050] Multimodal fusion capability: The Pearson correlation coefficient between feature fusion quality and prediction accuracy was calculated.

[0051] Dimension 3: Scenario adaptability dimension, including working condition adaptability, data quality tolerance, and anomaly handling capabilities; Operating condition adaptability: ; Data quality tolerance: ; Exception handling capabilities: .

[0052] Dimension 4: A dimension of security boundary capability that includes security boundary preservation, anomaly recovery speed, and conservatism index; Safety boundary preservation: ,in This refers to the number of samples whose model output falls within the preset industrial safety threshold range. This refers to the total number of inference samples.

[0053] Abnormal recovery speed:

[0054] Conservatism Index: ; in, This refers to the number of times the model chooses to output low-risk strategies such as "unknown", "suspension suggestion" or "manual review" when the uncertainty is high (such as when the confidence level is below the threshold). This refers to the total number of decisions made.

[0055] Dimension 5: Evolutionary capability dimension, including knowledge update speed, concept drift adaptability, and transfer learning efficiency.

[0056] Among them, the speed of knowledge updates: The accuracy rate increases per unit of time; Concept drift adaptability: ; Transfer learning efficiency: .

[0057] Based on the above dimensions, for each candidate model Generate a five-dimensional capability profile vector: in The standardized score (0-1) for the i-th model in the j-th dimension is assigned. The capability profile is updated in real-time using a sliding window mechanism, with the window size dynamically adjusted based on the industrial scenario. ,in, Let be the comprehensive ability profile score vector of model i at time t. Instantaneous capability score calculated based on measured data within the current time window; This is the attenuation factor, usually set to 0.95, to ensure that the profile reflects historical performance while adapting quickly to changes.

[0058] In detail, the model capability profile matrix used for model matching degree calculation is a five-dimensional standardized scoring matrix. For each industrial multimodal candidate model, a comprehensive capability assessment is carried out from five dimensions and a standardized score of 0-1 is given to generate the model capability profile matrix.

[0059] First, the evaluation focuses on basic capabilities, examining the model's core performance indicators, including the accuracy of inference, response time for inference, resource consumption required for operation, and robustness against noise interference. Second, the evaluation considers industry-specific capabilities, taking into account the specific characteristics of industrial scenarios, examining the consistency between the model output and industrial physical laws, the temporal consistency of inference results, and the ability to fuse and process multimodal data. Next, from the perspective of scenario adaptability, we examine the model's ability to adapt to different industrial conditions, its tolerance for low-quality industrial data, and its ability to handle abnormal data. Then, from the perspective of safety boundary capability, we examine the model's ability to maintain industrial safety boundaries when performing inference, its recovery speed after anomalies occur, and its conservative decision-making index under high uncertainty. Finally, the evaluation considers the evolutionary capability dimension, examining the model's online knowledge update speed, adaptability to conceptual drift in industrial data, and cross-scenario transfer learning efficiency. The standardized scores for all dimensions are organized in matrix form to create a model capability profile matrix for each candidate model.

[0060] In a specific implementation, feature-capability matching calculation can be performed as follows: Calculate the matching score between the input features and the capabilities of each model: in For feature weights, As input features, For the model Ability score on feature k, For similarity functions (such as cosine similarity).

[0061] Furthermore, the task complexity-aware adaptive routing method for industrial multimodal models in this embodiment may also include: The instantaneous capability score is calculated by collecting the measured performance data of each candidate model in real time within the current time window through the performance monitoring probe. The historical profile score is weighted using a decay factor and then combined with the instantaneous ability score to perform a moving average calculation to obtain the updated model ability profile score vector.

[0062] In detail, to ensure that the model capability profile matrix can reflect the actual running status of the model in real time, a performance monitoring probe deployed in the model execution stage is used to monitor the running status of each candidate model in real time within the preset current time window. The measured performance data of various dimensions such as the accuracy of model inference, response time, and physical law conformity are collected. Based on the measured data, the instantaneous capability score of each model is calculated according to the aforementioned scoring criteria.

[0063] Subsequently, a decay factor is pre-set to balance the weights of historical profile scores and instantaneous capability scores. The historical profile scores of the model are weighted using the decay factor to reduce the influence of outdated historical data. The weighted historical profile scores are then averaged with the current instantaneous capability scores. The calculation results are integrated into an updated model capability profile score vector according to the aforementioned five-dimensional structure, thereby dynamically updating the model capability profile matrix.

[0064] This embodiment realizes the dynamic real-time updating of the model capability profile matrix. By collecting measured data through performance monitoring probes, the model capability assessment is more in line with the actual operating state, avoiding the problem of static scoring being out of touch with the actual model performance. Through the calculation of decay factors and moving averages, an effective balance between historical performance and instantaneous performance is achieved, making the updated profile score more reasonable and continuous. The dynamically updated model capability profile matrix makes the subsequent model matching degree calculation more accurate, further improving the adaptive capability of routing decisions.

[0065] 104. Use the physical constraint set to perform security verification on each candidate model. By calculating the deviation between the predicted output of each candidate model and the constraint function in the physical constraint set, the security score of each candidate model is obtained. When using a set of physical constraints to conduct safety verification of each candidate model, the predicted output of each candidate model for the current working condition can be obtained first. Then, the result can be substituted into each constraint function in the set of physical constraints to calculate the specific deviation between the model output value and the compliance range defined by the constraint function. Subsequently, the deviation values ​​under all constraint functions are normalized to eliminate the dimensional differences between different constraints. Combined with the preset safety weights of each constraint function, the comprehensive deviation value of the model is calculated. Finally, the comprehensive deviation value is mapped to a standardized safety score. The smaller the comprehensive deviation value, the higher the safety score of the model. This completes the safety verification of a single model. The safety scores of all candidate models are calculated according to this rule.

[0066] Specifically, the following function can be used to verify whether the candidate model meets the safety constraints of the current operating condition: ; in For the model The predicted output, For the first A physical constraint function, This is the allowable deviation threshold. When (When the safety threshold is set to 0.2, the model) It has been marked as unsafe and has entered the security downgrade process.

[0067] 105. Combine the matching score, the security score, and the resource consumption index of each candidate model to perform multi-objective optimization calculations and determine the target routing model; Specifically, the complexity index and risk index of the current working condition can be obtained, and the values ​​of matching degree weight, security weight, resource cost weight and business value weight can be adjusted accordingly. The matching degree score, security score, resource consumption index and preset business value score are weighted and summed using the matching degree weight, security weight, resource cost weight and business value weight to obtain the comprehensive evaluation score of each candidate model. The candidate model with the largest comprehensive evaluation score is selected as the target routing model.

[0068] The function for multi-objective optimization calculation is: ; in, Each weighting coefficient is dynamically adjusted based on the importance of the industrial scenario to meet the following requirements. ,in, The matching weight represents the degree of matching and is positively correlated with the complexity of the working conditions. The more complex the working conditions, the more necessary it is to rely on a high-precision feature matching model. This represents the security weight, which is positively correlated with the risk index. When the risk index approaches the safety boundary, the security weight is forcibly increased. It represents the resource cost weight and is positively correlated with resource load, but it is suppressed under high risk. When the system load is high and the risk is low, cost reduction is given priority, and when the risk is high, the cost weight is reduced. It represents the weight of business value and serves as a basic adjustment item, usually remaining stable or adjusting with long-term strategies. For the model Resource consumption score; For the model Business value score (calculated based on historical decision benefits).

[0069] In detail, when conducting multi-objective optimization calculations to determine the target routing model, the operational complexity index is first calculated based on the current equipment operating status and process complexity of the industrial site. A risk index is then calculated based on the safety level of the industrial scenario and the degree of equipment hazard. The operational complexity index is calculated by extracting four indicators from the feature vectors: the number of data dimensions, feature entropy value, temporal change rate, and modal alignment error. These are then input into a pre-trained complexity assessment model, which outputs a normalized complexity index. This model is a support vector regression model trained based on historical data. The risk index is calculated by obtaining the current scenario's preset safety level, equipment health status score, and the approximation degree of each constraint in the physical constraint set. The weighted sum of these three factors is then mapped to the 0-1 range as the risk index.

[0070] Subsequently, the weights of each indicator are dynamically adjusted based on the values ​​of the two indices: the higher the complexity of the working condition, the higher the matching weight; the higher the risk index, the higher the safety weight; when the system resource load is high, the resource cost weight is appropriately increased; and a relatively stable business value weight is set in combination with the core needs of industrial production to ensure that the sum of each weight value is 1.

[0071] Next, a business value score is preset for each candidate model. This score is determined based on the actual application value of the model in industrial production. Using the adjusted weight values, the matching score, safety score, resource consumption index and preset business value score obtained in claim 1 are weighted respectively. The weighted results are then summed to obtain the comprehensive evaluation score of each candidate model.

[0072] Finally, the comprehensive evaluation scores of all candidate models are ranked, and the candidate model with the highest score is selected as the final target routing model.

[0073] It should be noted that when multiple models have similar scores (difference < 5%), the target routing model with lower resource consumption should be selected.

[0074] When none of the candidate models meet the safety constraints, the system automatically initiates a three-level degradation strategy: Level 1: Select the model with the highest degree of conformity to physical laws. When routine security checks fail, the system no longer considers the model's accuracy, response time, or cost (i.e., ignores these factors). In This means that although the model is not perfect in terms of efficiency and accuracy, it still maintains strict constraints of physical laws (such as the law of conservation of energy) and avoids serious damage caused by ignoring physical laws in dangerous operating conditions.

[0075] Level 2: Switch to traditional algorithms based on physical models, abandoning large-scale model inference. At this point, the scores of AI models are no longer compared, and traditional algorithms based on differential equations or expert rules are forcibly invoked.

[0076] Level 3: Trigger manual review process and suspend automatic decision-making. At this time, the output confidence level is 0, the security level is marked as "high risk", and an interrupt signal is sent to the control system.

[0077] This embodiment realizes the dynamic adjustment of weight coefficients in multi-objective optimization, making the weight allocation more in line with the actual needs of the current industrial operating conditions and avoiding the problem that fixed weights cannot adapt to dynamic operating conditions. By obtaining a comprehensive evaluation score through weighted summation, it realizes the comprehensive consideration of multiple dimensions of indicators such as matching degree, security, resource consumption and business value, making the selection of target routing models more scientific and comprehensive. By selecting the model with the highest comprehensive evaluation score, it ensures that the selected target routing model achieves the optimal balance of indicators of each dimension under the current operating conditions, effectively improving the rationality and adaptability of routing decisions.

[0078] Furthermore, the task complexity-aware industrial multimodal model adaptive routing method of this embodiment can also determine whether the safety scores of all candidate models are lower than a preset safety threshold; if they are all lower than the preset safety threshold, a degradation strategy is automatically initiated, the degradation strategy including: first level selecting the model with the highest physical law conformity for inference; if the first level strategy fails, the second level switching to a physical algorithm based on differential equations or expert rules; if the second level strategy fails, the third level triggering a manual review process and sending an interrupt signal to the control system.

[0079] In detail, a preset safety threshold can be set, which is determined according to the safety requirements of the industrial scenario. After the safety scores of each candidate model are calculated and the comprehensive evaluation scores are sorted, it can be determined whether the safety scores of all candidate models are lower than the threshold.

[0080] If a model's security score is higher than the threshold, the target routing model is determined according to the aforementioned rules; if all models' security scores are lower than the threshold, a three-level security degradation strategy is immediately and automatically initiated: the first-level strategy prioritizes discarding other indicators such as matching degree and resource consumption, and selects only the model with the highest physical law conformity from all candidate models, and calls that model to execute the inference task; If the model inference results still do not meet industrial safety requirements after the first-level strategy is executed, the first-level strategy is deemed to have failed. Then, the second-level strategy is activated, discarding all artificial intelligence models and switching to traditional physical algorithms based on industrial differential equations or expert rules for inference calculation. If the second-level strategy still fails to produce a result that meets safety requirements, it is determined that the second-level strategy has failed. The third-level strategy is then activated, immediately triggering a manual review process. At the same time, an equipment operation interruption signal is sent to the control system of the industrial site to suspend automatic reasoning and decision-making, awaiting manual intervention.

[0081] 106. Call the target routing model to perform the inference task and output the inference results.

[0082] Specifically, when the routing decision determines a single target routing model, the target routing model is directly loaded for inference; the execution flow is: model loading → input data preprocessing → model inference → postprocessing → result output; optimization technique: adopt a model preheating mechanism to preload frequently used models into memory during idle periods.

[0083] When multiple target routing models are executed in parallel, the inference progress of each target routing model is monitored through a barrier synchronization mechanism. After all target routing models have completed inference, the output weight coefficients of each model are dynamically calculated based on the model capability profile matrix. The predicted outputs of each model and the prediction results based on the physical model are weighted and fused to obtain the final inference result.

[0084] In detail, when calling the target routing model to perform inference tasks, if multiple target routing models are determined based on the multi-objective optimization results and need to be executed in parallel, the barrier synchronization mechanism is first activated to uniformly monitor the inference progress of each model. This mechanism uses atomic counter locking to count the number of models that have completed inference in real time, ensuring that the subsequent result fusion work is carried out only after all target routing models have completed inference.

[0085] After all models have completed inference, based on the model capability profile matrix and the standardized scores of each model in each dimension under the current working conditions, the output weight coefficient of each model is dynamically calculated. The higher the model capability is adapted to the current working conditions, the larger the corresponding output weight coefficient is.

[0086] Simultaneously, the corresponding physical prediction results are calculated through the industrial physical model as a safety fallback result; finally, the predicted outputs of each target routing model and the predicted results of the physical model are weighted and fused using the calculated output weight coefficients, and the fused result is output as the final inference result.

[0087] The function for the resulting fusion algorithm is: ; in: For the first The predicted output of each model; These are the model weight coefficients, dynamically calculated based on the model's capability profile under the current operating conditions. This serves as a safety net, representing the prediction results based on the physical model.

[0088] The above-mentioned fusion strategy automatically adjusts according to the type of industrial scenario. Typical adjustment rules are as follows: Safety-critical scenarios (such as equipment failure early warning): Preferring a conservative model Increased weighting; efficiency-critical scenarios (such as process optimization): Preferring high-precision models The weight has been reduced.

[0089] This embodiment supports the parallel execution of multi-target routing models, fully leveraging the advantages of multi-model collaborative reasoning and improving the comprehensiveness and accuracy of the reasoning results. A barrier synchronization mechanism enables unified monitoring of the reasoning progress of multiple models, avoiding distortion of the fusion results caused by some models not completing their reasoning. The output weight coefficients are calculated based on a dynamically updated model capability profile matrix, making weight allocation more precise. The introduction of physical model prediction results as a safety net further enhances the security and reliability of the reasoning results.

[0090] In some embodiments, the task complexity-aware industrial multimodal model adaptive routing method may further include: The system collects measured response time, measured accuracy, measured resource utilization, and measured security compliance rate during model execution. It then compares these measured values ​​with preset target values ​​to calculate the comprehensive benefit function of the current routing strategy. When the comprehensive benefit function is lower than a preset benefit threshold, it uses an online learning algorithm to calculate the strategy gradient and updates the weight parameters of the routing strategy.

[0091] In detail, during the inference task performed by the target routing model, the measured performance indicators of the model are collected in real time through performance monitoring probes, including measured response time, measured accuracy, measured resource utilization and measured security compliance rate.

[0092] Subsequently, preset target values ​​were set for the above four types of performance indicators. These target values ​​were determined based on the real-time, stability, and security requirements of the industrial scenario. The measured performance indicators were then compared and analyzed with their corresponding preset target values. Based on the comparison results and according to the preset function calculation rules, the comprehensive benefit function of the current routing strategy was calculated. The higher the function value, the better the execution effect of the current routing strategy.

[0093] A benefit threshold is preset. When the calculated comprehensive benefit function value is higher than the threshold, the current routing strategy remains unchanged. When the comprehensive benefit function value is lower than the threshold, the online learning algorithm is immediately invoked. Based on the deviation between the measured performance index and the preset target value, the policy gradient of the routing strategy is calculated. Then, the weight parameters of the routing strategy are dynamically updated based on the policy gradient to complete the optimization of the routing strategy.

[0094] In a specific implementation process, key performance indicators are collected in real time: Response time The total time taken from data input to result output; accuracy : Obtained through delayed feedback or manual verification; resource utilization rate Utilization rates of resources such as GPU, memory, and network; Safety compliance rate The proportion of output results that meet safety constraints; The overall benefit function of the current routing strategy is calculated as follows: ; in Weighted by benefits, , , Preset target values ​​for industrial scenarios.

[0095] Gradient calculation and parameter update: When Below the threshold At that time, parameter optimization is triggered: ; in These are routing policy parameters (including weight coefficients, security thresholds, etc.). For learning rate, Loss function: ; Let KL divergence measure the difference in distribution between the current policy and the security policy. Represents the safety baseline distribution. Represents the current policy distribution. This represents the loss weighting coefficient.

[0096] This embodiment achieves real-time closed-loop optimization of routing strategies. By collecting measured performance indicators and calculating the comprehensive benefit function, it can accurately evaluate the execution effect of the current routing strategy and promptly identify strategy failures. By using an online learning algorithm to calculate the strategy gradient and update the weight parameters, the routing strategy can be dynamically adjusted according to the actual operating state of the model, effectively solving the problem of lagging feedback optimization in existing technologies. The dynamic optimization of the routing strategy allows the model's routing decisions to continuously adapt to the dynamic working conditions of the industrial site, ensuring that the system always maintains the optimal operating state and improving the system's adaptability and practicality.

[0097] Furthermore, the fluctuation of the comprehensive benefit function can be monitored to determine whether the preset performance anomaly triggering condition is met. If the performance anomaly triggering condition is met, the routing strategy parameters are rolled back to the historical optimal parameter set. At the same time, the system switches to a conservative routing strategy based on preset rules until a manual intervention instruction is received.

[0098] In detail, during the process of calculating the comprehensive benefit function and optimizing the routing strategy, the numerical fluctuation of the comprehensive benefit function is monitored in real time, and performance anomaly triggering conditions are set in advance, such as the comprehensive benefit function falling below 80% of the preset benefit threshold multiple times in a row.

[0099] The system continuously assesses whether the fluctuation of the comprehensive benefit function meets the triggering condition. If it does not, the weight parameters of the routing strategy are updated normally according to the aforementioned rules. If it does, the system determines that the current routing strategy has a performance abnormality and immediately initiates the abnormality handling mechanism: first, all parameters of the routing strategy are rolled back to the historical optimal parameter set recorded by the system, restoring the routing strategy to the state of the historically best-performing strategy. At the same time, the current dynamic routing strategy will be switched to a conservative routing strategy based on preset rules. This strategy prioritizes the safety and stability of industrial production, abandoning non-core indicators such as resource consumption and response speed. The conservative routing strategy will continue to be executed until the system receives a manual intervention instruction, at which point the dynamic routing strategy will be restored or other adjustments will be made based on the manual judgment.

[0100] This embodiment constructs a performance anomaly handling mechanism for routing strategies. By monitoring fluctuations in the comprehensive benefit function, it can promptly identify strategy anomalies, avoiding system performance degradation caused by the continuous execution of abnormal strategies. Rolling back parameters to the historical optimal set can quickly restore the system to a stable operating state, reducing the impact of strategy anomalies. Switching to a conservative routing strategy and waiting for manual intervention provides a final safety net for industrial production, maximizing production safety in industrial scenarios and further improving the reliability and robustness of the entire routing method.

[0101] Based on the same general inventive concept, this invention also protects an industrial multimodal model adaptive routing system based on task complexity awareness. The industrial multimodal model adaptive routing system based on task complexity awareness provided by this invention will be described below. The industrial multimodal model adaptive routing system based on task complexity awareness described below can be referred to in correspondence with the industrial multimodal model adaptive routing method based on task complexity awareness described above.

[0102] Figure 2 This is a schematic diagram of the structure of the task complexity-aware industrial multimodal model adaptive routing system provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the task complexity-aware industrial multimodal model adaptive routing system of this embodiment includes an acquisition module 21, an extraction module 22, a matching module 23, a verification module 24, an optimization module 25, and an inference module 26.

[0103] The acquisition module 21 is used to acquire industrial multimodal data streams from industrial sites, clean and normalize the industrial multimodal data streams to obtain standardized multimodal datasets. Extraction module 22 is used to perform spatiotemporal feature decomposition on the standardized multimodal dataset using a physical feature extraction network, extract feature vectors and physical law features, and identify the set of physical constraints corresponding to the current working condition; The matching module 23 is used to perform matching calculations based on the feature vector and the preset model capability profile matrix to obtain the matching degree score between each candidate model and the current working condition. The verification module 24 is used to perform security verification on each candidate model using the physical constraint set. By calculating the deviation between the predicted output of each candidate model and the constraint function in the physical constraint set, the security score of each candidate model is obtained. Optimization module 25 is used to perform multi-objective optimization calculations by combining the matching score, the security score and the resource consumption index of each candidate model to determine the target routing model; The reasoning module 26 is used to call the target routing model to perform reasoning tasks and output reasoning results.

[0104] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a task complexity-aware adaptive routing method for industrial multimodal models.

[0105] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] It should be noted that all relevant information that may be involved in the various embodiments of the present invention is processed in strict accordance with the requirements of laws and regulations, following the principles of legality, legitimacy, and necessity, based on the reasonable purpose of the business scenario, and is information that users actively provide or generate during the use of the product / service, as well as information obtained with user authorization.

[0107] The information processed by this invention may vary depending on the specific product / service scenario and should be based on the specific scenario in which the user uses the product / service. This may involve user account information, device information, or other related information. This invention will treat the relevant information and its processing with the utmost diligence.

[0108] This invention places great emphasis on the security of relevant information and has adopted reasonable and feasible security protection measures that comply with industry standards to protect user information and prevent unauthorized access, public disclosure, use, modification, damage or loss of relevant information.

[0109] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0110] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A task complexity-aware adaptive routing method for industrial multimodal models, characterized in that, include: Acquire industrial multimodal data streams from industrial sites, clean and normalize the industrial multimodal data streams to obtain standardized multimodal datasets; The standardized multimodal dataset is decomposed into spatiotemporal features using a physical feature extraction network to extract feature vectors and physical law features, and to identify the set of physical constraints corresponding to the current working condition. The matching calculation is performed based on the feature vector and the preset model capability profile matrix to obtain the matching degree score between each candidate model and the current working condition; The physical constraint set is used to verify the safety of each candidate model. The safety score of each candidate model is obtained by calculating the deviation between the predicted output of each candidate model and the constraint function in the physical constraint set. By combining the matching score, the security score, and the resource consumption index of each candidate model, a multi-objective optimization calculation is performed to determine the target routing model; The target routing model is invoked to perform the inference task and the inference results are output.

2. The adaptive routing method for industrial multimodal models based on task complexity awareness according to claim 1, characterized in that, The standardized multimodal dataset is subjected to spatiotemporal feature decomposition using a physical feature extraction network to extract feature vectors and physical law features, including: The standardized multimodal dataset is decomposed into a time-varying feature field and a spatial feature field by using a dual-channel encoder structure in the physical feature extraction network. The time-varying feature field includes time-series dynamic features, frequency domain features, and trend features; The spatial feature field includes geometric features, topological features, and physical law features.

3. The adaptive routing method for industrial multimodal models based on task complexity awareness according to claim 2, characterized in that, Also includes: Calculate the feature variance and feature entropy values ​​of each feature dimension in the time-varying feature field and the spatial feature field; Based on the feature variance and the feature entropy value, a normalization calculation is performed to obtain the feature weight vector; The feature vector is generated by combining the feature weight vector, the time-varying feature field, the spatial feature field, and the physical constraint set.

4. The adaptive routing method for industrial multimodal models based on task complexity awareness according to claim 1, characterized in that, The model capability profile matrix includes standardized scores for the following dimensions: It includes fundamental capability dimensions such as accuracy, response time, resource consumption, and robustness; Industry-specific capability dimensions include physical law compliance, temporal consistency, and multimodal fusion capabilities; The dimensions of scenario adaptability include working condition adaptability, data quality tolerance, and anomaly handling capabilities; The safety boundary capability dimensions include safety boundary preservation, anomaly recovery speed, and a conservatism index. Evolutionary capability dimensions include knowledge update speed, concept drift adaptability, and transfer learning efficiency.

5. The adaptive routing method for industrial multimodal models based on task complexity awareness according to claim 4, characterized in that, Also includes: The instantaneous capability score is calculated by collecting the measured performance data of each candidate model in real time within the current time window through the performance monitoring probe. The historical profile score is weighted using a decay factor and then combined with the instantaneous ability score to perform a moving average calculation to obtain the updated model ability profile score vector.

6. The adaptive routing method for industrial multimodal models based on task complexity awareness according to claim 1, characterized in that, By combining the matching score, the security score, and the resource consumption index of each candidate model, a multi-objective optimization calculation is performed to determine the target routing model, including: Obtain the complexity index and risk index of the current working condition, and adjust the values ​​of matching degree weight, security weight, resource cost weight and business value weight accordingly; The matching score, security score, resource consumption index, and preset business value score are weighted and summed using the matching degree weight, security weight, resource cost weight, and business value weight to obtain the comprehensive evaluation score of each candidate model. The candidate model with the highest comprehensive evaluation score is selected as the target routing model.

7. The adaptive routing method for industrial multimodal models based on task complexity awareness according to claim 6, characterized in that, Also includes: Determine whether the safety scores of all candidate models are lower than a preset safety threshold; If all values ​​are below the preset safety threshold, a degradation strategy is automatically initiated. The degradation strategy includes: firstly, selecting the model with the highest physical law conformity for inference; If the first-level strategy fails, the second level switches to a physical algorithm based on differential equations or expert rules. If the second-level strategy fails, the third level will trigger a manual review process and send an interrupt signal to the control system.

8. The adaptive routing method for industrial multimodal models based on task complexity awareness according to claim 1, characterized in that, Invoking the target routing model to perform inference tasks includes: When multiple target routing models are determined to be executed in parallel, the inference progress of each target routing model is monitored through a barrier synchronization mechanism. After all target routing models have completed inference, the output weight coefficients of each model are dynamically calculated based on the model capability profile matrix. The prediction outputs of each model and the prediction results based on the physics model are weighted and fused to obtain the final inference result.

9. The adaptive routing method for industrial multimodal models based on task complexity awareness according to claim 1, characterized in that, Also includes: The measured response time, measured accuracy, measured resource utilization, and measured security compliance rate during the model execution process were collected. The measured response time, measured accuracy, measured resource utilization, and measured security compliance rate are compared with preset target values ​​to calculate the comprehensive benefit function of the current routing strategy. When the comprehensive benefit function is lower than the preset benefit threshold, the policy gradient is calculated using an online learning algorithm, and the weight parameters of the routing policy are updated.

10. The adaptive routing method for industrial multimodal models based on task complexity awareness according to claim 9, characterized in that, Also includes: Monitor the fluctuation of the comprehensive benefit function and determine whether the preset performance anomaly triggering conditions are met; If the aforementioned performance anomaly triggering condition is met, the routing policy parameters will be rolled back to the historical optimal parameter set. At the same time, it switches to a conservative routing strategy based on preset rules until a manual intervention instruction is received.

Images