Reasoning methods, systems, and storage media based on hierarchical hybrid expert models

Abstract

This application provides a reasoning method, system, and storage medium based on a hierarchical hybrid expert model. The method effectively solves the problems of computational redundancy, resource allocation imbalance, and insufficient dynamic adjustment capability of traditional hybrid expert models by combining tree-structured expert cluster selection, two-layer routing mechanism, and dynamic resource scheduling. It has the advantages of improving the reasoning efficiency of multi-carbon tasks, optimizing the utilization rate of heterogeneous resources, and enhancing the adaptability to dynamic scenarios.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a reasoning method, system, and storage medium based on a hierarchical hybrid expert model. Background Technology

[0002] As deep learning models continue to grow in scale, single models face three core challenges in multi-task or multi-scenario inference: First, general-purpose models suffer from computational redundancy when processing diverse inputs, leading to exponential growth in resource overhead; second, traditional architectures struggle to adapt to dynamic load scenarios, with response latency fluctuating significantly with task complexity; and third, static model combination methods lack scenario awareness and cannot dynamically adjust computational paths based on input features. While hybrid expert models alleviate some of these problems through task decomposition strategies, existing implementations suffer from three key flaws: their flattened expert selection mechanism ignores the inherent hierarchical relationships of domain knowledge, resulting in coarse semantic space partitioning; the routing decision process lacks awareness of the system's real-time state, leading to unbalanced resource allocation; and the dynamic adjustment mechanism relies solely on offline metrics, making it difficult to respond promptly to runtime environment changes. Particularly in high-concurrency carbon-related data scenarios, existing systems often exhibit resource waste due to the lack of hierarchical routing and resource coordination scheduling capabilities, resulting in both overloaded hot experts and idle unpopular experts. Furthermore, rigid cache management strategies during cross-device deployment fail to adapt to the dynamic characteristics of heterogeneous computing environments, further restricting overall system performance. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a reasoning method, system and storage medium based on a hierarchical hybrid expert model, so as to improve the reasoning efficiency of multiple carbon tasks, optimize the utilization of heterogeneous resources and enhance the adaptability to dynamic scenarios.

[0004] In a first aspect, embodiments of this application provide a reasoning method based on a hierarchical hybrid expert model, including:

[0005] The raw carbon-related data is preprocessed to generate carbon feature vectors in a unified format;

[0006] Using a first-layer router, the most relevant top M candidate expert clusters are selected from multiple expert clusters in a tree-structured hierarchical structure based on the carbon feature vector. Using a second-layer router, attention calculation is performed on the concatenated input of the carbon feature vector and the expert model description vector within each candidate expert cluster, combining the embedded representation of the corresponding expert model. The corresponding expert score is output, and the top K expert models with the highest scores and their weights are selected from each candidate expert cluster to form a candidate expert model set.

[0007] Based on the candidate expert model set and the current system resource status, expert model cache management and scheduling are performed to obtain an expert model execution configuration plan, wherein the execution configuration plan is used to specify the expert model to be executed and its target device;

[0008] Based on the execution configuration plan, inference execution is performed on each expert model to be executed, and the inference output results and running performance indicators of each expert model are collected asynchronously to obtain the result output set of each expert model;

[0009] The output set of results is fused according to the weights of each expert model to generate the final fusion inference result of carbon data.

[0010] In another feasible implementation, the preprocessing of the raw carbon-related data to generate carbon feature vectors in a uniform format includes:

[0011] Key emission factors are extracted from industrial emission data and standardized.

[0012] For traffic energy consumption data, construct vehicle emission feature vectors based on spatiotemporal trajectories;

[0013] Energy density indices were extracted from building energy consumption data according to building type and climate zone.

[0014] For remote sensing image data, surface temperature and vegetation index are extracted and principal component dimensionality reduction is performed;

[0015] In the presence of multimodal carbon-related data, carbon feature vectors from various modes are merged into a unified carbon feature vector through concatenation or cross-modal attention.

[0016] In another feasible implementation, the first-layer router is a multilayer perceptron network; the step of using the first-layer router to select the top M most relevant candidate expert clusters from multiple expert clusters in a tree-like hierarchical structure based on the carbon feature vector includes:

[0017] The carbon feature vector is input into the multilayer perceptron network to evaluate the matching degree of each expert cluster in the tree-structured hierarchical network and obtain the expert cluster score.

[0018] After performing Softmax normalization on the scores of each expert cluster, the top M candidate expert clusters with the highest scores are selected.

[0019] In another feasible implementation, the second-layer router is a lightweight attention network; the second-layer router is used to perform attention calculations based on the carbon feature vector and the expert models in each candidate expert cluster, outputting the corresponding expert scores, and selecting the top K expert models with the highest scores and their weights in each candidate expert cluster to form a candidate expert model set, including:

[0020] The carbon feature vector is concatenated with the description vectors of each expert model in the candidate expert cluster to form a composite input. Multi-head self-attention or Trans former calculation is performed on the composite input to obtain the expert score.

[0021] After normalizing the expert scores using Softmax, the top K expert models with the highest expert scores in each candidate expert cluster and their corresponding weights are selected, and a candidate expert model set is formed based on the top K expert models.

[0022] In another feasible implementation, the step of performing expert model cache management and scheduling based on the candidate expert model set and the current system resource status to obtain an expert model execution configuration plan includes:

[0023] The most frequently invoked candidate expert models in the candidate expert model set are kept in the cache;

[0024] Candidate expert models that have not been selected for N consecutive rounds in the candidate expert model set are marked as cold models and unloaded from video memory. When a cold model is selected again in a future task, it is restored to a usable state through an asynchronous loading mechanism.

[0025] Based on the current system resource status, the target device for the task is selected, and the candidate expert models are matched to the target device using a parallel asynchronous scheduling method to obtain the execution configuration plan of the expert model.

[0026] In another feasible implementation, the execution configuration plan includes: recording the expert model ID, the assigned inference device ID, the binding information of the input carbon feature vector, and the output cache identifier; the inference device ID is dynamically selected based on the heterogeneous computing devices available in the current system and is asynchronously distributed through the task scheduler.

[0027] In another feasible implementation, based on the execution configuration plan, inference execution is performed on each expert model to be executed, and the inference output results and runtime performance indicators of each expert model are collected asynchronously to obtain the result output set of each expert model, including:

[0028] The carbon feature vector is format-adapted and bound to the input interface of the expert model to be executed specified in the execution configuration plan to generate a standardized input tensor;

[0029] The standardized input tensors are distributed to the devices containing the corresponding expert models to be executed, and an inference session is initiated.

[0030] During the execution of each expert model to be executed, an asynchronous listening method is used to detect whether an inference completion signal exists;

[0031] After the inference completion signal is detected, the inference output results of each expert model are immediately extracted and the running performance indicators are recorded simultaneously. The running performance indicators include inference time, memory usage, hardware utilization and carbon index calculation efficiency.

[0032] The inference output and the performance metrics are stored in the result cache pool to form the result output set of the expert model.

[0033] In another feasible implementation, the step of performing a fusion operation on the result output set according to the weights of each expert model to generate the final fusion inference result of the carbon data includes:

[0034] Based on the weights of each expert model and its output confidence, the result output set is weighted and summed to generate the final fused carbon emission results, carbon intensity index, and carbon trend prediction.

[0035] Secondly, embodiments of this application provide a reasoning system based on a hierarchical hybrid expert model, including:

[0036] The input processing module is used to preprocess the raw carbon-related data and generate carbon feature vectors in a unified format.

[0037] The routing decision module is used to select the top M most relevant candidate expert clusters from multiple expert clusters in a tree-structured hierarchical structure based on the carbon feature vector using a first-layer router; and to perform attention calculation on the concatenated input of the carbon feature vector and the expert model description vector within each candidate expert cluster by combining the embedded representation of the corresponding expert model with the second-layer router, outputting the corresponding expert score, and selecting the top K expert models with the highest scores and their weights in each candidate expert cluster to form a candidate expert model set.

[0038] An expert model library is used to perform expert model cache management and scheduling based on the candidate expert model set and the current system resource status, and to obtain an execution configuration plan for the expert model, wherein the execution configuration plan is used to specify the expert model to be executed and its target device;

[0039] The fusion scheduling module is used to perform inference execution on each expert model to be executed based on the execution configuration plan, asynchronously collect the inference output results and running performance indicators of each expert model, and obtain the result output set of each expert model;

[0040] The inference module is used to perform fusion calculations on the result output set according to the weights of each expert model to generate the final fusion inference result of carbon data.

[0041] Thirdly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are invoked and executed by a processor, the computer-executable instructions cause the processor to implement the above-described reasoning method based on a hierarchical hybrid expert model.

[0042] The embodiments of this application bring the following beneficial effects:

[0043] The aforementioned inference method, system, and storage medium based on a hierarchical hybrid expert model preprocesses the raw carbon-related data to generate carbon feature vectors in a unified format. Using a first-layer router, the most relevant M candidate expert clusters are selected from multiple expert clusters in a tree-like hierarchical structure based on the carbon feature vectors. A second-layer router, within each candidate expert cluster, combines the embedded representation of the corresponding expert model to perform attention calculations on the concatenated input of the carbon feature vector and the expert model description vector, outputting the corresponding expert score. The top K expert models with the highest scores and their weights are selected from each candidate expert cluster to form a candidate expert model set. Based on the candidate expert model set and the current system resource status, expert model cache management and scheduling are performed to obtain an execution configuration plan for the expert models. This execution configuration plan specifies the expert models to be executed and their target devices. Based on the execution configuration plan, inference execution is performed on each expert model to be executed, asynchronously collecting the inference output results and performance indicators of each expert model to obtain a result output set for each expert model. The result output sets are then fused according to the weights of each expert model to generate the final fused inference result of the carbon data. This application preprocesses the raw carbon-related data to obtain carbon feature vectors with a unified format, which are compatible with multiple modal inputs and can extract key information of the task. It uses hierarchical routing to ensure efficient selection of expert clusters. The weighted fusion of multiple expert models ensures the high accuracy and robustness of the fusion results. Finally, it enables the rapid adjustment of routing strategies and model priorities under different loads and scenarios, which significantly improves the system's inference accuracy, reduces response latency and resource consumption.

[0044] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application are realized and obtained through the structures particularly pointed out in the description, claims and drawings.

[0045] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0047] Figure 1 A flowchart illustrating an embodiment of the reasoning method based on a hierarchical hybrid expert model provided in this application;

[0048] Figure 2 A flowchart illustrating another embodiment of the inference method based on a hierarchical hybrid expert model provided in this application;

[0049] Figure 3 A schematic diagram of a reasoning system based on a hierarchical hybrid expert model provided in an embodiment of this application;

[0050] Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0053] For ease of understanding, the specific process of the embodiments of this application is described below. The embodiments of this application are applied to system devices such as computers and servers. Please refer to [link / reference]. Figure 1An embodiment of the reasoning method based on a hierarchical hybrid expert model in this application specifically includes the following steps:

[0054] S110. Preprocess the raw carbon-related data to generate carbon feature vectors in a unified format;

[0055] In this step, the raw carbon-related data can be image, text, or audio data, specifically including industrial emissions data, transportation energy consumption data, building energy consumption data, and remote sensing image data. The system first determines the modal type of the carbon-related data and adopts the following preprocessing methods for different types:

[0056] Image data: Calculate the covariance matrix of its pixel matrix, and perform eigenvalue decomposition through principal component analysis (PCA). Select the carbon feature vectors corresponding to the top k largest eigenvalues ​​to form the low-dimensional semantic principal component vector of the image.

[0057] Text data: Input into a pre-trained language model such as BERT or RoBERTa, extract the context vector of each token, and generate a fixed-length sentence vector through pooling (average pooling or CLS token);

[0058] Speech data: The temporal features of the sound are extracted by Mel-frequency cepstral coefficients (MFCC) and then fed into an autoencoder structure for encoding and compression to obtain a low-dimensional representation vector of the speech signal. If it is a multimodal input, such as text-image hybrid, the above dimensionality reduction processing is performed on each modal data separately. Then, a concatenation mechanism or a cross-modal attention mechanism (such as CrossAttention) is used to align the modal information and fuse it into a unified carbon feature vector X.

[0059] For industrial emission data, key emission factors are extracted and standardized; for transportation energy consumption data, vehicle emission feature vectors are constructed based on spatiotemporal trajectories; for building energy consumption data, energy density indicators are extracted according to building type and climate zone; for remote sensing image data, surface temperature and vegetation index are extracted and principal component dimensionality reduction is performed.

[0060] S120. Perform multi-level expert routing based on carbon feature vectors and construct a candidate expert set;

[0061] Specifically, using the first-layer router, the most relevant top M candidate expert clusters are selected from multiple expert clusters in a tree-structured hierarchical structure based on the carbon feature vector; using the second-layer router, attention calculation is performed based on the carbon feature vector and the expert models in each candidate expert cluster, and the corresponding expert scores are output. In each candidate expert cluster, the top K expert models with the highest scores and their weights are selected to form a candidate expert model set.

[0062] In practical applications, the first-layer router uses a multilayer perceptron (MLP) to map the carbon feature vector X, outputting a scoring vector S=MLP(X) corresponding to each expert cluster. After normalization using Softmax, the top M expert clusters with the highest scores are selected as the candidate set. This process achieves coarse-grained semantic attribution determination for the task. For each candidate expert cluster, the second-layer router introduces the structural embedding vector ei of the expert model, concatenating it with the input feature X to form a composite vector zi = concat(X,ei). The composite vector zi is input into a lightweight attention module (which can be a 1-2 layer Transformer or Gated Attention mechanism) to calculate the attention score of each expert model. The expert scores within each cluster are normalized using the Softmax function, and the top K expert models in each cluster are selected, with their corresponding weights recorded, forming the final candidate expert model set E.

[0063] S130. Perform expert model cache management and scheduling based on the candidate expert model set and the current system resource status to obtain the expert model execution configuration plan;

[0064] It should be noted that the execution configuration plan is used to specify the expert model to be executed and its target device; based on the candidate expert set E generated in step S120, the inference system evaluates the historical call frequency and recent performance indicators of each expert model to determine its cache status:

[0065] If it is a high-frequency hot model, it will be directly stored in the GPU memory or memory to be executed;

[0066] If it is a low-frequency cold model and is not currently resident, then the asynchronous loading process is triggered.

[0067] Furthermore, the inference system collects the current computing resource load through the resource awareness module, including CPU core utilization, GPU memory usage, and NPU computing power peak.

[0068] Next, based on the computational graph complexity and resource requirement level of the expert model, the candidate expert set E is matched with the current resource status, and a scheduling plan is generated using a greedy or heuristic strategy.

[0069] The execution configuration plan P is a list of scheduled tasks, which includes: expert model ID, target execution device ID, input binding feature X, output buffer identifier, etc.

[0070] The inference system submits the execution configuration plan P to the scheduling service, which then distributes the tasks to heterogeneous devices for execution based on task priority and availability.

[0071] S140. Based on the execution configuration plan, perform inference execution on each expert model to be executed, asynchronously collect the inference output results and running performance indicators of each expert model, and obtain the result output set of each expert model.

[0072] In this embodiment, after receiving the scheduling instruction, each expert model calls the inference engine (such as ONNX Runtime, TensorRT, or TorchScript) to complete the inference task, as specifically implemented below:

[0073] The bound carbon feature vector X is converted into a model-compatible tensor structure and uploaded to the specified device memory;

[0074] Start the inference thread and return the processing handle to the scheduling module;

[0075] The system listens for the expert model's return status in non-blocking mode and detects task completion events;

[0076] Once the task is completed, the prediction result Yi of the corresponding expert model is retrieved from the output buffer.

[0077] Simultaneously, performance metrics such as execution time, memory usage, and core utilization of the current inference process are collected.

[0078] The result Yi and its corresponding performance metrics are packaged and stored in the central result cache pool to form the output set {Yi}.

[0079] S150. Perform fusion calculations on the result output set according to the weights of each expert model to generate the final fusion inference result.

[0080] Specifically, based on the score weights of the candidate expert models, the results in the set {Yi} are weighted: F = ∑αi × Yi, where αi is the normalized weight obtained from the second-layer router, and Yi is the expert model output. The system can further incorporate a consistency measurement module to determine the degree of consistency between the outputs of multiple expert models. If a model is found to be significantly abnormal (e.g., significantly diverging from other expert predictions), its fusion weights can be dynamically adjusted. When higher fusion reliability is required, the system can enable a gated fusion mechanism or an attention fusion network (such as Meta-Fusion or Gated Sum) to achieve dynamic weighting and control of the output result F. Finally, F serves as the output of this inference task, available for use by subsequent modules or returned to the user caller.

[0081] The above technical solutions achieve the following: A unified carbon feature vectorization preprocessing method is compatible with multiple modal inputs and extracts key task information; hierarchical routing utilizes a coarse-to-fine two-level learnable router, ensuring efficient selection of expert clusters while precisely selecting the set of expert models with the highest weights through a lightweight attention network; expert model caching and heterogeneous parallel scheduling significantly reduce GPU / memory usage and scheduling latency while maintaining high availability; asynchronous inference and real-time performance metric acquisition provide accurate data for dynamic feedback; and multi-expert weighted fusion ensures high accuracy and robustness of the fusion results. The overall inference process is closed-loop adaptive, and combined with subsequent reinforcement learning optimization, it can quickly adjust routing strategies and model priorities under different loads and scenarios, significantly improving system inference accuracy and reducing response latency and resource consumption.

[0082] like Figure 2 The illustration shows another implementation of the reasoning method based on a hierarchical hybrid expert model provided in this application, specifically including the following steps:

[0083] S210, carbon-related data preprocessing to generate carbon feature vectors in a unified format;

[0084] This preprocessing to generate a unified carbon feature vector refers to converting different modal inputs into a standardized representation. Specifically, principal component analysis can be used to process image data, a pre-trained language model can be used to extract text features, an autoencoder can be used to compress speech features, and a cross-modal attention mechanism can be used to fuse multimodal carbon-related data to achieve compatible processing of heterogeneous inputs.

[0085] In this embodiment, the raw carbon-related data includes image data, text data, and voice data, such as industrial emissions data, transportation energy consumption data, building energy consumption data, and remote sensing image data. The preprocessing of the raw carbon-related data to generate a unified format carbon feature vector specifically involves:

[0086] For image data, calculate the covariance of its pixel matrix and perform eigenvalue decomposition, then select principal component vectors to map the image to low-dimensional features;

[0087] For text data, sentence vectors are generated using a pre-trained language model;

[0088] For speech data, MFCC features are extracted and compressed into a low-dimensional representation using an autoencoder;

[0089] In the presence of multimodal carbon-related data, carbon feature vectors from various modes are merged into a unified carbon feature vector through concatenation or cross-modal attention.

[0090] It should be noted that pixel matrix covariance refers to constructing a covariance matrix by statistically analyzing the correlation between image pixels. Specifically, a sliding window can be used to traverse the pixels and calculate the covariance value, which is used to capture the spatial distribution characteristics of image texture.

[0091] Principal component vectors refer to the carbon eigenvectors corresponding to the largest eigenvalue after eigenvalue decomposition of the covariance matrix. Specifically, singular value decomposition algorithms can be used to reduce image dimensionality while retaining key information.

[0092] Pre-trained language models refer to neural networks that have been trained on large-scale corpora and possess semantic understanding capabilities. Specifically, BERT or RoBERTa models can be used to generate sentence vectors to capture the deep semantic features of text.

[0093] MFCC features refer to the spectral characteristics of speech signals extracted through Mel frequency cepstral coefficients. Specifically, they can be achieved by frame windowing, Fourier transform, and Mel filter bank processing to characterize the acoustic features of speech.

[0094] An autoencoder is a neural network structure that compresses carbon-related data through an encoder and then reconstructs it through a decoder. Specifically, it can be constructed by combining convolutional layers and fully connected layers to reduce the dimensionality of high-dimensional MFCC features to a compact representation.

[0095] Cross-modal attention refers to calculating the correlation between features of different modalities through an attention mechanism. Specifically, it can be implemented using a multi-head attention layer to dynamically fuse complementary information from multimodal carbon-related data.

[0096] In this embodiment, when the original carbon-related data is image data, the covariance of the pixel matrix is ​​first calculated to capture spatial correlation. Then, principal component vectors are selected through eigenvalue decomposition, mapping the original high-dimensional pixel data to a low-dimensional feature space. For example, for an RGB image of size 224×224, a 3×3 covariance matrix is ​​obtained after covariance calculation, and the first 128 dimensions of principal component vectors are selected after decomposition.

[0097] For text data, a pre-trained language model is used to directly extract sentence vectors. For example, after inputting the input text into the BERT model, the 768-dimensional vector corresponding to the [CLS] label is taken as the feature representation.

[0098] For speech data, MFCC features are extracted after frame segmentation to form a time-frequency two-dimensional matrix. Spatial compression is performed through the convolutional layer of an autoencoder to output a 128-dimensional low-dimensional vector.

[0099] When there is multimodal input, such as images and text, the carbon feature vectors of the two modalities are concatenated and input into the cross-modal attention layer. By calculating the interaction weights of image features and text features, a fused unified carbon feature vector is generated.

[0100] This paper adopts a hierarchical preprocessing architecture, designs and optimizes feature extraction methods for different data types, and combines cross-modal attention mechanism to achieve efficient fusion of multi-source data, which significantly reduces the computational complexity of heterogeneous data processing.

[0101] S220. Using the first-layer router, select the top M most relevant candidate expert clusters from multiple expert clusters in a tree-structured hierarchical structure based on the carbon feature vectors.

[0102] In this embodiment, the first-layer router is a multilayer perceptron network. Using the first-layer router, the most relevant top M candidate expert clusters are selected from multiple expert clusters in a tree-structured hierarchical structure based on carbon feature vectors. This includes: inputting the carbon feature vectors into the multilayer perceptron network, evaluating the matching degree of each expert cluster in the tree-structured hierarchical structure to obtain expert cluster scores; and after performing Softmax normalization on the scores of each expert cluster, selecting the top M candidate expert clusters with the highest expert cluster scores.

[0103] It should be noted that a multilayer perceptron network refers to a neural network structure consisting of fully connected layers and nonlinear activation functions. Specifically, it can be implemented by combining the ReLU activation function with a batch normalization layer, which is used to map the input carbon feature vector to the expert cluster matching degree space.

[0104] Tree-structured expert clusters refer to hierarchical expert model groupings formed by domain or function division. Specifically, they can be constructed using knowledge graphs or clustering algorithms, and the hierarchical structure reduces the complexity of routing calculations.

[0105] Matching evaluation refers to calculating the degree of association between input features and each expert cluster. Specifically, it can be represented by the scalar value output by the multilayer perceptron network, used to quantify the fit between the expert cluster and the current task. Softmax normalization refers to converting the expert cluster scores into a probability distribution, which can be calculated using an exponential function and a normalized denominator, ensuring that the selection of candidate expert clusters has probabilistic interpretability.

[0106] In this embodiment, after the input carbon feature vector enters the multilayer perceptron network, the network generates initial matching scores for each expert cluster through forward propagation. Each expert cluster in the tree hierarchy corresponds to a score, which reflects the relevance of the current carbon-related data within the task domain represented by that expert cluster. Softmax normalization transforms the original scores into a probability distribution, ensuring that the selection process for the top M candidate expert clusters has a clear probabilistic basis. For example, when processing image classification tasks, the multilayer perceptron network may assign higher scores to clusters containing image recognition experts, while assigning lower scores to text processing expert clusters, thereby ensuring that routing decisions are highly aligned with task requirements.

[0107] This step, by introducing a tree-like hierarchical structure and working in conjunction with a multilayer perceptron network, decomposes routing decisions into a hierarchical selection process, significantly reducing computational overhead. The tree-like hierarchical structure transforms global routing decisions into hierarchical local selections, avoiding routing bottlenecks in high-concurrency scenarios. Simultaneously, the nonlinear mapping capability of the multilayer perceptron network ensures the accuracy of expert cluster matching evaluation, enabling the system to quickly identify the subset of experts most relevant to carbon-related data.

[0108] S230. Using the second-layer router, the embedded representation of the corresponding expert model is combined within each candidate expert cluster. Attention calculation is performed on the concatenated input of carbon feature vector and expert model description vector, and the corresponding expert score is output. The top K expert models with the highest scores and their weights are selected in each candidate expert cluster to form a candidate expert model set.

[0109] The second-layer router is a lightweight attention network. When constructing a candidate expert model set using the second-layer router, the carbon feature vector is concatenated with the description vector of each expert model in the candidate expert cluster to form a composite input. Multi-head self-attention or Trans-form calculation is performed on the composite input to obtain the expert score. After normalizing the expert score using Softmax, the top K expert models with the highest expert scores in each candidate expert cluster and their corresponding weights are selected, and a candidate expert model set is constructed based on the top K expert models.

[0110] It should be noted that lightweight attention networks refer to attention mechanism networks with fewer parameters and controllable computational complexity. Specifically, they can be implemented by reducing the number of attention heads or compressing the dimensions of hidden layers. Their role is to balance computational overhead with the matching accuracy of expert models.

[0111] Composite input refers to the joint input formed by concatenating the input carbon feature vector with the expert model description vector. Specifically, it can be achieved by vector concatenation or projection fusion. Its purpose is to enable attention computation to simultaneously consider the characteristics of carbon-related data and the capabilities of the expert model.

[0112] Multi-head self-attention refers to decomposing the input into multiple subspaces and calculating attention weights for each. Specifically, it can be implemented using parallel linear transformation and scaled dot product attention mechanisms. Its role is to capture the multidimensional relationship between the input and the expert model.

[0113] Softmax normalization refers to converting the original score into a probability distribution, which can be achieved by exponential operation and normalization denominator calculation. Its purpose is to ensure the rationality of the weight allocation of the expert model.

[0114] In this embodiment, when selecting expert models within a candidate expert cluster, the input carbon feature vector is first concatenated with the expert model's description vector to form a composite input containing both input characteristics and expert capabilities. This composite input is fed into a lightweight attention network for multi-head self-attention computation, generating expert scores through feature interactions in parallel subspaces. After score normalization, the top K expert models with the highest scores in each expert cluster are selected, and a candidate set is constructed based on their weights. For example, when the input is image data, the carbon feature vector may contain visual semantic information, while the expert description vector encodes the model's ability to process image classification or object detection; attention computation can effectively match the correlation between the two. As a preferred implementation, the positional encoding in the Transform computation can be replaced with the expert model's registration order encoding to reduce computational complexity.

[0115] In practical applications, lightweight attention networks have a dynamic adaptive adjustment mechanism to dynamically adjust the weight distribution in attention calculation based on historical inference accuracy, expert model selection entropy, and the confidence requirements of the current inference task, so as to improve the discriminative ability of the candidate expert model set.

[0116] The dynamic adaptive adjustment mechanism refers to an algorithmic architecture that adjusts the attention weight distribution in real time through a feedback control module. Specifically, it can be implemented by combining historical data statistics based on a sliding window with an online optimizer, periodically updating the weight adjustment parameters to adapt to dynamic task requirements. Historical inference accuracy refers to the prediction accuracy of the expert model in historical tasks. It can be calculated by recording the matching degree between the output results of each expert model and the true labels, reflecting the reliability of the expert model. Expert model selection entropy refers to the degree of disorder in the probability distribution of expert models being selected during the routing process. It can be calculated using the information entropy formula and is used to measure the diversity of routing decisions. The confidence requirement of the current inference task refers to the required level of reliability of the output results. It can be quantified through preset thresholds or task type labels, used to control the concentration of attention weight distribution.

[0117] In this embodiment, during the operation of the lightweight attention network, the dynamic adaptive adjustment mechanism works synergistically through three parameters: First, it collects historical inference accuracy data of each expert model to construct an expert credibility evaluation matrix; second, it calculates the information entropy of the probability of expert models being selected in the most recent routing choices to determine the diversity level of the current selection strategy; and finally, it combines the confidence requirement parameter in the task configuration file to generate attention weight adjustment coefficients. These three types of parameters are input to the attention calculation layer in a linear weighted manner, dynamically changing the similarity calculation bias value between the query vector and the key vector in the attention head. This ensures that in task scenarios requiring high confidence, the attention mechanism tends to select expert models with stable historical performance, while in scenarios requiring exploratory inference, it enhances the diversity of model selection by increasing the selection entropy weight.

[0118] This lightweight design reduces computational resource consumption while ensuring the quality of routing decisions, enabling the system to maintain stable performance even in high-concurrency scenarios. Simultaneously, the introduction of an attention mechanism enhances the model's adaptability to different input characteristics, thereby achieving better expert combination selection in diverse tasks.

[0119] S240, Perform expert model cache management and scheduling, and generate expert model execution configuration plan;

[0120] Specifically, the most frequently invoked candidate expert models in the candidate expert model set are stored in the cache; candidate expert models that have not been selected for N consecutive rounds are marked as cold models and unloaded from the video memory. When a cold model is selected again in a future task, it is restored to an available state through an asynchronous loading mechanism; based on the current system resource status, the target device for executing the task is selected, and each candidate expert model is matched to the target device using a parallel asynchronous scheduling method to obtain the execution configuration plan of the expert model.

[0121] It's important to note that a cache refers to a storage area employing an LRU or LFU replacement strategy, specifically implemented using GPU memory or high-speed SSDs, to reduce loading latency for frequently accessed models. A cold model refers to an expert model that hasn't been invoked within a certain time window; this can be achieved using reference counters and timestamps to release resources occupied by idle models. An asynchronous loading mechanism means that when a background thread detects a cold model being reselected, it automatically restores the model to a usable state from disk or remote storage. This can be implemented using a combination of preloading and lazy loading to balance resource consumption and response speed. Target devices refer to available hardware resources in a heterogeneous computing environment; specifically, a resource monitoring module can collect the real-time load status of the CPU, GPU, and NPU, and dynamically allocate resources based on a load balancing algorithm.

[0122] In this embodiment, frequently accessed models in the candidate expert model set are retained in the cache to reduce the time overhead caused by repeated loading. For cold models that have not been selected for multiple consecutive rounds, resources are released through a memory unloading operation, and their metadata is recorded for subsequent recovery. When a cold model is selected again, model recovery is completed in the background through an asynchronous loading mechanism to avoid blocking the current inference task. The system resource status is dynamically calculated by monitoring the resource utilization of each computing device in real time, such as the number of idle CPU cores, the remaining amount of GPU memory, and the NPU computing power utilization rate, combined with the resource requirement parameters of the expert model. The final generated execution configuration plan includes the expert model identifier, the target device number, and the input / output binding information, and is distributed to the corresponding computing nodes for execution asynchronously through the task scheduler.

[0123] Furthermore, the expert model cache management also includes constructing a priority scoring function based on the inference latency and accuracy of the expert model. When system resources are limited, the cold model with the lowest score is eliminated first to ensure that the high-performance model gets priority scheduling rights.

[0124] The priority scoring function is a comprehensive indicator that quantifies model performance and resource consumption. Specifically, it can be implemented by using a weighted summation method to convert inference latency and accuracy into a single score value; for example, the reciprocal of the latency metric can be added proportionally to the accuracy. This function is used to dynamically evaluate the model's priority in resource-constrained scenarios, thereby optimizing cache space allocation. "Cold models" refer to candidate expert models that have not been selected in multiple consecutive rounds. This can be implemented by using a counter to record the number of times a model has not been invoked; when the number exceeds a set threshold, an unloading operation is triggered. This mechanism effectively releases idle resources and avoids memory waste.

[0125] During system operation, the inference latency and historical accuracy of each expert model are monitored in real time and input into a priority scoring function for calculation. The scoring results reflect the overall performance of the model in the current task scenario. When GPU memory or computing resources are insufficient, the system prioritizes retaining models with higher scores based on the scoring ranking, while removing the lowest-scoring "cold" models from GPU memory. For example, when GPU memory usage reaches a preset threshold, the system automatically triggers a resource reclamation process, determining the unloading targets by comparing the score values ​​of the "cold" models. Simultaneously, the metadata of the unloaded models remains on disk and can be quickly restored via an asynchronous loading mechanism when needed by subsequent tasks.

[0126] In another feasible implementation, the execution configuration plan includes recording the expert model ID, the assigned inference device ID, the binding information of the input carbon feature vector, and the output cache identifier; the inference device ID is dynamically selected based on the heterogeneous computing devices available in the current system and is asynchronously distributed through the task scheduler.

[0127] The expert model ID is a unique encoded sequence that identifies each model in the candidate expert model set. It can be implemented using a hash value or UUID and is used to quickly locate and call the target model in a distributed system. The inference device ID is a number that identifies the heterogeneous computing device. It can be obtained through the device registry or resource management module and is used to map expert models to physical computing resources. The binding information for the input carbon feature vector refers to the association between the preprocessed carbon feature vector and the expert model input interface, which can be implemented using tensor shape matching or data encapsulation to ensure compatibility between carbon-related data and the model interface. The output cache identifier is a cache address marker used to store inference results. It can be implemented using memory pointers or distributed storage paths to facilitate asynchronous collection and subsequent result fusion. Dynamic selection of heterogeneous computing devices refers to allocating appropriate computing units to the expert model based on real-time resource status. This can be achieved by collecting CPU, GPU, and NPU load data through the resource monitoring module and dynamically matching it with the computing power requirements of the expert model. Asynchronous task dispatch refers to transmitting the execution configuration plan to the target device in a non-blocking manner. This can be achieved using message queues or remote procedure calls, reducing the latency caused by task dispatch.

[0128] When generating the execution configuration plan, the real-time status data of each computing device in the current system is first obtained through the resource monitoring module, such as the number of idle CPU cores, the remaining capacity of GPU memory, and the computing power utilization of NPU. Then, based on the resource requirement parameters predefined during expert model registration, such as memory usage and computational complexity, a matching degree calculation is performed in conjunction with the device status data. For example, for image processing models requiring high memory usage, GPU devices with sufficient remaining memory are prioritized; for speech recognition models with low latency requirements, NPU devices with idle computing power are prioritized. After completing device matching, the expert model ID is bound to the target device ID, and the input carbon feature vector is converted to a data format compatible with the target model's input interface, such as encapsulating the text carbon feature vector into a tensor of a specific dimension. The output cache identifier generates a unique path based on the task ID and device ID, ensuring that the inference results of different tasks do not overwrite each other. Finally, the task scheduler sends the configuration plan to the target device through an asynchronous communication channel. Upon receiving the plan, the device immediately loads the model and starts the inference session without waiting for other tasks to complete scheduling.

[0129] In some specific implementations, expert model scheduling is performed based on the candidate expert model set E and the current system resource state, including:

[0130] Detect the resource status of various types of computing devices in the current system, including the number of idle CPU cores, GPU memory usage, and NPU computing power utilization.

[0131] Based on the resource requirement parameters of each expert model and the resource status, a mapping rule is established to calculate the optimal deployment device for each expert model.

[0132] Based on the deployment device selection results, construct an expert model execution configuration plan P, which includes expert model identifier, target device number, and input / output binding structure;

[0133] The system scheduling module is invoked to asynchronously distribute the execution configuration plan P to the corresponding computing nodes.

[0134] S250. Based on the execution configuration plan, perform inference execution on each expert model to be executed, asynchronously collect the inference output results and running performance indicators of each expert model, and obtain the result output set of each expert model.

[0135] In this step, the carbon feature vector is format-adapted and bound to the input interface of the expert model to be executed as specified in the execution configuration plan to generate a standardized input tensor; the standardized input tensor is distributed to the device where the corresponding expert model to be executed is located, and the inference session is started; during the execution of each expert model to be executed, the presence of an inference completion signal is monitored asynchronously; after the inference completion signal is detected, the inference output results of each expert model are immediately extracted, and the running performance indicators are recorded synchronously, including inference time, memory usage, and hardware utilization; the inference output results and performance indicators are stored in the result cache pool to form the result output set of the expert model.

[0136] The standardized input tensor refers to converting carbon feature vectors into a unified data format supported by the input interface of the expert model to be executed. This can be achieved by adjusting tensor dimensions or normalizing them, for example, by adding padding layers or using interpolation algorithms to adjust feature dimensions, ensuring compatibility between carbon-related data and the interfaces of different expert models. The asynchronous monitoring method refers to a non-blocking monitoring mechanism, which can be implemented using an event-driven architecture or callback functions. For example, it can monitor the inference task status in real time through message queues or semaphores, avoiding wasted computational resources due to polling checks. The result cache pool is a shared memory area used to temporarily store inference outputs. This can be implemented using a distributed key-value database or memory-mapped files, such as using Redis or Memcached to build a cache pool, supporting multi-threaded concurrent access and fast data retrieval.

[0137] During the inference execution phase, the carbon feature vectors are first format-adapted to the input interface of the target expert model. For example, the text carbon feature vectors are converted into image tensor input formats for visual models, or the dimensions are adjusted through interpolation to match the input requirements of different models. The standardized input tensors are distributed to heterogeneous computing devices through a task scheduler, such as allocating computationally intensive models to GPUs and memory-sensitive models to NPUs. After the inference session starts, the system continuously monitors the task status of each device through an asynchronous listening mechanism. For example, when the GPU completes computation, an interrupt signal is triggered, the output results are immediately extracted, and performance data is recorded. Performance metrics are collected synchronously and stored in a cache pool. For example, inference time and GPU memory usage are written to a time-series database to provide data support for subsequent model scheduling optimization.

[0138] In some specific implementations, the generation of standardized input tensors can be dynamically adjusted based on the expert model's metadata information, such as automatically selecting the adapter module according to the input format requirements declared during model registration. The asynchronous monitoring mechanism can employ multiplexing techniques to monitor multiple device ports simultaneously, for example, using epoll or kqueue system calls to efficiently manage large-scale concurrent tasks. The result cache pool can implement data partitioning strategies, such as storing data in shards according to the expert model ID hash, to avoid a single node becoming a performance bottleneck.

[0139] S260. Perform fusion calculations on the result output set according to the weights of each expert model to generate the final fusion inference result of carbon data;

[0140] In this step, the result output set is weighted and summed based on the weights of each expert model and its output confidence level to generate the final fused carbon emission results, carbon intensity index and carbon trend prediction results.

[0141] In this context, weights refer to the relative importance of each expert model in the fusion process. Specifically, they can be determined by calculating the attention score of the second-layer router, for example, using the softmax-normalized score as the weight parameter. Output confidence refers to the expert model's certainty assessment of its own inference results. Specifically, it can be calculated using the probability distribution entropy of the model's output or the maximum class probability value; for example, it can be calculated by using the probability value of the predicted class in a classification task as the confidence score. Weighted summation refers to multiplying the inference results of each expert model by its corresponding weight and confidence score, and then summing the results. This can be achieved using matrix multiplication or element-wise multiplication followed by summation.

[0142] After the inference execution of each expert model is completed, the collected output set contains the outputs of multiple expert models and their corresponding weights. The confidence score of each expert model's output is calculated based on its own inference results; for example, in an image classification task, the maximum class probability output by the model can be used as the confidence score. Subsequently, the output of each expert model is multiplied by the product of its weight and confidence score to obtain a weighted result. All weighted results are accumulated to generate the final fused inference result. For example, in a text sentiment analysis task, if two expert models output positive probabilities of 0.8 and 0.7 respectively, with weights of 0.6 and 0.4 and confidence scores of 0.9 and 0.85 respectively, the final fused result is 0.8 × 0.6 × 0.9 + 0.7 × 0.4 × 0.85. This process dynamically combines model weights and confidence scores, allowing expert models with high confidence and high weights to have a greater impact on the final result.

[0143] S270. Obtain the error between the fusion result and the true label, and construct a reward function by combining the inference delay and the distribution entropy selected by the expert model.

[0144] S280. Based on the reward function, the parameters of the first-layer router and the second-layer router are updated and optimized using a reinforcement learning algorithm.

[0145] In this step, the reinforcement learning algorithm is PPO (Proximal Policy Optimization), and the reward function R is in the following form: R = α × (prediction accuracy) − β × (average inference delay) − γ × (expert choice entropy), where α, β, and γ are empirical adjustment coefficients used to balance accuracy and resource overhead.

[0146] Furthermore, after each round of training, the updated parameters of the first and second layer routers are stored in the policy parameter server, and offline regression tests are triggered periodically to prevent policy degradation.

[0147] In summary, the system preprocesses raw carbon-related data to generate carbon feature vectors in a unified format; a first-layer router selects candidate clusters from a hierarchical tree-structured expert cluster; a second-layer router performs attention calculations using expert model embedding representations, selecting the expert model with the highest score; cache management and scheduling are implemented based on system resource status to generate an execution configuration plan; expert model inference is executed asynchronously, and results are collected; the result set is weighted and fused to generate the final output. This approach effectively coordinates hierarchical expert models with heterogeneous computing resources, reducing response latency while ensuring inference accuracy. A unified processing mechanism for multimodal inputs improves system compatibility, a dynamic resource scheduling strategy optimizes hardware utilization, hierarchical routing design enhances the accuracy of expert selection, an asynchronous execution mechanism accelerates task processing, and a weighted fusion method ensures the reliability of the final result.

[0148] For the corresponding method embodiments described above, see [link to relevant documentation]. Figure 3 The diagram shown illustrates a reasoning system based on a hierarchical hybrid expert model. The system includes:

[0149] The input processing module 310 is used to preprocess the raw carbon-related data to generate carbon feature vectors in a unified format.

[0150] The routing decision module 320 is used to select the top M most relevant candidate expert clusters from multiple expert clusters in a tree-structured hierarchical structure based on the carbon feature vector using a first-layer router; and to perform attention calculation on the concatenated input of the carbon feature vector and the expert model description vector within each candidate expert cluster by combining the embedded representation of the corresponding expert model with the second-layer router, outputting the corresponding expert score, and selecting the top K expert models with the highest scores and their weights in each candidate expert cluster to form a candidate expert model set.

[0151] The expert model library 330 is used to perform expert model cache management and scheduling based on the candidate expert model set and the current system resource status to obtain an execution configuration plan for the expert model, wherein the execution configuration plan is used to specify the expert model to be executed and its target device;

[0152] The fusion scheduling module 340 is used to perform inference execution on each expert model to be executed based on the execution configuration plan, asynchronously collect the inference output results and running performance indicators of each expert model, and obtain the result output set of each expert model;

[0153] The inference module 350 is used to perform fusion calculations on the result output set according to the weights of each expert model to generate the final fusion inference result of carbon data.

[0154] Optionally, the input processing module 310 is specifically used for:

[0155] For image data, calculate the covariance of its pixel matrix and perform eigenvalue decomposition, then select principal component vectors to map the image to low-dimensional features;

[0156] For text data, a pre-trained language model is used to generate sentence vectors. For speech data, MFCC features are extracted and compressed into a low-dimensional representation using an autoencoder.

[0157] In the presence of multimodal carbon-related data, carbon feature vectors from various modes are merged into a unified carbon feature vector through concatenation or cross-modal attention.

[0158] Optionally, the first-layer router is a multilayer sensor network; the routing decision module 320 is specifically used for:

[0159] The carbon feature vector is input into the multilayer perceptron network to evaluate the matching degree of each expert cluster in the tree-structured hierarchical network and obtain the expert cluster score.

[0160] After performing Softmax normalization on the scores of each expert cluster, the top M candidate expert clusters with the highest scores are selected.

[0161] Optionally, the second-layer router is a lightweight attention network; the routing decision module 320 is specifically used for:

[0162] The carbon feature vector is concatenated with the description vectors of each expert model in the candidate expert cluster to form a composite input. Multi-head self-attention or Trans former calculation is performed on the composite input to obtain the expert score.

[0163] After normalizing the expert scores using Softmax, the top K expert models with the highest expert scores in each candidate expert cluster and their corresponding weights are selected, and a candidate expert model set is formed based on the top K expert models.

[0164] Optionally, the expert module library 330 is specifically used for:

[0165] The most frequently invoked candidate expert models in the candidate expert model set are kept in the cache.

[0166] Candidate expert models that have not been selected for N consecutive rounds in the candidate expert model set are marked as cold models and unloaded from video memory. When a cold model is selected again in a future task, it is restored to a usable state through an asynchronous loading mechanism.

[0167] Based on the current system resource status, the target device for the task is selected, and the candidate expert models are matched to the target device using a parallel asynchronous scheduling method to obtain the execution configuration plan of the expert model.

[0168] Optionally, the execution configuration plan includes: recording the expert model ID, the assigned inference device ID, the binding information of the input carbon feature vector, and the output cache identifier; the inference device ID is dynamically selected based on the heterogeneous computing devices available in the current system and is asynchronously distributed through the task scheduler.

[0169] Optionally, the fusion scheduling module 340 is specifically used for:

[0170] The carbon feature vector is format-adapted and bound to the input interface of the expert model to be executed specified in the execution configuration plan to generate a standardized input tensor;

[0171] The standardized input tensors are distributed to the devices containing the corresponding expert models to be executed, and an inference session is initiated.

[0172] During the execution of each expert model to be executed, an asynchronous listening method is used to detect whether an inference completion signal exists;

[0173] After the inference completion signal is detected, the inference output results of each expert model are immediately extracted and the running performance indicators are recorded simultaneously. The running performance indicators include inference time, memory usage, hardware utilization and carbon index calculation efficiency.

[0174] The inference output and the performance metrics are stored in the result cache pool to form the result output set of the expert model.

[0175] Optionally, the inference module 350 is specifically used for:

[0176] Based on the weights of each expert model and its output confidence, the result output set is weighted and summed to generate the final fused carbon emission results, carbon intensity index, and carbon trend prediction.

[0177] The system provided in the above embodiments is compatible with multiple modal inputs and extracts key task information through a unified carbon feature vectorization preprocessing method. Hierarchical routing utilizes a coarse-to-fine two-level learnable router, ensuring efficient selection of expert clusters while accurately selecting the set of expert models with the highest weights through a lightweight attention network. Expert model caching and heterogeneous parallel scheduling significantly reduce GPU / memory usage and scheduling latency while maintaining high availability. Asynchronous inference and real-time performance indicator acquisition provide accurate data for dynamic feedback. Multi-expert weighted fusion ensures high accuracy and robustness of the fusion results. The overall inference process is closed-loop adaptive, and combined with subsequent reinforcement learning optimization, it can quickly adjust routing strategies and model priorities under different loads and scenarios, significantly improving system inference accuracy and reducing response latency and resource consumption.

[0178] This embodiment also provides an electronic device, including a processor and a memory. The memory stores machine-executable instructions that can be executed by the processor. The processor executes the machine-executable instructions to implement the above-described reasoning method based on a hierarchical hybrid expert model. This electronic device can be a server or a terminal device.

[0179] See Figure 4 As shown, the electronic device includes a processor 400 and a memory 401. The memory 401 stores machine-executable instructions that can be executed by the processor 400. The processor 400 executes the machine-executable instructions to implement the above-described reasoning method based on a hierarchical hybrid expert model.

[0180] Furthermore, Figure 4The electronic device shown also includes a bus 402 and a communication interface 403. The processor 400, the communication interface 403 and the memory 401 are connected via the bus 402.

[0181] The memory 401 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 403 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The bus 402 may be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0182] The processor 400 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 400 or by instructions in software form. The processor 400 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 401. The processor 400 reads the information in memory 401 and, in conjunction with its hardware, completes the steps of the hierarchical hybrid expert model-based reasoning method described in the aforementioned embodiments.

[0183] This embodiment also provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are invoked and executed by a processor, the computer-executable instructions cause the processor to implement the above-described reasoning method based on a hierarchical hybrid expert model.

[0184] The computer program product of the reasoning method, apparatus, electronic device and storage medium based on hierarchical hybrid expert model provided in this application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0185] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0186] Furthermore, in the description of the embodiments of this application, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0187] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0188] In the description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0189] Finally, it should be noted that the above embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the technical scope disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A reasoning method based on a hierarchical hybrid expert model, applied to a reasoning system, characterized in that, The method includes: The raw carbon-related data is preprocessed to generate carbon feature vectors in a unified format; The carbon feature vector is mapped using a first-layer router, and a score corresponding to each expert cluster is output. Based on the score, the top M candidate expert clusters with the highest scores are selected from multiple expert clusters in a tree-structured hierarchical structure. The carbon feature vector and the expert model description vector are concatenated using a second-layer router within each candidate expert cluster by combining the embedded representation of the corresponding expert model. Attention calculation is performed on the concatenated vector, and the corresponding expert score is output. The top K expert models with the highest scores and their weights are selected from each candidate expert cluster to form a candidate expert model set. Based on the candidate expert model set and the current system resource status, expert model cache management and scheduling are performed to obtain an expert model execution configuration plan, wherein the execution configuration plan is used to specify the expert model to be executed and its target device; Based on the execution configuration plan, inference execution is performed on each expert model to be executed, and the inference output results and running performance indicators of each expert model are collected asynchronously to obtain the result output set of each expert model; The output set of results is fused according to the weights of each expert model to generate the final fusion inference result of carbon data; The error between the final fusion inference result and the true label is calculated, and a reward function is constructed by combining the inference delay and the expert model selection distribution entropy. The formula of the reward function is R=α×(prediction accuracy)−β×(average inference delay)−γ×(expert selection entropy), where α, β, and γ are empirical adjustment coefficients used to balance accuracy and resource overhead. Based on the reward function, the parameters of the first-layer router and the second-layer router are updated and optimized using a reinforcement learning algorithm.

2. The method according to claim 1, characterized in that, The preprocessing of the raw carbon-related data to generate carbon feature vectors in a unified format includes: Key emission factors are extracted from industrial emission data and standardized. For traffic energy consumption data, construct vehicle emission feature vectors based on spatiotemporal trajectories; Energy density indices were extracted from building energy consumption data according to building type and climate zone. For remote sensing image data, surface temperature and vegetation index are extracted and principal component dimensionality reduction is performed; In the presence of multimodal carbon-related data, carbon feature vectors from various modes are merged into a unified carbon feature vector through concatenation or cross-modal attention.

3. The method according to claim 1, characterized in that, The first-layer router is a multilayer perceptron network; the process of mapping the carbon feature vector using the first-layer router, outputting a score corresponding to each expert cluster, and selecting the top M candidate expert clusters with the highest scores from multiple expert clusters in a tree-structured hierarchical structure based on the scores includes: The carbon feature vector is input into the multilayer perceptron network to evaluate the matching degree of each expert cluster in the tree-structured hierarchical network and obtain the expert cluster score. After performing Softmax normalization on the scores of each expert cluster, the top M candidate expert clusters with the highest scores are selected.

4. The method according to claim 1, characterized in that, The second-layer router is a lightweight attention network. Within each candidate expert cluster, the second-layer router combines the embedded representation of the corresponding expert model to concatenate the carbon feature vector and the expert model description vector. Attention calculation is then performed on the concatenated vector, outputting the corresponding expert score. Finally, the top K expert models with the highest scores in each candidate expert cluster and their weights are selected to form a candidate expert model set, including: The carbon feature vector is concatenated with the description vectors of each expert model in the candidate expert cluster to form a composite input. Multi-head self-attention or Trans former calculation is performed on the composite input to obtain the expert score. After normalizing the expert scores using Softmax, the top K expert models with the highest expert scores in each candidate expert cluster and their corresponding weights are selected, and a candidate expert model set is formed based on the top K expert models.

5. The method according to any one of claims 1-4, characterized in that, The step of performing expert model cache management and scheduling based on the candidate expert model set and the current system resource status to obtain the expert model execution configuration plan includes: The most frequently invoked candidate expert models in the candidate expert model set are kept in the cache. Candidate expert models that have not been selected for N consecutive rounds in the candidate expert model set are marked as cold models and unloaded from video memory. When a cold model is selected again in a future task, it is restored to a usable state through an asynchronous loading mechanism. Based on the current system resource status, the target device for the task is selected, and the candidate expert models are matched to the target device using a parallel asynchronous scheduling method to obtain the execution configuration plan of the expert model.

6. The method according to claim 5, characterized in that, The execution configuration plan includes: recording the expert model ID, the assigned inference device ID, the binding information of the input carbon feature vector, and the output cache identifier; the inference device ID is dynamically selected based on the heterogeneous computing devices available in the current system and is asynchronously distributed through the task scheduler.

7. The method according to claim 5, characterized in that, Based on the execution configuration plan, inference execution is performed on each expert model to be executed, and the inference output results and runtime performance indicators of each expert model are collected asynchronously to obtain the result output set of each expert model, including: The carbon feature vector is format-adapted and bound to the input interface of the expert model to be executed specified in the execution configuration plan to generate a standardized input tensor; The standardized input tensors are distributed to the devices containing the corresponding expert models to be executed, and an inference session is initiated. During the execution of each expert model to be executed, an asynchronous listening method is used to detect whether an inference completion signal exists; After the inference completion signal is detected, the inference output results of each expert model are immediately extracted and the running performance indicators are recorded simultaneously. The running performance indicators include inference time, memory usage, hardware utilization and carbon index calculation efficiency. The inference output and the performance metrics are stored in the result cache pool to form the result output set of the expert model.

8. The method according to claim 5, characterized in that, The final fusion inference result of the carbon data is generated by fusing the output set of the results according to the weights of each expert model, including: Based on the weights of each expert model and its output confidence, the result output set is weighted and summed to generate the final fused carbon emission results, carbon intensity index, and carbon trend prediction.

9. A reasoning system based on a hierarchical hybrid expert model, characterized in that, The system includes: The input processing module is used to preprocess the raw carbon-related data and generate carbon feature vectors in a unified format. The routing decision module is used to map the carbon feature vector using a first-layer router, output the score corresponding to each expert cluster, and select the top M candidate expert clusters with the highest scores from multiple expert clusters in a tree-structured hierarchical structure based on the scores. The second-layer router is used to combine the embedded representation of the corresponding expert model within each candidate expert cluster, concatenate the carbon feature vector and the expert model description vector, input the concatenated vector to perform attention calculation, output the corresponding expert score, and select the top K expert models with the highest scores and their weights from each candidate expert cluster to form a candidate expert model set. An expert model library is used to perform expert model cache management and scheduling based on the candidate expert model set and the current system resource status, and to obtain an execution configuration plan for the expert model, wherein the execution configuration plan is used to specify the expert model to be executed and its target device; The fusion scheduling module is used to perform inference execution on each expert model to be executed based on the execution configuration plan, asynchronously collect the inference output results and running performance indicators of each expert model, and obtain the result output set of each expert model; The inference module is used to perform fusion operations on the output set of results according to the weights of each expert model to generate the final fused inference result of carbon data; calculate the error between the final fused inference result and the true label, and construct a reward function by combining the inference latency and the expert model selection distribution entropy; wherein the formula of the reward function is R=α×(prediction accuracy)−β×(average inference latency)−γ×(expert selection entropy), where α, β, and γ are empirical adjustment coefficients used to balance accuracy and resource overhead; based on the reward function, the parameters of the first layer router and the second layer router are updated and optimized using a reinforcement learning algorithm.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the reasoning method based on a hierarchical hybrid expert model as described in any one of claims 1-8.

Images