Hierarchical hybrid expert model-based reasoning method and system, and storage medium

Abstract

The invention provides an inference method and system based on a hierarchical hybrid expert model, and a storage medium. According to the method, through combination of tree-shaped hierarchical structure expert cluster selection, a double-layer routing mechanism and dynamic resource scheduling, the problems of calculation redundancy, unbalanced resource distribution and insufficient dynamic adjustment capability of a traditional hybrid expert model are effectively solved; the method has the advantages that the multi-carbon task reasoning efficiency is improved, the heterogeneous resource utilization rate is optimized, and the dynamic scene adaptability is enhanced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a reasoning method, system, and storage medium based on a hierarchical hybrid expert model. Background Art

[0002] As deep learning models continue to scale, single models face three core challenges when reasoning across multiple tasks or scenarios. First, general-purpose models suffer from computational redundancy when processing diverse inputs, leading to exponentially increasing resource overhead. Second, traditional architectures struggle to adapt to dynamic load scenarios, with response latency fluctuating significantly with task complexity. Third, static model combinations lack contextual awareness and are unable to dynamically adjust computational paths based on input features. While hybrid expert models alleviate some of these issues through task decomposition strategies, existing implementations suffer from three key flaws: their flat expert selection mechanism ignores the inherent hierarchical relationships of domain knowledge, resulting in a coarse semantic space partitioning; the routing decision process lacks awareness of the system's real-time state, leading to unbalanced resource allocation; and the dynamic adjustment mechanism relies solely on offline metrics, making it difficult to respond promptly to runtime changes. Particularly in highly concurrent carbon-related data scenarios, existing systems, lacking hierarchical routing and resource coordination, often experience resource waste, with popular experts overloaded while unpopular experts idle. Furthermore, the rigid cache management strategies employed in cross-device deployments fail to adapt to the dynamic nature of heterogeneous computing environments, further limiting overall system performance. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a reasoning method, system and storage medium based on a hierarchical hybrid expert model to improve the reasoning efficiency of multiple carbon tasks, optimize the utilization of heterogeneous resources, and enhance the adaptability to dynamic scenarios.

[0004] In a first aspect, an embodiment of the present application provides a reasoning method based on a hierarchical hybrid expert model, comprising: Preprocess the original carbon-related data to generate carbon feature vectors in a unified format; Using the first-layer router, the top M most relevant candidate expert clusters are selected from multiple expert clusters in a tree-like hierarchical structure based on the carbon feature vector. Using the second-layer router, the embedded representation of the corresponding expert model is combined within each candidate expert cluster to perform attention calculation on the concatenated input of the carbon feature vector and the expert model description vector, outputting the corresponding expert score. In addition, the top K expert models with the highest scores and their weights are selected in each candidate expert cluster to form a candidate expert model set. Executing expert model cache management and scheduling based on the candidate expert model set and the current system resource status to obtain an expert model execution configuration plan, wherein the execution configuration plan is used to specify the expert model to be executed and its target device; Based on the execution configuration plan, perform reasoning execution on each expert model to be executed, asynchronously collect the reasoning output results and operation performance indicators of each expert model, and obtain the result output set of each expert model; The result output set is fused according to the weights of the expert models to generate the final fusion reasoning result of the carbon data.

[0005] In another feasible embodiment, the preprocessing of the original carbon-related data to generate a carbon feature vector in a unified format includes: For industrial emission data, extract key emission factors and standardize them; For traffic energy consumption data, vehicle emission feature vectors are constructed based on spatiotemporal trajectories; For building energy consumption data, energy consumption density indicators are extracted according to building type and climate zone; For remote sensing image data, extract surface temperature and vegetation index and perform principal component dimensionality reduction; In the presence of multimodal carbon-related data, the carbon feature vectors of each modality are fused into a unified carbon feature vector through splicing or cross-modal attention.

[0006] In another feasible embodiment, the first-layer router is a multi-layer perceptron network; and selecting the top M most relevant candidate expert clusters from a plurality of expert clusters in a tree-like hierarchical structure according to the carbon feature vector using the first-layer router includes: Inputting the carbon feature vector into the multi-layer perceptron network, performing a matching evaluation on each of the multiple expert clusters in the tree-structured hierarchy to obtain an expert cluster score; After performing Softmax normalization on the scores of each expert cluster, the top M candidate expert clusters with the highest expert cluster scores are selected.

[0007] In another feasible embodiment, the second-layer router is a lightweight attention network; the second-layer router performs attention calculation based on the carbon feature vector and the expert models in each candidate expert cluster, outputs the corresponding expert score, and selects the top K expert models with the highest scores and their weights in each candidate expert cluster to form a candidate expert model set, including: The carbon feature vector is concatenated with the description vectors of each expert model in the candidate expert cluster to form a composite input, and a multi-head self-attention or Transformer calculation is performed on the composite input to obtain an expert score; After the expert scores are normalized using Softmax, the top K expert models with the highest expert scores in each candidate expert cluster and their corresponding weights are selected, and a candidate expert model set is formed based on the top K expert models.

[0008] In another feasible implementation, the performing of expert model cache management and scheduling based on the candidate expert model set and the current system resource status to obtain an expert model execution configuration plan includes: retaining the candidate expert models that are most recently and frequently called in the candidate expert model set in a cache; Marking a candidate expert model that has not been selected in N consecutive rounds in the candidate expert model set as a cold model and unloading it from the video memory, wherein when the cold model is selected again in a future task, it is restored to a usable state through an asynchronous loading mechanism; Based on the current system resource status, the target device for executing the task is selected, and each candidate expert model is matched to the target device using a parallel asynchronous scheduling method to obtain the execution configuration plan of the expert model.

[0009] In another feasible implementation, the execution configuration plan includes: recording the expert model ID, the assigned inference device ID, the binding information of the input carbon feature vector, and the output cache identifier; the inference device ID is dynamically selected based on the heterogeneous computing devices available in the current system and is asynchronously issued through the task scheduler.

[0010] In another feasible implementation, based on the execution configuration plan, reasoning and executing are performed on each expert model to be executed, and reasoning output results and operating performance indicators of each expert model are asynchronously collected to obtain a result output set of each expert model, including: Perform format adaptation and binding on the carbon feature vector and the input interface of the expert model to be executed specified in the execution configuration plan to generate a standardized input tensor; Distribute the normalized input tensor to the device where the corresponding expert model to be executed is located and start the inference session; During the execution of each expert model to be executed, whether there is an inference completion signal through asynchronous monitoring; After listening to the inference completion signal, the inference output results of each expert model are immediately extracted, and the running performance indicators are synchronously recorded, where the running performance indicators include inference time, memory usage, hardware utilization, and carbon index calculation efficiency; The inference output result and the performance index are stored in a result buffer pool to form a result output set of the expert model.

[0011] In another feasible embodiment, performing a fusion operation on the result output set according to the weight of each expert model to generate a final fusion reasoning result of the carbon data includes: Based on the weight of each expert model and its output confidence, the result output set is weighted and summed to generate the final result that integrates carbon emission results, carbon intensity indicators and carbon trend forecasts.

[0012] In a second aspect, an embodiment of the present application provides a reasoning system based on a hierarchical hybrid expert model, including: Input processing module, used to pre-process the raw carbon-related data and generate carbon feature vectors in a unified format; A routing decision module is configured to use a first-layer router to select the top M most relevant candidate expert clusters from multiple expert clusters in a tree-like hierarchical structure based on the carbon feature vector; use a second-layer router to combine the embedded representation of the corresponding expert model within each candidate expert cluster, perform attention calculation on the concatenated input of the carbon feature vector and the expert model description vector, output the corresponding expert score, and select the top K expert models with the highest scores and their weights in each candidate expert cluster to form a candidate expert model set; An expert model library, configured to perform expert model cache management and scheduling based on the candidate expert model set and the current system resource status, and obtain an execution configuration plan for the expert model, wherein the execution configuration plan is used to specify the expert model to be executed and its target device; A fusion scheduling module is used to perform reasoning execution on each expert model to be executed based on the execution configuration plan, asynchronously collect the reasoning output results and operation performance indicators of each expert model, and obtain a result output set of each expert model; The inference module is used to perform a fusion operation on the result output set according to the weights of each expert model to generate a final fusion inference result of the carbon data.

[0013] On the third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned reasoning method based on the hierarchical hybrid expert model.

[0014] The embodiments of the present application bring the following beneficial effects: The above-mentioned reasoning method, system and storage medium based on the hierarchical hybrid expert model generate a carbon feature vector in a unified format by preprocessing the original carbon-related data; using the first-layer router, the top M most relevant candidate expert clusters are selected from multiple expert clusters in a tree-like hierarchical structure according to the carbon feature vector; using the second-layer router, the embedded representation of the corresponding expert model is combined within each candidate expert cluster to perform attention calculation on the spliced input of the carbon feature vector and the expert model description vector, output the corresponding expert score, and select the top K expert models with the highest scores and their weights in each candidate expert cluster to form a candidate expert model set; expert model cache management and scheduling are performed according to the candidate expert model set and the current system resource status to obtain an execution configuration plan for the expert model, and the execution configuration plan is used to specify the expert model to be executed and its target device; based on the execution configuration plan, each expert model to be executed is inferred and executed, and the inference output results and operating performance indicators of each expert model are asynchronously collected to obtain a result output set of each expert model; the result output set is fused according to the weight of each expert model to generate the final fused inference result of the carbon data. This application preprocesses the original carbon-related data to obtain a carbon feature vector with a unified format, which is compatible with multiple modal inputs and can extract mission-critical information; the use of hierarchical routing ensures the efficient screening of expert clusters; the weighted fusion of multiple expert models ensures the high accuracy and robustness of the fusion results, and ultimately achieves the ability to quickly adjust routing strategies and model priorities under different loads and scenarios, significantly improving the system reasoning accuracy, reducing response delays and resource consumption.

[0015] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description or be understood by practicing the present application. The objectives and other advantages of the present application are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0016] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A flowchart of an embodiment of a reasoning method based on a hierarchical hybrid expert model provided in an embodiment of the present application; Figure 2 A flowchart of another embodiment of the reasoning method based on the hierarchical hybrid expert model provided in an embodiment of the present application; Figure 3 A schematic diagram of a reasoning system based on a hierarchical hybrid expert model provided in an embodiment of the present application; Figure 4 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0020] The terms "first," "second," "third," "fourth," and so forth (if any) in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that shown or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0021] For ease of understanding, the specific process of the embodiment of the present application is described below. The embodiment of the present application is applied to system devices such as computers and servers. Please refer to Figure 1 An embodiment of the reasoning method based on the hierarchical hybrid expert model in the present application specifically includes the following steps: S110, preprocessing the original carbon-related data to generate a carbon feature vector in a unified format; In this step, the raw carbon-related data can be image, text, or voice data, specifically including industrial emissions data, transportation energy consumption data, building energy data, and remote sensing image data. The system first determines the modal type of the carbon-related data and adopts the following preprocessing methods for different types: Image data: Calculate the covariance matrix of its pixel matrix and perform eigenvalue decomposition through principal component analysis (PCA). Select the carbon eigenvectors corresponding to the first k largest eigenvalues to form the low-dimensional semantic principal component vector of the image; Text data: Input into a pre-trained language model such as BERT or RoBERTa, extract the context vector of each token, and generate a fixed-length sentence vector through pooling (average pooling or CLS token); Speech data: Mel-frequency cepstral coefficients (MFCCs) are used to extract temporal features of the sound, which are then fed into an autoencoder structure for encoding and compression to obtain a low-dimensional representation vector of the speech signal. For multimodal input, such as mixed text and image data, the aforementioned dimensionality reduction process is performed on each modal data separately. Then, a concatenation mechanism or a cross-modal attention mechanism (such as CrossAttention) is used to align the modal information and fuse them into a unified Carbon feature vector X.

[0022] For industrial emission data, key emission factors are extracted and standardized; for transportation energy consumption data, vehicle emission characteristic vectors are constructed based on spatiotemporal trajectories; for building energy consumption data, energy consumption density indicators are extracted according to building type and climate zone; for remote sensing image data, surface temperature and vegetation index are extracted and principal component dimensionality reduction is performed.

[0023] S120, performing multi-level expert routing based on the carbon feature vector to construct a candidate expert set; Specifically, the first-layer router is used to select the top M most relevant candidate expert clusters from multiple expert clusters in a tree hierarchy according to the carbon feature vector; the second-layer router is used to perform attention calculation based on the carbon feature vector and the expert models in each candidate expert cluster, output the corresponding expert score, and select the top K expert models with the highest scores and their weights in each candidate expert cluster to form a candidate expert model set.

[0024] In practice, the first-layer router uses a multi-layer perceptron (MLP) to map the carbon feature vector X, outputting a score vector S = MLP(X) corresponding to each expert cluster. After Softmax normalization, the top M expert clusters with the highest scores are selected as the candidate set. This process enables coarse-grained semantic attribution determination of tasks. For each candidate expert cluster, the second-layer router incorporates the expert model's structural embedding vector ei and concatenates it with the input feature X to form a composite vector zi = concat(X, ei). This composite vector zi is input into a lightweight attention module (which can be a 1-2-layer Transformer or a gated attention mechanism) to calculate the attention score for each expert model. The expert scores within each cluster are normalized using the Softmax function, and the top K expert models in each cluster are selected. Their corresponding weights are recorded to form the final set of candidate expert models E.

[0025] S130, performing expert model cache management and scheduling according to the candidate expert model set and the current system resource status to obtain an execution configuration plan for the expert model; It should be noted that the execution configuration plan is used to specify the expert model to be executed and its target device. Based on the candidate expert set E generated in step S120, the inference system evaluates the historical call frequency and recent performance indicators of each expert model to determine its cache status: If it is a high-frequency thermal model, it will be directly retained in the GPU memory or main memory for execution; If it is a low-frequency cold model and is not currently resident, the asynchronous loading process is triggered.

[0026] Furthermore, the inference system collects the current computing resource load through the resource perception module, including CPU core usage, GPU memory occupancy, NPU computing power peak, etc.

[0027] Then, according to the computational graph complexity and resource demand level of the expert model, the candidate expert set E is matched with the current resource status, and a scheduling plan is generated using a greedy or heuristic strategy.

[0028] The execution configuration plan P is a scheduling task list, which includes: expert model ID, target execution device ID, input binding feature X, output buffer area identifier, etc.

[0029] The inference system submits the execution configuration plan P to the scheduling service, which then distributes it to various heterogeneous devices for execution based on task priority and availability.

[0030] S140: Based on the execution configuration plan, perform reasoning execution on each expert model to be executed, asynchronously collect the reasoning output results and operation performance indicators of each expert model, and obtain a result output set of each expert model; In this embodiment, after receiving the scheduling instruction, each expert model calls the inference engine (such as ONNX Runtime, TensorRT or TorchScript) to complete the inference task. The specific implementation is as follows: Convert the bound carbon feature vector X into a model-compatible tensor structure and upload it to the specified device memory; Start the inference thread and return the processing handle to the scheduling module; The system monitors the expert model's return status in non-blocking mode to detect task completion events; Once the task is completed, the prediction result Yi of the corresponding expert model is extracted from the output buffer; At the same time, performance indicators such as the execution time, video memory occupancy, and core utilization of the current inference are collected; The result Yi and its corresponding performance indicators are packaged and stored in the central result buffer pool to form the output set {Yi}.

[0031] S150: Perform a fusion operation on the result output set according to the weights of each expert model to generate a final fusion reasoning result.

[0032] Specifically, based on the score weights of the candidate expert models, the results in the set {Yi} are weighted: F = ∑αi × Yi, where αi is the normalized weight derived by the second-layer router and Yi is the expert model output. The system can further incorporate a consistency measurement module to determine the degree of consistency between the outputs of multiple expert models. If a model is found to be significantly outlier (e.g., significantly diverging from other expert predictions), its fusion weight can be dynamically adjusted. When higher fusion reliability is required, the system can enable a gated fusion mechanism or an attention fusion network (such as Meta-Fusion or Gated Sum) to dynamically weight and control the output F. Ultimately, F serves as the output of this inference task, which can be used by subsequent modules or returned to the user.

[0033] The above technical solutions enable the pre-processing of carbon feature vectors in a unified format to accommodate multiple modal inputs and extract mission-critical information. Hierarchical routing utilizes coarse-fine learnable routers, ensuring efficient screening of expert clusters while accurately selecting the most weighted expert model set through a lightweight attention network. Expert model caching and heterogeneous parallel scheduling significantly reduce graphics memory / memory usage and scheduling latency while maintaining high availability. Asynchronous reasoning and real-time collection of performance indicators provide precise data for dynamic feedback. Multi-expert weighted fusion ensures high accuracy and robustness of the fusion results. The closed-loop self-adaptation of the overall reasoning process, combined with subsequent reinforcement learning optimization, allows for rapid adjustment of routing strategies and model priorities under different loads and scenarios, significantly improving system reasoning accuracy, reducing response latency, and reducing resource consumption.

[0034] like Figure 2 As shown, another embodiment of the reasoning method based on the hierarchical hybrid expert model provided by this application specifically includes the following steps: S210, preprocessing carbon-related data to generate a carbon feature vector in a unified format; The preprocessing to generate a unified format carbon feature vector refers to converting different modal inputs into standardized representations. Specifically, principal component analysis can be used to process image data, pre-trained language models can be used to extract text features, autoencoders can be used to compress speech features, and cross-modal attention mechanisms can be used to fuse multi-modal carbon-related data to achieve compatible processing of heterogeneous inputs.

[0035] In this embodiment, the raw carbon-related data includes image data, text data, and voice data, such as industrial emission data, traffic energy consumption data, building energy consumption data, and remote sensing image data. When preprocessing the raw carbon-related data to generate a unified format carbon feature vector, specifically: For image data, calculate its pixel matrix covariance and perform eigenvalue decomposition, and select the principal component vector to map the image to low-dimensional features; For text data, use a pre-trained language model to generate sentence vectors; For speech data, MFCC features are extracted and compressed into low-dimensional representations through autoencoders; In the presence of multimodal carbon-related data, the carbon feature vectors of each modality are fused into a unified carbon feature vector through splicing or cross-modal attention.

[0036] It should be noted that pixel matrix covariance refers to constructing a covariance matrix by counting the correlation between image pixels. Specifically, a sliding window can be used to traverse the pixel points to calculate the covariance value, which is used to capture the spatial distribution characteristics of the image texture.

[0037] The principal component vector refers to the carbon eigenvector corresponding to the maximum eigenvalue selected after performing eigenvalue decomposition on the covariance matrix. Specifically, it can be implemented using a singular value decomposition algorithm to reduce the image dimension while retaining the main information.

[0038] A pre-trained language model refers to a neural network that has been trained on large-scale corpus and has the ability to understand semantics. Specifically, the BERT or RoBERTa model can be used to generate sentence vectors to capture the deep semantic features of the text.

[0039] MFCC features refer to the spectral characteristics of speech signals extracted through Mel-frequency cepstral coefficients. Specifically, they can be implemented through frame windowing, Fourier transform, and Mel filter bank processing to characterize the acoustic characteristics of speech.

[0040] An autoencoder refers to a neural network structure that compresses carbon-related data through an encoder and then reconstructs it through a decoder. Specifically, it can be constructed by combining convolutional layers and fully connected layers to reduce high-dimensional MFCC features to a compact representation.

[0041] Cross-modal attention refers to calculating the correlation between features of different modalities through an attention mechanism. Specifically, it can be implemented using a multi-head attention layer to dynamically fuse the complementary information of multimodal carbon-related data.

[0042] In this embodiment, when the raw carbon-related data is image data, the covariance of the pixel matrix is first calculated to capture spatial correlations. Eigenvalue decomposition is then used to select principal component vectors, mapping the raw high-dimensional pixel data into a low-dimensional feature space. For example, for a 224×224 RGB image, the covariance calculation yields a 3×3 covariance matrix, which is then decomposed to select the first 128 principal component vectors.

[0043] For text data, a pre-trained language model is used to directly extract sentence vectors. For example, after inputting the input text into the BERT model, the 768-dimensional vector corresponding to the [CLS] tag is taken as the feature representation.

[0044] For speech data, MFCC features are extracted after frame processing to form a time-frequency two-dimensional matrix, which is spatially compressed through the convolutional layer of the autoencoder to output a 128-dimensional low-dimensional vector.

[0045] When there is multimodal input, such as images and text, the carbon feature vectors of the two modalities are concatenated and input into the cross-modal attention layer. By calculating the interaction weights of image features and text features, a fused unified carbon feature vector is generated.

[0046] Here, a hierarchical preprocessing architecture is used to design optimized feature extraction methods for different data types, and a cross-modal attention mechanism is combined to achieve efficient fusion of multi-source data, significantly reducing the computational complexity of heterogeneous data processing.

[0047] S220, using the first layer router, selecting the most relevant top M candidate expert clusters from multiple expert clusters in the tree hierarchy according to the carbon feature vector; In this embodiment, the first-layer router is a multi-layer perceptron network; using the first-layer router, the top M most relevant candidate expert clusters are selected from multiple expert clusters in a tree-like hierarchical structure based on the carbon feature vector, including: inputting the carbon feature vector into the multi-layer perceptron network, performing a matching evaluation on each expert cluster in the multiple expert clusters in the tree-like hierarchical structure to obtain an expert cluster score; after performing Softmax normalization on each expert cluster score, the top M candidate expert clusters with the highest expert cluster scores are selected.

[0048] It should be noted that the multilayer perceptron network refers to a neural network structure composed of a fully connected layer and a nonlinear activation function. Specifically, it can be implemented by combining the ReLU activation function with the batch normalization layer to map the input carbon feature vector to the expert cluster matching space.

[0049] The multiple expert clusters in the tree-like hierarchical structure refer to hierarchical expert model groupings formed by fields or functions. Specifically, they can be constructed using knowledge graphs or clustering algorithms to reduce the complexity of routing calculations through a hierarchical structure.

[0050] Matching evaluation involves calculating the degree of correlation between input features and each expert cluster. This can be represented by a scalar value output by a multilayer perceptron network, quantifying the suitability of the expert cluster for the task at hand. Softmax normalization converts the expert cluster scores into a probability distribution, using an exponential function and a normalized denominator. This ensures that the selection of candidate expert clusters is probabilistically interpretable.

[0051] In this embodiment, when the carbon feature vector is input into the multilayer perceptron network, the network generates initial matching scores for each expert cluster through forward propagation. Each expert cluster in the tree hierarchy is assigned a score that reflects the relevance of the current carbon-related data within the task domain represented by that expert cluster. Softmax normalization converts the raw scores into probability distributions, providing a clear probabilistic basis for selecting the top M candidate expert clusters. For example, when processing an image classification task, the multilayer perceptron network might assign a higher score to a cluster containing image recognition experts, while assigning a lower score to a cluster containing text processing experts, thereby ensuring that routing decisions are highly aligned with task requirements.

[0052] This step significantly reduces computational overhead by introducing a tree-like hierarchy that works in conjunction with a multilayer perceptron network to decompose routing decisions into a hierarchical selection process. This tree-like hierarchy transforms global routing decisions into hierarchical local selections, avoiding routing bottlenecks in high-concurrency scenarios. The nonlinear mapping capabilities of the multilayer perceptron network ensure the accuracy of expert cluster matching assessments, enabling the system to quickly identify the subset of experts most relevant to carbon-related data.

[0053] S230. Utilize the second-layer router to combine the embedded representation of the corresponding expert model within each candidate expert cluster, perform attention calculation on the concatenated input of the carbon feature vector and the expert model description vector, output the corresponding expert score, and select the top K expert models with the highest scores and their weights in each candidate expert cluster to form a candidate expert model set. The second-layer router is a lightweight attention network. When using the second-layer router to form a set of candidate expert models, the carbon feature vector and the description vector of each expert model in the candidate expert cluster are concatenated into a composite input, and multi-head self-attention or Trans former calculation is performed on the composite input to obtain the expert score. After the expert score is normalized using Softmax, the top K expert models with the highest expert scores in each candidate expert cluster and their corresponding weights are selected, and the candidate expert model set is formed based on the top K expert models.

[0054] It should be noted that a lightweight attention network refers to an attention mechanism network with a small number of parameters and controllable computational complexity. It can be achieved by reducing the number of attention heads or compressing the hidden layer dimensions. Its role is to balance the computational overhead and the accuracy of expert model matching.

[0055] Composite input refers to the joint input formed by splicing the input carbon feature vector and the expert model description vector. It can be implemented by vector splicing or projection fusion. Its function is to enable the attention calculation to consider both the carbon-related data characteristics and the expert model capabilities.

[0056] Multi-head self-attention refers to decomposing the input into multiple subspaces and calculating the attention weights separately. It can be implemented by using parallel linear transformation and scaled dot product attention mechanism. Its function is to capture the multi-dimensional correlation between the input and the expert model.

[0057] Softmax normalization refers to converting the original score into a probability distribution, which can be achieved by using exponential operations and normalized denominator calculations. Its role is to ensure the rationality of the weight distribution of the expert model.

[0058] In this embodiment, when selecting an expert model within a candidate expert cluster, the input carbon feature vector is first concatenated with the description vector of the expert model to form a composite input that includes input characteristics and expert capabilities. This composite input is fed into a lightweight attention network for multi-head self-attention calculation, and expert scores are generated through feature interactions in parallel subspaces. After the scores are normalized, the top K expert models with the highest scores in each expert cluster are screened out, and a candidate set is constructed based on their weights. For example, when the input is image data, the carbon feature vector may contain visual semantic information, while the expert description vector encodes the model's ability to process image classification or target detection. The attention calculation can effectively match the correlation between the two. As a preferred embodiment, the position encoding in the Transformer calculation can be replaced with the registration order encoding of the expert model to reduce computational complexity.

[0059] In practical applications, the lightweight attention network has a dynamic adaptive adjustment mechanism, which is used to dynamically adjust the weight distribution in the attention calculation based on the historical reasoning accuracy, expert model selection entropy and the confidence requirements of the current reasoning task, so as to improve the discrimination ability of the candidate expert model set.

[0060] Among them, the dynamic adaptive adjustment mechanism refers to an algorithmic architecture that adjusts the attention weight distribution in real time through a feedback control module. Specifically, it can be implemented by combining historical data statistics based on a sliding window with an online optimizer, and periodically updating the weight adjustment parameters to adapt to dynamic task requirements. The historical reasoning accuracy refers to the prediction accuracy indicator of the expert model in historical tasks. Specifically, it can be calculated by recording the matching degree between the output results of each expert model and the true label, and is used to reflect the reliability of the expert model. The expert model selection entropy refers to the degree of confusion in the probability distribution of the expert model being selected during the route selection process. Specifically, it can be calculated using the information entropy formula and is used to measure the diversity of routing decisions. The confidence requirement of the current reasoning task refers to the task's required level of reliability of the output results. Specifically, it can be quantified through a preset threshold or task type label to control the concentration of the attention weight distribution.

[0061] In this embodiment, during the operation of the lightweight attention network, a dynamic adaptive adjustment mechanism works in tandem with three parameters: first, it collects historical reasoning accuracy data for each expert model to construct an expert credibility evaluation matrix; second, it calculates the information entropy of the expert model's selection probability in recent routing selections to determine the diversity level of the current selection strategy; and finally, it combines the confidence requirement parameters in the task configuration file to generate an attention weight adjustment coefficient. These three parameters are linearly weighted and input into the attention calculation layer, dynamically changing the bias value used in the similarity calculation between the query vector and the key vector in the attention head. This results in the attention mechanism favoring expert models with stable historical performance in tasks requiring high confidence, while enhancing model selection diversity by increasing the selection entropy weight in scenarios requiring exploratory reasoning.

[0062] This lightweight design reduces computing resource consumption while ensuring routing decision quality, enabling the system to maintain stable performance in high-concurrency scenarios. Furthermore, the introduction of an attention mechanism enhances the model's adaptability to varying input characteristics, enabling better expert combination selection across diverse tasks.

[0063] S240, executing expert model cache management and scheduling, and generating an expert model execution configuration plan; Specifically, the candidate expert models in the candidate expert model set that have been called frequently recently are retained in the cache; the candidate expert models in the candidate expert model set that have not been selected for N consecutive rounds are marked as cold models and unloaded from the video memory, wherein, when the cold model is selected again in a future task, it is restored to a usable state through an asynchronous loading mechanism; based on the current system resource status, the target device for executing the task is selected, and each candidate expert model is matched to the target device by using a parallel asynchronous scheduling method to obtain an execution configuration plan for the expert model.

[0064] It should be noted that the cache refers to a storage area that adopts the LRU or LFU replacement strategy, which can be implemented by GPU memory or high-speed SSD to reduce the loading delay of frequently accessed models. A cold model refers to an expert model that has not been called within a certain time window. It can be marked with a reference counter and timestamp mechanism to release the resources occupied by the idle model. The asynchronous loading mechanism means that when the background thread detects that the cold model has been reselected, it automatically restores the model from the disk or remote storage to a usable state. It can be implemented by combining preloading and lazy loading to balance resource occupancy and response speed. The target device refers to the available hardware resources in a heterogeneous computing environment. Specifically, the resource monitoring module can be used to collect the load status of the CPU, GPU, and NPU in real time, and dynamically allocate them based on the load balancing algorithm.

[0065] In this embodiment, the frequently accessed models in the candidate expert model set are retained in the cache to reduce the time overhead caused by repeated loading. For cold models that have not been selected for multiple consecutive rounds, resources are released through memory unloading operations, and their metadata is recorded for subsequent recovery. When the cold model is selected again, the model recovery is completed in the background through the asynchronous loading mechanism to avoid blocking the current reasoning task. The system resource status is dynamically calculated by combining the resource demand parameters of the expert model with the resource utilization of each computing device by real-time monitoring, such as the number of idle CPU cores, the remaining amount of GPU memory, and the NPU computing power occupancy rate. The optimal deployment device is calculated. The final generated execution configuration plan contains the expert model identifier, target device number, and input and output binding information, and is distributed to the corresponding computing node for execution in an asynchronous manner through the task scheduler.

[0066] Furthermore, the expert model cache management also includes building a priority scoring function based on the inference latency and accuracy of the expert model. When system resources are limited, the cold model with the lowest score is eliminated first to ensure that high-performance models receive priority scheduling.

[0067] The priority scoring function refers to a comprehensive indicator that quantifies model performance and resource consumption. This can be achieved by converting the inference latency and accuracy into a single scoring value using a weighted summation method. For example, the inverse of the latency indicator is added proportionally to the accuracy. This function is used to dynamically evaluate the priority of the model in resource-constrained scenarios, thereby optimizing cache space allocation. A cold model refers to a candidate expert model that has not been selected for multiple consecutive rounds. This can be achieved by using a counter to record the number of times the model has not been called. When the number exceeds the set threshold, the unloading operation is triggered. This mechanism can effectively release idle resources and avoid memory waste.

[0068] During system operation, the inference latency and historical accuracy of each expert model are monitored in real time and input into the priority scoring function for calculation. The scoring result reflects the comprehensive performance of the model in the current task scenario. When the video memory or computing resources are insufficient, the system prioritizes retaining models with higher scores based on the scoring sorting results, and removes the cold models with the lowest scores from the video memory. For example, when the GPU memory occupancy reaches the preset threshold, the system automatically triggers the resource recovery process and determines the unloading object by comparing the scoring values of the cold models. At the same time, the metadata of the unloaded model remains on disk and can be quickly restored through the asynchronous loading mechanism when required by subsequent tasks.

[0069] In another feasible implementation, the execution configuration plan includes recording the expert model ID, the assigned inference device ID, the binding information of the input carbon feature vector, and the output cache identifier; the inference device ID is dynamically selected based on the heterogeneous computing devices available in the current system and is asynchronously issued through the task scheduler.

[0070] The expert model ID is a coding sequence used to uniquely identify each model in the set of candidate expert models. This can be implemented using a hash value or a UUID, allowing for rapid location and call of the target model in a distributed system. The inference device ID is a number identifying a heterogeneous computing device. This can be obtained through the device registry or resource management module and is used to map the expert model to physical computing resources. The binding information for the input carbon feature vector is the association between the preprocessed carbon feature vector and the expert model input interface for format adaptation. This can be implemented using tensor shape matching or data encapsulation to ensure compatibility between carbon-related data and the model interface. The output cache identifier is the address tag for the cache used to store inference results. This can be implemented using a memory pointer or a distributed storage path, facilitating asynchronous collection and subsequent result fusion. Dynamic selection of heterogeneous computing devices involves allocating appropriate computing units to the expert model based on real-time resource status. This can be achieved by collecting CPU, GPU, and NPU load data through the resource monitoring module and dynamically matching them based on the computing power requirements of the expert model. Asynchronous dispatch of the task scheduler refers to the non-blocking transmission of the execution configuration plan to the target device. This can be achieved by using a message queue or remote procedure call mechanism to reduce the delay caused by task dispatch.

[0071] When generating an execution configuration plan, the resource monitoring module first obtains real-time status data for each computing device in the current system, such as the number of idle CPU cores, remaining GPU memory capacity, and NPU computing power utilization. A matching degree is then calculated based on the resource requirement parameters predefined during expert model registration, such as memory usage and computational complexity, combined with the device status data. For example, image processing models requiring high memory usage prioritize GPU devices with sufficient memory availability; while speech recognition models requiring low latency prioritize NPU devices with idle computing power. After device matching is complete, the expert model ID is bound to the target device ID, and the input Carbon feature vector is converted into a data format compatible with the target model's input interface, for example, by encapsulating the text Carbon feature vector into a tensor of a specific dimension. An output cache identifier generates a unique path based on the task ID and device ID to ensure that inference results from different tasks do not overwrite each other. Finally, the task scheduler sends the configuration plan to the target device via an asynchronous communication channel. Upon receiving the plan, the device immediately loads the model and initiates the inference session without waiting for other scheduled tasks to complete.

[0072] In some specific implementations, performing expert model scheduling according to the candidate expert model set E and the current system resource status includes: Detect the resource status of various types of computing devices in the current system, including the number of idle CPU cores, GPU memory usage, and NPU computing power utilization; Establishing mapping rules based on the resource requirement parameters of each expert model and the resource status, and calculating the optimal deployment device for each expert model; Based on the deployment device selection results, an expert model execution configuration plan P is constructed, which includes the expert model identifier, target device number, and input and output binding structure; The system scheduling module is called to distribute the execution configuration plan P to the corresponding computing nodes in an asynchronous manner.

[0073] S250: Based on the execution configuration plan, perform reasoning execution on each expert model to be executed, asynchronously collect the reasoning output results and operation performance indicators of each expert model, and obtain a result output set of each expert model; In this step, the carbon feature vector is format-adapted and bound to the input interface of the expert model to be executed specified in the execution configuration plan to generate a standardized input tensor; the standardized input tensor is distributed to the device where the corresponding expert model to be executed is located, and the inference session is started; during the execution of each expert model to be executed, whether there is an inference completion signal is monitored asynchronously; after the inference completion signal is monitored, the inference output results of each expert model are immediately extracted, and the operating performance indicators are synchronously recorded, where the operating performance indicators include inference time, memory occupied and hardware utilization; the inference output results and performance indicators are stored in the result cache pool to constitute the result output set of the expert model.

[0074] Among them, standardized input tensor refers to the conversion of carbon feature vectors into a unified data format supported by the input interface of the expert model to be executed. Specifically, this can be achieved by adjusting the tensor dimension or normalization processing, such as by adding a padding layer or using an interpolation algorithm to adjust the feature dimension to ensure that carbon-related data is compatible with the interfaces of different expert models. Asynchronous listening mode refers to a non-blocking monitoring mechanism, which can be implemented by an event-driven architecture or callback function. For example, the status of the inference task can be monitored in real time through a message queue or semaphore mechanism to avoid wasting computing resources due to polling checks. The result cache pool refers to a shared memory area used to temporarily store inference outputs. Specifically, it can be implemented by a distributed key-value database or a memory-mapped file. For example, a cache pool can be built by Redis or Memcached to support multi-threaded concurrent access and fast data retrieval.

[0075] During the inference execution phase, the carbon feature vector is first format-adapted to the input interface of the target expert model, such as converting the text carbon feature vector into the image tensor input format of the visual model, or adjusting the dimension through interpolation to match the input requirements of different models. The standardized input tensor is distributed to heterogeneous computing devices through the task scheduler, such as allocating compute-intensive models to the GPU and memory-sensitive models to the NPU. After the inference session is started, the system continuously monitors the task status of each device through an asynchronous listening mechanism. For example, when the GPU completes the calculation, an interrupt signal is triggered, and the output results are immediately extracted and performance data is recorded. The running performance indicators are collected synchronously and stored in the cache pool. For example, the inference time and video memory occupancy are written into the time series database to provide data support for subsequent model scheduling optimization.

[0076] In some specific embodiments, the generation of standardized input tensors can be dynamically adjusted based on expert model metadata, for example, automatically selecting an adapter module based on the input format requirements specified during model registration. Asynchronous monitoring mechanisms can employ multiplexing techniques to simultaneously monitor multiple device ports, such as using the epoll or kqueue system calls to efficiently manage large-scale concurrent tasks. The result cache can implement data partitioning strategies, such as sharding storage based on expert model ID hashing, to prevent a single node from becoming a performance bottleneck.

[0077] S260: performing a fusion operation on the result output set according to the weights of each expert model to generate a final fusion inference result of the carbon data; In this step, based on the weight of each expert model and its output confidence, the result output set is weighted and summed to generate the final result that integrates carbon emission results, carbon intensity indicators and carbon trend forecasts.

[0078] The weight refers to the relative importance of each expert model in the fusion process. This can be determined by the attention score calculated by the second-layer router, for example, using the Softmax-normalized score as the weight parameter. Output confidence refers to the expert model's assessment of the certainty of its own inference results. This can be calculated using the model's output probability distribution entropy or the maximum category probability value. For example, the confidence value is calculated by calculating the probability value of the predicted category in a classification task. Weighted summation involves multiplying the inference results of each expert model by the product of its corresponding weight and confidence, and then accumulating them. This can be achieved using matrix multiplication or element-by-element multiplication followed by summation.

[0079] After each expert model completes inference, the resulting output set contains the outputs of multiple expert models and their corresponding weights. The confidence level of each expert model's output is calculated based on its own inference results. For example, in image classification tasks, the maximum class probability output by the model serves as the confidence level. Subsequently, the output of each expert model is multiplied by the product of its weight and confidence level to obtain a weighted result. All weighted results are accumulated to generate the final fused inference result. For example, in a text sentiment analysis task, if two expert models output positive probabilities of 0.8 and 0.7, respectively, with weights of 0.6 and 0.4, and confidences of 0.9 and 0.85, respectively, the final fused result is 0.8 × 0.6 × 0.9 + 0.7 × 0.4 × 0.85. This process dynamically combines model weights and confidence levels, allowing highly confident and highly weighted expert models to have a greater impact on the final result.

[0080] S270, obtaining the error between the fusion result and the true label, combining the inference delay and the expert model selection distribution entropy to construct a reward function; S280, based on the reward function, using a reinforcement learning algorithm to update and optimize the parameters of the first-layer routers and the second-layer routers; In this step, the reinforcement learning algorithm is PPO (Proximal Policy Optimization), and the reward function R is in the following form: R = α × (prediction accuracy) − β × (average inference delay) − γ × (expert selection entropy), where α, β, and γ are experience adjustment coefficients used to balance accuracy and resource consumption.

[0081] Furthermore, after each round of training, the updated first-layer and second-layer router parameters are stored in the policy parameter server, and offline regression testing is triggered regularly to prevent policy degradation.

[0082] In summary, the raw carbon-related data is preprocessed to generate a unified carbon feature vector format. A first-layer router selects candidate clusters from the tree-structured hierarchical expert clusters. Attention calculations are performed using a second-layer router in conjunction with the expert model embedding representation to select the expert model with the highest score. Cache management and scheduling are performed based on the system resource status to generate an execution configuration plan. Expert model inference is performed asynchronously and results are collected. The result set is weighted and fused to generate the final output. This system effectively coordinates hierarchical expert models with heterogeneous computing resources, ensuring inference accuracy while reducing response latency. A unified processing mechanism for multimodal input improves system compatibility, a dynamic resource scheduling strategy optimizes hardware utilization, a hierarchical routing design enhances the accuracy of expert selection, an asynchronous execution mechanism accelerates task processing, and a weighted fusion method ensures the reliability of the final results.

[0083] Corresponding to the above method embodiment, see Figure 3 A schematic diagram of a reasoning system based on a hierarchical hybrid expert model is shown, the system comprising: An input processing module 310 is used to pre-process the raw carbon-related data and generate a carbon feature vector in a unified format; The routing decision module 320 is configured to use a first-layer router to select the top M most relevant candidate expert clusters from multiple expert clusters in a tree-like hierarchical structure based on the carbon feature vector; use a second-layer router to combine the embedded representation of the corresponding expert model within each candidate expert cluster, perform an attention calculation on the concatenated input of the carbon feature vector and the expert model description vector, output the corresponding expert score, and select the top K expert models with the highest scores and their weights in each candidate expert cluster to form a candidate expert model set; An expert model library 330 is configured to perform expert model cache management and scheduling based on the candidate expert model set and the current system resource status, and obtain an execution configuration plan for the expert model, wherein the execution configuration plan is used to specify the expert model to be executed and its target device; The fusion scheduling module 340 is used to perform reasoning execution on each expert model to be executed based on the execution configuration plan, asynchronously collect the reasoning output results and operation performance indicators of each expert model, and obtain a result output set of each expert model; The inference module 350 is used to perform a fusion operation on the result output set according to the weights of each expert model to generate a final fusion inference result of the carbon data.

[0084] Optionally, the input processing module 310 is specifically configured to: For image data, calculate its pixel matrix covariance and perform eigenvalue decomposition, and select the principal component vector to map the image to low-dimensional features; For text data, a pre-trained language model is used to generate sentence vectors. For speech data, MFCC features are extracted and compressed into low-dimensional representations through an autoencoder. In the presence of multimodal carbon-related data, the carbon feature vectors of each modality are fused into a unified carbon feature vector through splicing or cross-modal attention.

[0085] Optionally, the first layer router is a multi-layer perceptron network; the routing decision module 320 is specifically configured to: Inputting the carbon feature vector into the multi-layer perceptron network, performing a matching evaluation on each of the multiple expert clusters in the tree-structured hierarchy to obtain an expert cluster score; After performing Softmax normalization on the scores of each expert cluster, the top M candidate expert clusters with the highest expert cluster scores are selected.

[0086] Optionally, the second-layer router is a lightweight attention network; the routing decision module 320 is specifically configured to: The carbon feature vector is concatenated with the description vectors of each expert model in the candidate expert cluster to form a composite input, and a multi-head self-attention or Transformer calculation is performed on the composite input to obtain an expert score; After the expert scores are normalized using Softmax, the top K expert models with the highest expert scores in each candidate expert cluster and their corresponding weights are selected, and a candidate expert model set is formed based on the top K expert models.

[0087] Optionally, the expert module library 330 is specifically used to: retaining the candidate expert models that are most recently and frequently called in the candidate expert model set in a cache; Marking a candidate expert model that has not been selected in N consecutive rounds in the candidate expert model set as a cold model and unloading it from the video memory, wherein when the cold model is selected again in a future task, it is restored to a usable state through an asynchronous loading mechanism; Based on the current system resource status, the target device for executing the task is selected, and each candidate expert model is matched to the target device using a parallel asynchronous scheduling method to obtain the execution configuration plan of the expert model.

[0088] Optionally, the execution configuration plan includes: recording the expert model ID, the assigned inference device ID, the binding information of the input carbon feature vector, and the output cache identifier; the inference device ID is dynamically selected based on the heterogeneous computing devices available in the current system, and is asynchronously issued through the task scheduler.

[0089] Optionally, the fusion scheduling module 340 is specifically configured to: Perform format adaptation and binding on the carbon feature vector and the input interface of the expert model to be executed specified in the execution configuration plan to generate a standardized input tensor; Distribute the normalized input tensor to the device where the corresponding expert model to be executed is located and start the inference session; During the execution of each expert model to be executed, whether there is an inference completion signal through asynchronous monitoring; After listening to the inference completion signal, the inference output results of each expert model are immediately extracted, and the running performance indicators are synchronously recorded, where the running performance indicators include inference time, memory usage, hardware utilization, and carbon index calculation efficiency; The inference output result and the performance index are stored in a result buffer pool to form a result output set of the expert model.

[0090] Optionally, the reasoning module 350 is specifically configured to: Based on the weight of each expert model and its output confidence, the result output set is weighted and summed to generate the final result that integrates carbon emission results, carbon intensity indicators and carbon trend forecasts.

[0091] The system provided by the above embodiment is compatible with multiple modal inputs and extracts mission-critical information through the pre-processing of carbon feature vectors in a unified format. The use of hierarchical routing utilizes coarse-fine two-level learnable routers, ensuring efficient screening of expert clusters while accurately selecting the set of expert models with the highest weights through a lightweight attention network. Expert model caching and heterogeneous parallel scheduling significantly reduce video memory / memory usage and scheduling delays while maintaining high availability. Asynchronous reasoning and real-time collection of performance indicators provide accurate data for dynamic feedback. Multi-expert weighted fusion ensures high accuracy and robustness of the fusion results. The overall closed-loop adaptive reasoning process, combined with subsequent reinforcement learning optimization, can quickly adjust routing strategies and model priorities under different loads and scenarios, significantly improving system reasoning accuracy, reducing response delays, and reducing resource consumption.

[0092] This embodiment further provides an electronic device including a processor and a memory, wherein the memory stores machine-executable instructions executable by the processor, and the processor executes the machine-executable instructions to implement the aforementioned reasoning method based on the hierarchical hybrid expert model. The electronic device can be a server or a terminal device.

[0093] See also Figure 4 As shown, the electronic device includes a processor 400 and a memory 401 , wherein the memory 401 stores machine executable instructions that can be executed by the processor 400 , and the processor 400 executes the machine executable instructions to implement the above-mentioned reasoning method based on the hierarchical hybrid expert model.

[0094] Further, Figure 4 The electronic device shown further includes a bus 402 and a communication interface 403 , and the processor 400 , the communication interface 403 and the memory 401 are connected via the bus 402 .

[0095] The memory 401 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is achieved through at least one communication interface 403 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 402 may be an ISA bus, a PCI bus, or an EISA bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0096] The processor 400 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 400 or by instructions in the form of software. The above-mentioned processor 400 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in memory 401. Processor 400 reads information from memory 401 and, in conjunction with its hardware, completes the steps of the hierarchical hybrid expert model-based reasoning method of the aforementioned embodiment.

[0097] This embodiment also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned reasoning method based on the hierarchical hybrid expert model.

[0098] The computer program product of the reasoning method, device, electronic device and storage medium based on the hierarchical hybrid expert model provided in the embodiments of the present application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. For specific implementation, please refer to the method embodiments and will not be repeated here.

[0099] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0100] In addition, in the description of the embodiments of this application, unless otherwise specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0101] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0102] In the description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate the description of this application and simplify the description. They do not indicate or imply that the devices or components referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0103] Finally, it should be noted that the above embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art who is familiar with the technical field can still modify the technical solutions described in the above embodiments within the technical scope disclosed in the present application, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A reasoning method based on a hierarchical hybrid expert model, applied to a reasoning system, characterized in that: The method comprises: Preprocess the original carbon-related data to generate carbon feature vectors in a unified format; Using the first-layer router, the top M most relevant candidate expert clusters are selected from multiple expert clusters in a tree-like hierarchical structure based on the carbon feature vector. Using the second-layer router, the embedded representation of the corresponding expert model is combined within each candidate expert cluster to perform attention calculation on the concatenated input of the carbon feature vector and the expert model description vector, outputting the corresponding expert score. In addition, the top K expert models with the highest scores and their weights are selected in each candidate expert cluster to form a candidate expert model set. Executing expert model cache management and scheduling based on the candidate expert model set and the current system resource status to obtain an expert model execution configuration plan, wherein the execution configuration plan is used to specify the expert model to be executed and its target device; Based on the execution configuration plan, perform reasoning execution on each expert model to be executed, asynchronously collect the reasoning output results and operation performance indicators of each expert model, and obtain the result output set of each expert model; The result output set is fused according to the weights of the expert models to generate the final fusion reasoning result of the carbon data.

2. The method according to claim 1, characterized in that The preprocessing of the original carbon-related data to generate a carbon feature vector in a unified format includes: For industrial emission data, extract key emission factors and standardize them; For traffic energy consumption data, vehicle emission feature vectors are constructed based on spatiotemporal trajectories; For building energy consumption data, energy consumption density indicators are extracted according to building type and climate zone; For remote sensing image data, extract surface temperature and vegetation index and perform principal component dimensionality reduction; In the presence of multimodal carbon-related data, the carbon feature vectors of each modality are fused into a unified carbon feature vector through splicing or cross-modal attention.

3. The method according to claim 1, characterized in that The first-layer router is a multi-layer perceptron network; the first-layer router is used to select the most relevant top M candidate expert clusters from multiple expert clusters in a tree-like hierarchical structure according to the carbon feature vector, including: Inputting the carbon feature vector into the multi-layer perceptron network, performing a matching evaluation on each of the multiple expert clusters in the tree-structured hierarchy to obtain an expert cluster score; After performing Softmax normalization on the scores of each expert cluster, the top M candidate expert clusters with the highest expert cluster scores are selected.

4. The method according to claim 1, wherein The second-layer router is a lightweight attention network. The second-layer router performs attention calculation based on the carbon feature vector and the expert models in each candidate expert cluster, outputs the corresponding expert score, and selects the top K expert models with the highest scores and their weights in each candidate expert cluster to form a candidate expert model set, including: The carbon feature vector is concatenated with the description vectors of each expert model in the candidate expert cluster to form a composite input, and a multi-head self-attention or Transformer calculation is performed on the composite input to obtain an expert score; After the expert scores are normalized using Softmax, the top K expert models with the highest expert scores in each candidate expert cluster and their corresponding weights are selected, and a candidate expert model set is formed based on the top K expert models.

5. The method according to any one of claims 1 to 4, characterized in that The expert model cache management and scheduling are performed according to the candidate expert model set and the current system resource status to obtain an execution configuration plan of the expert model, including: retaining the candidate expert models that are most recently and frequently called in the candidate expert model set in a cache; Marking a candidate expert model that has not been selected in N consecutive rounds in the candidate expert model set as a cold model and unloading it from the video memory, wherein when the cold model is selected again in a future task, it is restored to a usable state through an asynchronous loading mechanism; Based on the current system resource status, the target device for executing the task is selected, and each candidate expert model is matched to the target device using a parallel asynchronous scheduling method to obtain the execution configuration plan of the expert model.

6. The method according to claim 5, characterized in that The execution configuration plan includes: recording the expert model ID, the assigned inference device ID, the binding information of the input carbon feature vector, and the output cache identifier; the inference device ID is dynamically selected based on the heterogeneous computing devices available in the current system and asynchronously issued through the task scheduler.

7. The method according to claim 5, characterized in that Based on the execution configuration plan, the inference execution is performed on each expert model to be executed, and the inference output results and operation performance indicators of each expert model are asynchronously collected to obtain the result output set of each expert model, including: Perform format adaptation and binding on the carbon feature vector and the input interface of the expert model to be executed specified in the execution configuration plan to generate a standardized input tensor; Distribute the normalized input tensor to the device where the corresponding expert model to be executed is located and start the inference session; During the execution of each expert model to be executed, whether there is an inference completion signal through asynchronous monitoring; After listening to the inference completion signal, the inference output results of each expert model are immediately extracted, and the running performance indicators are synchronously recorded, where the running performance indicators include inference time, memory usage, hardware utilization, and carbon index calculation efficiency; The inference output result and the performance index are stored in a result buffer pool to form a result output set of the expert model.

8. The method according to claim 5, characterized in that The result output set is fused according to the weight of each expert model to generate the final fusion reasoning result of the carbon data, including: Based on the weight of each expert model and its output confidence, the result output set is weighted and summed to generate the final result that integrates carbon emission results, carbon intensity indicators and carbon trend forecasts.

9. A reasoning system based on a hierarchical hybrid expert model, characterized in that: The system comprises: Input processing module, used to pre-process the raw carbon-related data and generate carbon feature vectors in a unified format; A routing decision module is configured to use a first-layer router to select the top M most relevant candidate expert clusters from multiple expert clusters in a tree-like hierarchical structure based on the carbon feature vector; use a second-layer router to combine the embedded representation of the corresponding expert model within each candidate expert cluster, perform attention calculation on the concatenated input of the carbon feature vector and the expert model description vector, output the corresponding expert score, and select the top K expert models with the highest scores and their weights in each candidate expert cluster to form a candidate expert model set; An expert model library, configured to perform expert model cache management and scheduling based on the candidate expert model set and the current system resource status, and obtain an execution configuration plan for the expert model, wherein the execution configuration plan is used to specify the expert model to be executed and its target device; A fusion scheduling module is used to perform reasoning execution on each expert model to be executed based on the execution configuration plan, asynchronously collect the reasoning output results and operation performance indicators of each expert model, and obtain a result output set of each expert model; The inference module is used to perform a fusion operation on the result output set according to the weights of each expert model to generate a final fusion inference result of the carbon data.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to implement the reasoning method based on the hierarchical hybrid expert model according to any one of claims 1 to 8.

Images