Low-computing-power language expert system based on layered architecture
By using a low-computing-power language expert system based on a hierarchical architecture, and combining multiple virtual expert generation layers, inference optimization layers, and domain adaptation layers, the illusion problem of large language models in edge computing environments is solved, achieving efficient and reliable domain transfer and inference, and improving the performance of the model in resource-constrained environments.
Patent Information
- Application Number
- CN202511389333.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-26
- Publication Date
- 2026-01-23
AI Technical Summary
Existing large-scale language models suffer from hallucinations in resource-constrained edge computing environments, and existing methods are insufficient in reducing hallucinations in model outputs, improving inference accuracy and robustness, especially in real-time control applications such as medical and industrial manufacturing, which are limited by computing power and power consumption.
A low-computing-power language expert system based on a hierarchical architecture is adopted, including an expert generation layer, an inference optimization layer, and a domain adaptation layer. By generating multiple virtual experts, independently predicting probabilities, fixed-vote fusion, confidence and expert orthogonality weighting, and pluggable memory components, it achieves instant domain transfer and efficient reasoning.
It significantly reduces hallucinations, improves reasoning accuracy and robustness, supports second-level domain transfer, reduces computational overhead, and ensures efficient operation and highly reliable output of the model in low-computing-power environments.
Smart Images
Figure CN121390230A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence, specifically relating to a low-computing-power language expert system based on a hierarchical architecture. Background Technology
[0002] With the development of artificial intelligence technology, large language models (LLMs) have demonstrated powerful capabilities in natural language processing tasks. However, adapting these models to specific domains remains a challenge. Existing domain adaptation methods, such as Domain Adaptive Pre-training (DAPT) and Retrieval Augmentation Generation (RAG), either require expensive full-parameter training or introduce significant computational overhead during inference. Furthermore, existing methods also have shortcomings in reducing the illusion phenomenon in model output and improving inference accuracy and robustness.
[0003] In resource-constrained edge computing environments, current generative models exhibit a cascading degradation phenomenon of "illusion-bias-fragility." The root cause lies in the dual loss of knowledge and parameters resulting from model compression (such as pruning, distillation, and quantization): weight sparsity and representation dimension collapse jointly weaken the capacity boundaries of factual memory and logical reasoning, causing the illusion rate to increase exponentially with the compression rate. More critically, the lack of energy redundancy required for online backpropagation and external knowledge retrieval at the edge leads to broken error feedback loops, making the model more sensitive to input perturbations. Furthermore, the discretization noise introduced by low-bit quantization further erases the interpretability of gradients, making it difficult for developers to effectively intervene within the limited inter-layer representation space. Traditional posterior correction, data augmentation, or RAG paradigms are all limited by computational power and power consumption under these constraints, thus hindering the practical application of generative models in real-time control applications such as healthcare and industrial manufacturing. Summary of the Invention
[0004] To overcome the shortcomings of the prior art, the present invention aims to provide a low-computing-power language expert system based on a hierarchical architecture, which solves the problem of small language model illusion under low computing power conditions.
[0005] This invention is achieved through the following technical solution:
[0006] A low-computing-power language expert system based on a hierarchical architecture consists of a vertically coupled expert generation layer, an inference optimization layer, and a domain adaptation layer. The expert generation layer generates a batch of virtual experts with heterogeneous knowledge, each carrying exclusive domain hints and randomly selecting a subset of experts. The inference optimization layer uses the randomly selected subset of experts from the expert generation layer to generate prediction probabilities, performs anomaly truncation and fixed voting, eliminates extreme predictions using a mean-standard deviation threshold, and then weights and fuses the confidence score with the orthogonality of the experts to determine the final output label. The domain adaptation layer pre-trains a pluggable memory component based on the final output label obtained from the inference optimization layer and inserts the pre-trained pluggable memory component. During inference, the pre-trained pluggable memory component is interpolated with the probability of the basic language model to complete real-time domain transfer.
[0007] Furthermore, the expert generation layer generates N virtual experts, each with different domain knowledge backgrounds and independent viewpoints; domain-specific prompts are designed for each virtual expert to guide the model in generating diverse predictions, and k experts are randomly selected from the N experts to form a subset S.
[0008] Furthermore, the specific implementation method of the inference optimization layer is as follows:
[0009] Each selected expert independently generates a prediction probability p based on their personalized prompts. i Let i = 1 / 2 / 3...k, calculate the prediction probability p of all selected experts. i Mean μ and standard deviation σ:
[0010]
[0011] A threshold θ is set based on the mean μ and standard deviation σ, where θ = μ + σ, to filter out p. i The prediction of >θ yields the refined set S. r :
[0012] S r =[{t1,t2,…,t m}∣p i ≤θ]
[0013] Where m represents the number of refinement candidate tags;
[0014] In the refined set S r In this process, the probability f(t) of each refined candidate label t is calculated. Assuming j∈(1,m), the label with the highest probability is selected as the output label t. * :
[0015]
[0016] For each refined candidate label t, calculate its confidence level C. jDefined as the negative mean logarithmic probability:
[0017]
[0018] Among them, logp ij It is the prediction probability of the i-th expert for the j-th label;
[0019] In each refined candidate label t generation step, each expert independently predicts the next label based on their specific hints, and a fixed voting mechanism is applied to determine the final output label. For each refined candidate label t, the refined candidate labels t selected by the participating experts are extracted. j Embedded vector e j Calculate the cosine similarity matrix:
[0020]
[0021] Wherein, cos(e u ,e v ) represents the u-th embedding vector e u and the v-th embedding vector e v The cosine similarity value between them Represents the embedding vector e u transpose, Represents the embedding vector e v transpose, ||e u || represents the embedding vector e u The norm, ||e v || represents the embedding vector e v The norm;
[0022] The average similarity is derived from the upper triangular portion of the cosine similarity matrix.
[0023]
[0024] The expert orthogonality score O is defined as:
[0025]
[0026] A higher expert orthogonality score indicates greater independence in predictions among experts, while a lower expert orthogonality score suggests that predictions tend to be consistent.
[0027] Confidence level C j The score S is obtained by fusing the orthogonality score O with the overall score S. c (t j ), used to evaluate the overall quality of refined candidate marker t:
[0028] S c (t j)=α·C j +(1-α)·O
[0029] Here, α is a non-negative weighting parameter, i.e., α∈(0,1), used to balance the contributions of confidence and expert ratings. S is chosen as the weighting parameter. c The highest value is used as the final output value t. ** :
[0030]
[0031] Furthermore, the implementation method of the pre-trained pluggable memory component in the domain adaptation layer is as follows:
[0032] When expert disagreement is significant enough and the decision deviates from the statistical mainstream, the sample is considered a difficult case and deserves further study.
[0033]
[0034] Where, τ O The orthogonality threshold, The characteristic function is δ, which is 1 if the condition is met and 0 otherwise. hard =1 indicates a difficult example;
[0035] Only when the model is sufficiently confident in its current decision can it be written to the external memory bank of the pluggable memory component to prevent low-confidence noise from contaminating the domain corpus.
[0036]
[0037] Where, τ C δ represents the upper confidence level. write =1 allows writing; otherwise, discard.
[0038] To avoid the risk of overfitting due to one-hot pseudo-labels, a differentiated smoothing strategy is used for difficult and easy examples:
[0039] p pseudo (·|x)=(1-λ)OneHot(t ** )+λ[δ hard U(·)+(1-δ hard )Softmax(S c )]
[0040] In the formula, λ is the smoothing coefficient, λ∈(0,1); U(·) is the smoothing coefficient in S r The model uses a uniform distribution on the Softmax layer for exploring difficult examples; the Softmax part preserves the model's own confidence distribution.
[0041] The resulting new training pair (x, p) pseudo Directly add to the pre-training queue Dnew , where x is the context of the input;
[0042] The pluggable memory component is trained using a distribution alignment loss, minimizing the output distribution p of the pluggable memory component. Mem (·∣x) and the smooth pseudo-label distribution p pseudo KL divergence between (·|x):
[0043] L KL (x)=KL(p pseudo (·∣x)||p Mem (·∣x))
[0044] At the same time, in conjunction with the standard language modeling objectives:
[0045] L LM (x)=-logp Mem (y∣x)
[0046] The final loss function is:
[0047] L(x) = β·L KL (x)+(1-β)·L LM (x)
[0048] Here, β is a non-negative weight parameter, i.e., β∈(0,1).
[0049] Furthermore, the specific implementation of the pre-trained pluggable memory component and the probability interpolation of the basic language model in the domain adaptation layer is as follows:
[0050] Interpolate the output distribution of the pluggable memory component with the output distribution of the base language model to generate domain-enhanced predictions:
[0051] p Mem-PLM (y t |x)=γ·p Mem (y t |x)+(1-γ)·p PLM (y t |x)
[0052] Where γ is a non-negative weight parameter, i.e., γ∈(0,1); p PLM This represents the output probability distribution of the pre-trained language model; specifically, it represents the probability distribution of the base language model for a given label y. t The predicted probability.
[0053] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0054] This invention constructs multi-domain virtual experts who independently output prediction probabilities under domain-specific prompts; it removes extreme probabilities by statistically truncating outliers, integrates expert consensus through parameterless fixed voting, and introduces orthogonality metrics to further suppress conformity bias and significantly reduce illusions; during the inference phase, it implements early stopping using internal confidence signals, dynamically truncates low-quality generation paths, and streamlines computing power; with a pluggable memory module, the system supports second-level domain migration and zero manual parameter tuning, maintaining highly consistent and reliable output even in low-computing-power environments.
[0055] This invention introduces a pre-trained pluggable memory component, Memory Decoder, which encodes domain knowledge into a compact parametric model by mimicking the behavior of a non-parametric retrieval machine, achieving domain adaptation without modifying the original model parameters. Attached Figure Description
[0056] Figure 1 This is a schematic diagram of the overall framework of the present invention;
[0057] Figure 2 For comparison of evaluation results from multiple virtual experts (including models with 128, 32, and 3 experts). Detailed Implementation
[0058] The present invention will be further described in detail below with reference to specific embodiments. These descriptions are for explanation purposes only and are not intended to limit the scope of the invention.
[0059] like Figure 1 As shown, a low-computing-power language expert system based on a hierarchical architecture consists of a vertically coupled expert generation layer, an inference optimization layer, and a domain adaptation layer. The expert generation layer first generates a batch of virtual experts with heterogeneous knowledge, each carrying exclusive domain hints and randomly selecting a subset of experts. The inference optimization layer uses the randomly selected subset of experts from the expert generation layer to generate prediction probabilities, immediately performs anomaly truncation and fixed voting, uses the mean-standard deviation threshold to remove extreme predictions, and then weights and fuses the confidence score with the orthogonality of the experts to determine the final output price. The final label is determined with zero dynamic gating and zero additional parameters, and early stopping is driven by confidence score to cut off low-quality inference paths in real time. The domain adaptation layer inserts a pre-trained Memory Decoder to compress massive domain knowledge into plug-and-play modules in a way that mimics the kNN nonparametric retrieval mechanism. During inference, the pre-trained Memory Decoder interpolates with the probability of the basic language model, and real-time domain transfer can be completed without fine-tuning the original weights. As a result, illusions are suppressed by multi-perspective cross-validation, and the consistency of facts is significantly improved; modularization and fixed voting mechanisms make decisions transparent and traceable; memory components compatible with any word segmenter enable the entire framework to be seamlessly migrated to different architecture LLMs, achieving low-latency, low-power, and highly robust trusted generation on edge devices.
[0060] (1) Expert Generation Layer
[0061] - Generate virtual experts: Generate N virtual experts, each with different domain knowledge backgrounds, and their viewpoints are independent of each other. In this invention, N = 128.
[0062] - Define expert tips: Design domain-specific tips for each virtual expert to guide the model in generating diverse predictions. For example, tips from historian experts relate to historical events, while tips from literature experts relate to literary creation.
[0063] - Randomly select a subset of experts: Randomly select k experts from N experts to form a subset S. In this invention, k = 32.
[0064] (2) Inference Optimization Layer
[0065] Furthermore, in the inference optimization layer, each selected expert independently generates a prediction probability p based on specific prompts. i p i , i = 1 / 2 / 3...k;
[0066] Calculate the prediction probability p of all selected experts i Mean μ and standard deviation σ:
[0067]
[0068] A threshold θ is set based on the mean μ and standard deviation σ, where θ = μ + σ, to filter out p. i The prediction of >θ yields the refined set S. r :
[0069] S r =[{t1,t2,…,t m}∣p i ≤θ]
[0070] Where m represents the number of refined candidate tags.
[0071] In the refined set S r In this process, the probability f(t) of each refined candidate label t is calculated. Assuming j∈(1,m), the label with the highest probability is selected as the output label t. * :
[0072]
[0073] For each refined candidate label t, calculate its confidence level C. j Defined as the negative mean logarithmic probability:
[0074]
[0075] Among them, logp ij It is the prediction probability of the i-th expert for the j-th label;
[0076] In each refined candidate label t generation step, each expert independently predicts the next label based on their specific hints, and a fixed voting mechanism is applied to determine the final output label. For each generated refined candidate label t, the refined candidate labels t selected by the participating experts are extracted. j Embedded vector e j Calculate the cosine similarity matrix:
[0077]
[0078] Wherein, cos(e u ,e v ) represents the u-th embedding vector e u and the v-th embedding vector e v The cosine similarity value between them Represents the embedding vector e u transpose, Represents the embedding vector e v transpose, ||e u || represents the embedding vector e u The norm, ||e v || represents the embedding vector e v The norm;
[0079] The average similarity is derived from the upper triangular part of the matrix.
[0080]
[0081] The expert orthogonality score O is defined as:
[0082]
[0083] Higher scores indicate greater independence in the predictions among experts, while lower scores suggest that the predictions tend to be consistent.
[0084] Confidence level C j The overall score S is obtained by combining the expert score O with the overall score S. c (t j ), used to evaluate the overall quality of refined candidate marker t:
[0085] S c (t j )=α·C j +(1-α)·O
[0086] Where α is a non-negative weighting parameter, i.e., α∈(0,1), used to balance the contributions of confidence and expert ratings. S is chosen as the weighting parameter. c The highest value is used as the final output value t. ** :
[0087]
[0088] (3) Domain Adaptation Layer
[0089] Difficult case triggers: When expert disagreement is sufficiently large and the decision deviates from the statistical mainstream, the sample is considered a difficult case and deserves further study.
[0090]
[0091] Where, τ O The orthogonality threshold is set to 0.5. The characteristic function is δ, which is 1 if the condition is met and 0 otherwise. hard =1 indicates a difficult example.
[0092] Memory write trigger: Writes to the external memory only when the model is sufficiently confident in the current decision, preventing low-confidence noise from contaminating the domain corpus.
[0093]
[0094] Where, τ C δ represents the upper confidence level (taken as 0.2). write =1 Allow writing, otherwise discard.
[0095] Smoothing pseudo-label distribution: To avoid the risk of overfitting due to one-hot pseudo-labels, the system employs a differentiated smoothing strategy for difficult and easy examples.
[0096] p pseudo (·|x)=(1-λ)OneHot(t ** )+λ[δ hard U(·)+(1-δ hard )Softmax(S c )]
[0097] In the formula, λ is the smoothing coefficient, λ∈(0,1); U(·) is the smoothing coefficient in S r The uniform distribution on the model is used for exploring difficult examples: the Softmax part preserves the model's own confidence distribution.
[0098] The resulting new training pair (x, p) pseudo Directly add to the pre-training queue D new No manual intervention is required. Here, x represents the context of the input.
[0099] - Pre-training objective: Train the Memory Decoder using distribution alignment loss, minimizing the output distribution p of the Memory Decoder. Mem (·∣x) and the smooth pseudo-label distribution p pseudo KL divergence between (·|x):
[0100] L KL (x)=KL(p pseudo (·∣x)||p Mem (·∣x))
[0101] At the same time, in conjunction with the standard language modeling objectives:
[0102] L LM (x)=-logp Mem (y∣x)
[0103] The final loss function is:
[0104] L(x) = β·L KL (x)+(1-β)·L LM (x)
[0105] Where β is a non-negative weight parameter, i.e., β∈(0,1).
[0106] - Plug-in integration: Seamlessly integrate the trained Memory Decoder with any pre-trained language model that shares the same tokenizer, without modifying the original model parameters.
[0107] - Interpolation during inference: During inference, the output distribution of the pluggable memory component Memory Decoder is interpolated with the output distribution of the base language model to generate domain-enhanced predictions.
[0108] p Mem-PLM (y t |x)=γ·p Mem (y t |x)+(1-γ)·p PLM (y t |x)
[0109] Where γ is a non-negative weight parameter, i.e., γ∈(0,1); p PLM This represents the output probability distribution of a pre-trained language model. Specifically, it represents the probability distribution of the base language model for a given label y. t The predicted probability.
[0110] like Figure 2As shown, the evaluation results of multiple virtual experts are compared (including models with 128, 32, and 3 experts). The vertical axis corresponds to the orthogonality score (abbreviated as O) calculated by the formula. The higher the score, the lower the similarity between experts and the more independent the predictions. The horizontal axis represents the generation sequence of each token in the text generation process. The calculation is performed once a new token is generated.
[0111] Tell a story – an open-ended creative text generation tool with no standard answer; the more diverse, the better.
[0112] The Economics question – which asks for a forecast of the “World Economic Outlook for 2025” – falls under the category of fact / knowledge questions, and the answers should converge to a reliable consensus as much as possible.
[0113] This invention significantly reduces illusion phenomena and improves factual consistency through multi-virtual expert fusion strategies, confidence filtering, pre-trained memory modules, and dynamic early stopping. Simultaneously, by utilizing domain-specific expert hints and a fixed voting mechanism, the model can comprehensively judge from multiple perspectives, generating more accurate and stable outputs, enhancing its performance in complex tasks and improving inference accuracy and robustness. Furthermore, employing a fixed voting mechanism instead of a dynamic gating network avoids additional parameters and computational overhead, ensuring efficient model operation in resource-constrained environments such as edge devices, reducing deployment costs, and optimizing computational resource utilization. The clear modular design and fixed voting mechanism make the model's decision-making process more transparent, facilitating developer understanding and debugging, improving the model's practicality and reliability, and enhancing its interpretability. By optimizing the number of experts and embedding space noise injection, an optimal balance between output diversity and stability is achieved, avoiding excessive smoothing or excessive fluctuations, and improving the overall performance of the model. The pre-trained memory module supports plug-and-play functionality, requiring no fine-tuning of the original model, and possesses efficient domain adaptation capabilities. Dynamic early stopping and the hybrid voting mechanism reduce computational overhead and improve computational efficiency. Orthogonality constraints and confidence visualization make the decision-making process more transparent, further enhancing model interpretability. This invention also exhibits cross-model compatibility, supporting LLMs with different architectures and demonstrating strong generalization capabilities, making it particularly suitable for practical applications in low-computational-power environments.
Claims
1. A low-computing-power language expert system based on a layered architecture, characterized in that, It consists of a vertically coupled expert generation layer, reasoning optimization layer, and domain adaptation layer: the expert generation layer generates a batch of virtual experts with heterogeneous knowledge, each carrying exclusive domain hints and randomly selecting a subset of experts; The inference optimization layer generates prediction probabilities using a subset of experts randomly selected by the expert generation layer, performs anomaly truncation and fixed voting, eliminates extreme predictions using the mean-standard deviation threshold, and then weights and fuses the confidence score with the orthogonality of the experts to determine the final output label. The domain adaptation layer pre-trains pluggable memory components based on the final output label obtained by the inference optimization layer and inserts the pre-trained pluggable memory components. During inference, the pre-trained pluggable memory components are interpolated with the probabilities of the basic language model to complete the real-time domain transfer.
2. The low-computing-power language expert system based on a layered architecture according to claim 1, characterized in that, The expert generation layer generates N virtual experts, each with different domain knowledge and independent viewpoints. Domain-specific prompts are designed for each virtual expert to guide the model in generating diverse predictions. K experts are randomly selected from the N experts to form a subset S.
3. The low-computing-power language expert system based on a layered architecture according to claim 1, characterized in that, The specific implementation method of the inference optimization layer is as follows: Each selected expert independently generates a prediction probability p based on their personalized prompts. i Let i = 1 / 2 / 3...k, calculate the prediction probability p of all selected experts. i Mean μ and standard deviation σ: A threshold θ is set based on the mean μ and standard deviation σ, where θ = μ + σ, to filter out p. i The prediction of >θ yields the refined set S. r : S r =[{t1,t2…,t m }∣p i ≤θ] Where m represents the number of refinement candidate tags; In the refined set S r In this process, the probability f(t) of each refined candidate label t is calculated. Assuming j∈(1,m), the label with the highest probability is selected as the output label t. * : For each refined candidate label t, calculate its confidence level C. j Defined as the negative mean logarithmic probability: Among them, logp ij It is the prediction probability of the i-th expert for the j-th label; In each refined candidate label t generation step, each expert independently predicts the next label based on their specific hints, and a fixed voting mechanism is applied to determine the final output label. For each refined candidate label t, the refined candidate labels t selected by the participating experts are extracted. j Embedded vector e j Calculate the cosine similarity matrix: Wherein, cos(e u ,e v ) represents the u-th embedding vector e u and the v-th embedding vector e v The cosine similarity value between them Represents the embedding vector e u transpose, Represents the embedding vector e v transpose, ||e u || represents the embedding vector e u The norm, ||e v || represents the embedding vector e v The norm; The average similarity is derived from the upper triangular portion of the cosine similarity matrix. The expert orthogonality score O is defined as: A higher expert orthogonality score indicates greater independence in predictions among experts, while a lower expert orthogonality score suggests that predictions tend to be consistent. Confidence level C j The score S is obtained by fusing the orthogonality score O with the overall score S. c (t j ), used to evaluate the overall quality of refined candidate marker t: S c (t j )=α·C j +(1-a)·O Here, α is a non-negative weighting parameter, i.e., α∈(0,1), used to balance the contributions of confidence and expert ratings. S is chosen as the weighting parameter. c The highest value is used as the final output value t. ** :
4. A low-computing-power language expert system based on a layered architecture according to claim 1, characterized in that, The implementation method of the pre-trained pluggable memory component in the domain adaptation layer is as follows: When expert disagreement is significant enough and the decision deviates from the statistical mainstream, the sample is considered a difficult case and deserves further study. Where, τ O The orthogonality threshold, The characteristic function is δ, which is 1 if the condition is met and 0 otherwise. hard =1 indicates a difficult example; Only when the model is sufficiently confident in its current decision can it be written to the external memory bank of the pluggable memory component to prevent low-confidence noise from contaminating the domain corpus. Where, τ C δ represents the upper confidence level. write =1 allows writing; otherwise, discard. To avoid the risk of overfitting due to one-hot pseudo-labels, a differentiated smoothing strategy is used for difficult and easy examples: p pseudo (·|x)=(1-λ)OneHot(t ** )+λ[δ hard U(·)+(1-δ hard )Softmax(S c )] In the formula, λ is the smoothing coefficient, λ∈(0,1); U(·) is the smoothing coefficient in S r The model uses a uniform distribution on the Softmax layer for exploring difficult examples; the Softmax part preserves the model's own confidence distribution. The resulting new training pair (x, p) pseudo Directly add to the pre-training queue D new , where x is the context of the input; The pluggable memory component is trained using a distribution alignment loss, minimizing the output distribution p of the pluggable memory component. Mem (·∣x) and the smooth pseudo-label distribution p pseudo KL divergence between (·|x): L KL (x)=KL(p pseudo (·∣x)||p Mem (·∣x)) At the same time, in conjunction with the standard language modeling objectives: L LM (x)=-logp Mem (y∣x) The final loss function is: L(x)=β·L KL (x)+(1-β)·L LM (x) Here, β is a non-negative weight parameter, i.e., β∈(0,1).
5. A low-computing-power language expert system based on a layered architecture according to claim 4, characterized in that, The specific implementation of the pre-trained pluggable memory component and the probability interpolation of the basic language model in the domain adaptation layer is as follows: Interpolate the output distribution of the pluggable memory component with the output distribution of the base language model to generate domain-enhanced predictions: p Mem-PLM (y t ∣x)=γ·p Mem (y t ∣x)+(1-γ)·p PLM (u t ∣x) Where γ is a non-negative weight parameter, i.e., γ∈(0,1); p PLM This represents the output probability distribution of the pre-trained language model; specifically, it represents the probability distribution of the base language model for a given label y. t The predicted probability.