A large language model optimization method and system for high-concurrency online medical conversations

Abstract

This invention discloses a heterogeneous hybrid expert large model architecture and alignment method for high-concurrency online medical dialogue. First, the method proposes a dual-pool heterogeneous routing strategy of "4 general experts + 2 departmental experts." Through physically isolated parallel routing channels, it forces the model to simultaneously activate general language capabilities and specialized medical reasoning when processing input, fundamentally ensuring the focus and professionalism of responses. Second, it adopts a structural evolution strategy based on parameter expansion and selective freezing. By completely freezing the pre-trained basic parameters and training only newly added departmental experts, it completely eliminates catastrophic forgetting. Finally, in the model alignment stage, it employs the Group Relative Policy Optimization (GRPO) algorithm for valueless models to reduce memory overhead and designs a dual-track hybrid reward function combining low-level semantic similarity rewards and high-level structured large model referee scoring, effectively solving the "reward hacking" problem and guiding the model to generate refined responses that combine clinical accuracy and humanized interaction. This invention significantly improves the performance of medical dialogue systems. Its sparse activation and parameter freezing characteristics further ensure low latency and high throughput in online high-concurrency scenarios, providing a complete solution for the deployment of reliable medical dialogue systems.

Description

[0001] This invention relates to the interdisciplinary field of artificial intelligence, natural language processing, and clinical medicine, specifically to a method and system for constructing a heterogeneous hybrid expert large model for high-concurrency online medical dialogue. More specifically, this invention aims to solve the technical problems of large language models in real medical consultations, such as lengthy responses, blurred focus, insufficient professional depth, and difficulties in high-concurrency deployment, through a heterogeneous routing mechanism of "general expert - department expert," a continuous learning strategy based on parameter expansion, and an alignment method based on group relative strategy optimization. Background Technology

[0002] Although large language models have developed rapidly in recent years and have shown great promise in medical dialogue systems, namely to simulate professional doctor-patient interactions and provide reliable medical advice, it is undeniable that directly applying existing technologies to high-concurrency real-world online consultation scenarios will inevitably encounter various difficulties.

[0003] While the generalized large language model performs exceptionally well on medical knowledge question-answering benchmarks, it undeniably leans more towards "probabilistic reasonableness" than "factual accuracy" when handling open-domain patient complaints. Consequently, its responses are often verbose, deviate from clinical focus, lack professionalism, and can even induce serious medical illusions. Therefore, a clear "accuracy gap" exists, and the model's fluent output is difficult to directly apply to clinical decision-making.

[0004] Because hybrid expert architectures are well-suited for supporting high-concurrency access and naturally and reasonably controlling computational costs, their sparse activation approach is an excellent attempt to expand model capacity while maintaining low inference overhead. However, the traditional homogeneous MoE architecture, when ported to the medical field, has very clear and significant drawbacks: First, the spontaneous and undifferentiated routing strategy employed cannot effectively distinguish between general language skills and specialized medical logic, thus easily leading to an imbalance between professionalism and accessibility, and insufficiently focused output. Second, when the model continuously learns across multiple disciplines, it is highly susceptible to "catastrophic forgetting," meaning that when learning new knowledge, it forgets its original basic dialogue and reasoning abilities.

[0005] Since the goal of alignment optimization is to ensure that the model output conforms to clinical standards, reinforcement learning alignment should theoretically be used. However, it is undeniable that traditional proximal strategy optimization methods require loading large-scale value models, which can easily lead to memory overflow under the MoE architecture, making them highly impractical in engineering. More seriously, when using an external large model as a referee to design the reward mechanism, a typical "reward hacking" phenomenon occurs: the model generates lengthy, complex-formatted but empty text to cheat for high scores, which completely conflicts with the fundamental goal of improving the conciseness and professionalism of responses.

[0006] In summary, existing technologies have significant bottlenecks in high-concurrency online medical dialogue scenarios, including model response quality, architectural adaptability, and alignment feasibility. Therefore, it is natural to seek new architectures and training paradigms to effectively improve the accuracy, focus, and professionalism of responses generated in medical dialogues while ensuring extremely low online inference latency. Summary of the Invention

[0007] The purpose of this invention is to propose a heterogeneous hybrid expert model architecture and optimization and deployment method suitable for high-concurrency online medical dialogues. This naturally and reasonably solves the technical problems exposed by traditional dense large language models and standard hybrid expert models in medical dialogue scenarios, such as lengthy responses, blurred clinical focus, insufficient professional depth, susceptibility to hallucinations, large computational latency, catastrophic forgetting, and high alignment costs during high-concurrency inference. Therefore, it will inevitably improve the professionalism, accuracy, smoothness of interaction, and online service carrying capacity of medical dialogue systems.

[0008] To achieve the above objectives, the present invention provides the following solution:

[0009] A heterogeneous hybrid expert large model approach for high-concurrency online medical dialogue includes:

[0010] The basic model is Mixtral 8x7B, an open-source hybrid expert model that has been pre-trained on massive amounts of general data and has powerful general dialogue capabilities. This basic model contains a multi-layer Transformer structure, in which the feedforward network layer is a standard hybrid expert layer.

[0011] Starting from the heterogeneous hybrid expert routing architecture, the hybrid expert layers in the foundation were restructured in a very natural and reasonable way: the original homogeneous expert pool was decoupled into two physically isolated sub-pools, namely a general expert pool and a department-specific expert pool, and independent routing gating networks were designed for each. For each input lexical representation, the system executes two sets of routing calculations in parallel: the general routing channel selects the four general experts with the highest scores based on the probability distribution of the gating network output, thus effectively preserving the processing capabilities of natural language syntax, common sense logic, and dialogue empathy; while the department routing channel forcibly selects the two department experts with the highest scores, thereby reliably and fully extracting specialized medical terminology and clinical reasoning logic. More importantly, the final output of the heterogeneous hybrid expert layer is the weighted sum of the outputs of the experts activated by the two channels, and its mandatory "four general experts plus two department experts" allocation mechanism strictly ensures from the network structure that the model processes general semantics and professional medical features in parallel and in a balanced manner in each forward computation, thus fundamentally solving the fundamental problem of the model responding in a general and detached manner from clinical focus.

[0012] By employing a pre-defined structural evolution strategy of parameter expansion and selective freezing, during medical domain adaptation, all self-attention layer parameters, layer normalization layer parameters, and gradient updates of the original general expert pool and its routers in the foundational model are completely disabled. Simultaneously, within each hybrid expert layer, untrained, department-specific expert pools and their corresponding routers are horizontally expanded to carry knowledge specific to a particular department; these newly added parameters are defined as trainable increments. In subsequent supervised fine-tuning and reinforcement learning alignment of multi-departmental medical data, only the newly added departmental parameters are optimized using gradient descent, while the basic general parameters remain unchanged during training. This achieves lossless injection of new medical knowledge and fundamentally prevents the catastrophic loss of existing general dialogue capabilities due to learning new knowledge.

[0013] A group-based relative policy optimization algorithm based on a valueless model is used for reinforcement learning alignment of the model. This algorithm abandons the traditional approach of requiring additional training and loading of a large value model in proximal policy optimization. Instead, it generates multiple candidate responses in parallel from the current policy model within the dialogue context of the same patient complaint. Then, it calculates the mean and standard deviation of the rewards for all responses within the group, thereby obtaining the empirical advantage value of each response relative to the group average. Based on this advantage value, the policy model parameters are optimized. This method reduces the memory overhead of reinforcement learning alignment training by nearly half, making efficient and low-cost alignment of large-scale hybrid expert models possible.

[0014] By combining a dual-track hybrid reward function that integrates low-level factual constraints with high-level style evaluation, precise and robust gradient signals are provided for the reinforcement learning. This reward function consists of a weighted mixture of semantic similarity reward based on dense vectors and multi-dimensional scoring reward from a large model judge based on a structured scale. The semantic similarity reward uses a lightweight encoder model fine-tuned specifically for medical literature to calculate the cosine similarity between the generated response and the anonymized gold standard doctor's response in the high-dimensional embedding space, serving as a rigid baseline anchor point to ensure the clinical factual accuracy of the generated content. The multi-dimensional scoring reward from the large model judge utilizes a larger out-of-the-box model as the judge, deconstructing and scoring the generated response based on a pre-set structured evaluation scale containing three core dimensions: "focus and conciseness," "clinical professionalism," and "humanistic care and empathy." This guides the model to develop an interaction style that is both concise and professional, yet also empathetic. The two mechanisms counterbalance each other, effectively addressing the "reward hacking" phenomenon that easily arises from solely relying on the large model judge's scoring, where the model outputs lengthy and empty text to obtain high scores.

[0015] Based on the sparse activation characteristics and fully frozen state of the basic parameters of the heterogeneous routing architecture, deep system-level optimizations are achieved when deploying for high-concurrency online inference scenarios. Since only a fixed number of six experts are activated when processing each input lexical unit, the computational cost of the model's inference is strictly controlled at a low constant level. Simultaneously, all concurrent user requests share the exact same frozen general parameters, enabling high reuse of the key-value cache in the Transformer self-attention mechanism across different patient dialogue sessions. Combined with a decoupled inference service architecture, the system can separate general language understanding tasks from lightweight specialist logic computations at the service layer, thereby significantly reducing the latency of first-word generation, minimizing memory fragmentation, and significantly improving the system's request throughput and overall response efficiency when dealing with massive concurrent access.

[0016] This invention discloses the following technical effects: The heterogeneous routing mechanism of "general expert pool and departmental expert pool" proposed in this invention has a very clear and rigorous dual-channel design, thus naturally and reasonably achieving the isolation of information processing paths in terms of structure. This allows the model to truly and fully combine general understanding and specialized knowledge in each response, directly and thoroughly solving the fundamental problems of lengthy responses, deviation from clinical focus, and insufficient professionalism in medical consultations by general large models. More importantly, the evolutionary strategy of "absolute freezing of basic parameters and horizontal expansion of departmental parameters" physically isolates the gradient interference between new and old tasks. Therefore, when continuously injecting multi-departmental medical knowledge, it can fully retain the model's original powerful general dialogue capabilities, thus naturally solving the catastrophic forgetting problem that has long existed in vertical domain models. Complementing this perfectly is the alignment scheme proposed in this paper, which combines a group-relative strategy optimization algorithm with a dual-track hybrid reward function. This reduces the hardware resource requirements for training while employing a collaborative and balancing mechanism guided by factual accuracy anchors and interactive styles, allowing the model output to naturally converge to an optimal balance between the precision of evidence-based medicine and the art of humanized communication. More importantly, the entire architecture achieves extreme computational efficiency due to sparse activation and excellent memory reuse advantages due to parameter freezing. Therefore, a super model carrying massive amounts of cross-departmental medical knowledge can stably and reliably serve high-concurrency online medical dialogue scenarios in the real world with low latency, high throughput, and high availability, representing a significant theoretical breakthrough and engineering application value. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 is a schematic diagram of the overall architecture and workflow of the heterogeneous hybrid expert medical dialogue system proposed in this invention.

[0019] Figure 2 is a schematic diagram of the core principle of the heterogeneous routing mechanism of the present invention, which consists of "4 general experts + 2 departmental experts".

[0020] Figure 3 is a schematic diagram of the alignment training process based on group relative policy optimization and dual-track hybrid reward function of the present invention.

[0021] Figure 4 is an optimized service architecture diagram of the system of the present invention for high-concurrency online inference. Specific implementation methods

[0022] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0023] Example 1

[0024] This embodiment describes the construction and deployment process of a heterogeneous hybrid expert medical dialogue system in the context of online consultation in cardiovascular medicine.

[0025] Step 101: This paper provides a clear and hierarchical explanation of the two parts: the selection of the base model and the preprocessing of cardiovascular data. First, a powerful, general-purpose MoE model is chosen as the base. Then, cardiovascular specialist dialogue data is systematically prepared. Specifically, the open-source model Mixtral-8x7B-v0.1 is selected as the base model, and all its parameters are used as the pre-training parameter set. This process naturally leads to its excellent general language capabilities. Next, data on doctor-patient dialogues in the cardiology department was collected. The data sources included publicly available, anonymized electronic medical record fragments and high-quality content from professional medical Q&A communities. This data was then cleaned and organized into a multi-turn dialogue sample format. Each sample contained multiple rounds of patient complaints and medical history as input (q), and a standard response reviewed by a senior cardiologist and strictly adhering to treatment guidelines as a reference. Finally, the training set, validation set, and test set were divided into a reasonable ratio of 8:1:1.

[0026] Step 102: This paper provides a clear and hierarchical explanation of the methods for heterogeneous model modification and parameter freezing: First, the physical structure of the basic model is adjusted to naturally and reasonably introduce specialized capabilities while preventing forgetting. Specifically, parameter freezing is performed first, setting the weights of all self-attention modules, layer normalization modules, and the original eight homogeneous experts and routers in all Transformer layers of the Mixtral-8x7B-v0.1 model to an untrainable state. Then, structural expansion is performed by adding a "department-specific expert pool" in parallel near the frozen MoE layers in each layer of the model. In this example, this pool consists of four newly initialized expert networks, specifically handling finer-grained cardiovascular knowledge partitioning, and is equipped with a trainable router. This set of newly added parameters is defined as an extended parameter set. Its gradient can be updated.

[0027] Step 103: This paper provides a clear and hierarchical explanation of the supervised fine-tuning method based on heterogeneous routing: First, cardiovascular data is used to train the newly added pool of departmental experts. During the forward propagation of the model, each layer uses heterogeneous routing, that is, the input token is simultaneously fed into both the frozen general router (activating the Top-4 from 8 frozen experts) and the trainable departmental router (activating the Top-2 from 4 departmental experts). The outputs of the two paths are simply added together to obtain the output of that layer. Then, supervised fine-tuning is performed using the data prepared in Step 101 with a standard language modeling objective. The selected optimizer is AdamW, with a learning rate of 3e-5, a batch size of 64, a gradient accumulation step of 4, and training for 3 epochs. Most importantly, in this stage, the gradient of the loss function only updates the expanded parameter set. That is, the department experts and their routers.

[0028] Step 104: This paper optimizes the reinforcement learning alignment method based on GRPO and hybrid reward to improve the accuracy and professionalism of the model's response: First, a dual-track hybrid reward function is designed. ,in, Responses and reference responses are generated using a medical text encoder. semantic similarity; By employing large-scale models such as GPT-4, the generated responses are scored based on a three-dimensional scale of "focus and conciseness," "clinical professionalism," and "humanistic empathy." Then, GRPO training is performed: for the same patient input q, the current strategy model generates four candidate responses in parallel as a group, calculating the mixed reward and within-group statistics for each response, and using this to determine the relative advantage value to update the strategy. More importantly, this training only updates the parameters of the department's expert pool. The KL divergence penalty coefficient β is set to 0.01, and the training takes about 1000 steps until convergence.

[0029] Step 105: This section introduces the method for deploying online high-concurrency services: First, the trained model is formally deployed to the production environment. Then, the advantages of the chosen architecture are fully utilized, namely, during inference, only 6 experts are activated per token per layer, thus keeping the computational overhead constant. Furthermore, because all general parameters are frozen, the general computation portion of a large number of concurrent requests can be naturally and reasonably merged, while reusing the key-value cache (KV Cache) generated in its attention mechanism, resulting in extremely high memory utilization efficiency. Even more remarkable is the adoption of a decoupled service architecture: requests are first processed by a unified general computation module, and then distributed to cardiovascular computing nodes for specialty computation based on departmental routing results.

[0030] Figure 2 This paper demonstrates the main logic of the "dual-pool heterogeneous routing" mechanism proposed in this invention, and also provides a clear comparison of the data flow between the traditional MoE architecture and the architecture of this invention. Therefore, it clearly and rigorously introduces each step of heterogeneous routing: Input splitting: For the current token representation x entering a certain Transformer layer, two copies are first made and sent to the general routing channel and the departmental routing channel respectively. Independent gating calculation: x in the general channel is compared with the frozen routing weight matrix. After multiplying, its value is calculated using Softmax. Probability distribution on a general expert The relationship between x in the departmental passage and the trainable routing weight matrix After multiplying, its value is calculated using Softmax. Probability distribution of experts in each department Top-K expert selection: Choose the channel with the highest probability. =4 experts as the activation expert set The department entrance has the highest probability of being selected. =2 experts as the activation expert set Parallel computing and weighted aggregation: The four activated general experts and two departmental experts compute their respective outputs in parallel, and the resulting output for this layer is the final output. This mechanism strictly separates the general language flow from the specialized medical flow in the computational graph, thus making it the most direct and effective method to solve the problem of unfocused responses.

[0031] Figure 3 The presentation showcases an alignment training process based on GRPO and dual-track rewards, presenting the main steps in an iterative loop: First, patient input and group sampling occur, meaning the training loop begins with a specific patient query context q, from which the current policy model randomly samples and generates G (G = 4) candidate responses. Then comes the dual-track reward calculation, for each candidate response. All responses are entered into a dual-track reward system. Track one is a semantic similarity reward: candidate responses. and gold standard response Both vectors are obtained through a medical encoder, and then the cosine similarity is calculated. Track 2 is the large-scale model referee reward: candidate responses will be awarded... The question q is input into the judge's big data model, which performs structured reasoning based on the given three-dimensional scale (conciseness, professionalism, empathy) and gives a score. Next comes the reward fusion, where the two-track rewards are reasonably weighted and summed according to predetermined weights (0.6 and 0.4) to obtain the result. Next, we perform within-group advantage estimation, which involves collecting the reward for all responses from the group. Calculate the average reward within the group. and standard deviation This defines each reply. Advantages ,like A positive value indicates that the response is better than the current group average. Finally, the policy gradient is updated using the obtained advantage value. The GRPO objective function is constructed by combining importance sampling and KL divergence penalty. Then, the gradient is obtained using backpropagation, but only this gradient is used to update the parameters of the department's expert pool. Therefore, it retains general capabilities while allowing for adjustments to medical interaction strategies.

[0032] Figure 4 This diagram illustrates a system-level optimized architecture for high-concurrency online inference. It explains how a trained model can be efficiently deployed from the perspective of server clusters and request processing: Request reception and distribution: The online service gateway receives a large number of concurrent patient consultation requests and distributes them reasonably and evenly to the general computing service cluster, which has already loaded all the model's frozen parameters. and parameters of the trainable departmental expert pool General Computation and KV Cache Reuse: In the general computing service, the input sequence of each request is first processed through shared general parameters. Since these parameters are identical and immutable, the key-value vectors generated by different requests at the same sequence position can be merged, stored, and reused, naturally forming a shared KV cache pool and greatly alleviating memory bandwidth pressure. Departmental Routing and Decoupled Computation: When general computing reaches the point of generating a response, the forward propagation process performs departmental routing to determine which (Top-2) departmental experts the current request needs to activate. Specialty Computation Distribution: Based on this, the system distributes the intermediate representation of the request and the corresponding departmental expert ID to dedicated departmental computing service nodes. It is worth noting that each departmental computing service node is lightweight, typically loading only one or a few departmental experts. The corresponding departmental expert node returns the result directly after completing the computation. Result Aggregation and Response: After receiving the computation results from each departmental node, the general computing service appropriately aggregates them with the output of the general experts to obtain the final token, which is then streamed back to the user. Therefore, the architecture proposed in this paper can be called "general centralization and specialized distribution", which also balances resource utilization efficiency and response speed well.

[0033] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for optimizing a large language model for high-concurrency online medical dialogue, characterized in that, include: A heterogeneous hybrid expert (MoE) routing layer is constructed, comprising physically isolated general expert pools and department-specific expert pools. The general route and department-specific route are executed in parallel for each input representation, wherein the general route is selected from the general expert pool based on gating scores. Each expert, the department-specific route, is selected from the pool of department-specific experts based on their gating scores. One expert, and ; The pre-trained base MoE model is subjected to parameter evolution training to adapt the model to the medical field. In this process, all original parameters in the base MoE model, except for those of the specialists to be expanded, are completely frozen. These original parameters include the weights of the self-attention layer, the layer normalization layer, and the general expert pool. At the same time, the specialist expert pool and its corresponding routing gating network are horizontally expanded as new parameters, and gradient updates and training are performed only on the new parameters. The Group Relative Policy Optimization (GRPO) algorithm is adopted, and reinforcement learning alignment is performed on the model trained by parameter evolution based on a dual-track hybrid reward function. The dual-track hybrid reward function is a weighted sum of a first reward based on the semantic similarity of the medical encoder and a second reward based on the multi-dimensional structured scoring by the external large language model judge.

2. The method according to claim 1, characterized in that, The output of the heterogeneous hybrid expert routing layer is calculated as: the activated... The weighted output and activation of general experts The weighted sum of the outputs of specialists in each department, among which... and This is a preset fixed value.

3. The method according to claim 2, characterized in that, The The value is 4, the The value is 2.

4. The method according to claim 1, characterized in that, In the parameter evolution training, the initialization method of the department-specific expert pool is random initialization or pre-training initialization based on small-scale medical data.

5. The method according to claim 1, characterized in that, The dual-track hybrid reward function Represented as: ， in, The cosine similarity score between the generated response and the reference standard answer, calculated by a medical encoder. The external judging model provides standardized scores for the generated responses based on a pre-set scoring table that considers focus, clinical expertise, and humanistic care. and These are the preset weighting coefficients.

6. The method according to claim 5, characterized in that, The scoring dimensions used in the external referee model include: The focus and conciseness dimension is used to assess whether the response directly addresses the patient's main complaint and conducts refined medical follow-up questions; the clinical professionalism dimension is used to assess whether the reasoning process and terminology used in the response conform to the clinical guidelines of the specific department; and the humanistic care and empathy dimension is used to assess whether the tone of the response demonstrates appropriate professional care.

7. The method according to claim 1, characterized in that, When training with the GRPO algorithm, multiple candidate responses are sampled under the same patient inquiry prompt to form a group, and the relative advantage value of each candidate response is calculated based on the statistical distribution of the in-group reward calculated by the dual-track hybrid reward function, thereby updating the strategy model parameters.

8. A large language model optimization system for high-concurrency online medical dialogue, characterized in that, include: A heterogeneous routing module is used to implement the heterogeneous hybrid expert routing mechanism of claim 1 in at least one feedforward network layer of the model; The parameter evolution training module is used to execute the parameter expansion and freezing training process as described in claim 1; The model alignment module is used to integrate the GRPO algorithm and dual-track hybrid reward function described in claim 1 to perform reinforcement learning optimization on the model.

Images