Physical separation expert routing network-based scientific calculation and language thinking collaborative reasoning method and system, and storage medium

Abstract

The invention discloses a scientific calculation and language generation collaborative reasoning method and system based on a physical separation hybrid expert architecture and a storage medium, and relates to the technical field of large model and scientific calculation fusion. According to the method, a novel architecture named PiMoE is provided, and intelligent collaboration of a language task and a numerical calculation task on Token granularity is achieved by integrating a frozen high-precision scientific calculation expert module, a text-to-calculation alignment module and a dynamic token router module. Wherein scientific calculation experts pre-train and freeze parameters on specific field data, and calculation precision and interpretability are ensured; the text-to-calculation module learns to map natural language input into numerical representation which can be processed by experts; the token router then dynamically decides, based on context semantics, that each Token should be generated by an expert or LLM. The training process adopts a three-stage decoupling strategy: in the first stage, independently training and freezing an expert model; in the second stage, a text-numerical value alignment module is optimized; and in the third stage, a router is trained to realize dynamic scheduling of experts and LLMs. During reasoning, the system is seamlessly switched between language generation and scientific calculation according to the semantic context, so that high precision of complex calculation is guaranteed, and semantic reasoning and generation capabilities of LLM are kept. According to the method, the problems that a large model is insufficient in precision, uncontrollable and unextensible in a scientific calculation scene are effectively solved, and deep fusion and dynamic collaboration of language understanding and numerical reasoning are realized.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence and scientific computing integration technology, specifically to a scientific computing and language generation collaborative reasoning method, system, electronic device and storage medium based on a physically separated hybrid expert architecture. It is particularly suitable for dynamically integrating high-precision numerical computing capabilities in large language models (LLM) to achieve intelligent collaboration and seamless switching between language understanding and scientific computing at the token granularity. Background Technology

[0002] Currently, Large Language Models (LLMs) demonstrate powerful capabilities in natural language understanding, text generation, and dialogue interaction, and are widely used in education, healthcare, industry, scientific research, and other fields. However, LLMs suffer from inherent limitations when handling tasks requiring high-precision numerical computation (such as physics simulation, engineering modeling, financial pricing, and battery health assessment), including insufficient computational accuracy, uncontrollable results, and a lack of interpretability. The fundamental reason for this is that LLMs rely on probabilistic modeling for token prediction, making it difficult to accurately fit the rigorous functional mapping relationships required in scientific computing.

[0003] In existing technologies, some solutions attempt to introduce computing power through Prompt projects, tool calling, or external API access, but these generally suffer from the following problems: The computation is separated from the language model reasoning process, making it impossible to achieve semantically driven dynamic invocation; Uncontrollable accuracy: The calculation results depend on an external black-box system and lack an accuracy guarantee mechanism during the training phase; Poor scalability: Adding new computing power requires restructuring the system architecture, making it difficult to support dynamic hot-swapping of expert modules; Contextual discontinuity: The computational results are difficult to integrate naturally into the language generation process, leading to a break in subsequent semantic reasoning.

[0004] Therefore, there is an urgent need for a new architecture and method that can deeply integrate high-precision scientific computing capabilities into large language models, support dynamic collaborative reasoning, and ensure computational accuracy and semantic coherence. Summary of the Invention

[0005] In view of this, this disclosure provides a scientific computing and language thinking collaborative reasoning method, system, electronic device and storage medium based on a physically separated expert routing network. It aims to solve the technical problems of insufficient accuracy, uncontrollability, non-scalability and semantic fragmentation of existing large language models in scientific computing tasks, and realize the deep integration and dynamic collaboration of language understanding and numerical reasoning at the token granularity.

[0006] First aspect: Collaborative reasoning methods The first aspect of this disclosure provides a method for collaborative reasoning in scientific computing and language thinking based on a physically separated expert routing network, the method comprising: The PiMoE architecture is constructed, which includes: a high-precision scientific computing expert module with a set of frozen parameters, a text-to-computation alignment module, and a token router module; During the training phase, a step-by-step decoupled training strategy is adopted: In the first stage, the scientific computing expert module is independently trained based on numerical input-output pairs from scientific or industrial fields, and its parameters are frozen after convergence. In the second stage, the text-to-computation alignment module is trained to map the natural language input into a numerical representation compatible with the scientific computing expert module. In the third stage, the token router module is trained to output the call probability distribution between the expert module and the large language model (LLM) based on the hidden representation of the current token. During the reasoning phase, based on the current semantic context, the token router module dynamically decides whether each token is generated by the scientific computing expert module or the LLM, enabling alternating calls and collaborative reasoning of language generation and scientific computing at the token granularity.

[0007] Beneficial effects: The collaborative reasoning method provided in this disclosure achieves the organic integration of scientific computing capabilities and language modeling capabilities by constructing a three-module architecture of "expert freezing + alignment mapping + dynamic routing". The step-by-step decoupling training strategy effectively isolates the heterogeneous optimization objectives of numerical computation and language modeling, improving training stability; freezing expert parameters ensures computational accuracy and interpretability; and the token router enables semantically driven token-level dynamic scheduling, allowing the system to maintain language fluency when computation is not required and automatically trigger high-precision experts when computation is needed. The computation results are seamlessly integrated into subsequent language generation, forming a closed-loop reasoning chain of "semantics → computation → semantics". This method significantly improves the accuracy, controllability, and scalability of large models in scientifically computation-intensive tasks.

[0008] The second aspect: Collaborative reasoning system The second aspect of this disclosure provides a scientific computing and language-based collaborative reasoning system based on a physically separated expert routing network, comprising: Scientific computing expert module: used to perform high-precision numerical calculations, parameters are frozen after training, and dynamic expansion by domain is supported; Text-to-Computation Alignment Module: Used to convert natural language input into a numeric format that can be processed by experts; Token Router Module: Used for dynamically deciding to invoke expert modules or LLMs at the token granularity; Integrated Inference Engine: Used to coordinate the alternating generation of tokens by the expert module and LLM based on router decisions during inference, achieving seamless integration of language and computation; The system is trained and deployed using the aforementioned collaborative reasoning method.

[0009] Beneficial effects: This system architecture supports modular design and dynamic expansion. Each component has a clear responsibility and low coupling, facilitating independent optimization and hot-swappable upgrades. During inference, the system can intelligently identify computational needs in the semantic context, achieving "seamless switching." It retains the powerful language generation and inference capabilities of LLM while endowing it with high-precision, interpretable, and verifiable scientific computing capabilities, making it suitable for scenarios with dual requirements for computational accuracy and semantic coherence, such as industrial simulation, scientific research assistance, and intelligent diagnostics.

[0010] Third aspect: Electronic devices A third aspect of this disclosure provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the above-described collaborative reasoning method and / or run the above-described collaborative reasoning system.

[0011] Fourth aspect: Computer-readable storage media The fourth aspect of this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to execute the above-described collaborative reasoning method and / or deploy the above-described collaborative reasoning system.

[0012] The beneficial effects of electronic devices and computer-readable storage media, corresponding to the beneficial effects of collaborative reasoning methods and systems, will not be elaborated upon here. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0014] Figure 1 This is a flowchart illustrating the first PiMoE model training method according to an embodiment of the present disclosure; Figure 2 This is a flowchart illustrating the second PiMoE model training method according to an embodiment of the present disclosure; Figure 3This is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0016] In some applications, Large Language Models (LLMs) are widely used for natural language understanding and generation tasks. However, in tasks involving scientific computing, such as battery health assessment, thermodynamic modeling, and financial risk prediction, LLMs, due to their probabilistic language modeling mechanism, often struggle to provide high-precision, interpretable, and stable numerical results. Current technologies typically rely on external tools or manual intervention to complete the computation, resulting in a disconnect between the computation process and language reasoning, failing to achieve semantically driven closed-loop reasoning. Therefore, there is an urgent need for a novel architecture and method that can deeply integrate high-precision scientific computing capabilities into the large language model and support dynamic, intelligent scheduling.

[0017] In view of the above, according to the embodiments of this disclosure, a scientific computing and language thinking collaborative reasoning method and system based on a physically separated expert routing network is provided. The specific implementation method is described in detail below with reference to the accompanying drawings.

[0018] I. Overall Architecture Overview like Figure 1 As shown in the embodiments of this disclosure, a scientific computing and language thinking collaborative reasoning system based on a physically separated expert routing network is provided, comprising three core modules: High-precision scientific computing expert module: a set of dedicated models that are independently pre-trained in specific fields (such as battery life prediction, material performance simulation, etc.). The parameters are frozen after training to ensure their computational accuracy and interpretability. Text-to-computation alignment module: responsible for converting natural language input into a numerical representation that can be processed by expert models; The Token Router module dynamically determines whether each token is generated by the expert module or the Large Language Model (LLM) based on the current context semantics during inference.

[0019] The three modules mentioned above together constitute the PiMoE (Probabilistic Integration of Models and Experts) architecture. They are integrated into a unified large model framework through neural network connections, enabling seamless switching and collaborative reasoning between language generation and scientific computing at the token granularity.

[0020] II. Step-by-step training method To effectively decouple the heterogeneous optimization objectives between numerical computation and natural language modeling and improve training stability, this disclosure adopts a three-stage step-by-step training strategy, as follows: Phase 1: Scientific Computing Expert Module Training like Figure 2 As shown in (1), multiple domain expert models are first trained independently based on real-world scientific or industrial datasets (such as battery charge-discharge cycle records, temperature change curves, etc.).

[0021] The input is a numerical input-output pair, for example: [Usage period = 3 years, Temperature = 40℃, Number of cycles = 500] → [Health = 0.87]; The output is the corresponding target numerical result; The loss function uses mean squared error (MSE) to minimize the difference between the predicted and the true values; After the expert model converges, its parameters are frozen to ensure high accuracy and stability during subsequent joint training and inference.

[0022] After this training phase is completed, a set of reusable, high-precision, and interpretable scientific computing expert modules is obtained, which supports dynamic expansion.

[0023] Phase Two: Training of the Text-to-Computational Alignment Module like Figure 2 As shown in (2), based on the expert module parameter freeze, the text to computation alignment module is trained so that it can map the language computation task described by natural language to the numerical input format required by the expert model.

[0024] The input is a computation request described in natural language, such as: "Please calculate the health status of the battery based on its age, ambient temperature, and number of charge / discharge cycles"; The output is a normalized numerical vector, for example: [3, 40, 500]; The training objective is to minimize the MSE loss, that is, to make the numerical vector output by the alignment module as close as possible to the real input data; The alignment module can be built on the Transformer structure to learn the mapping relationship from semantics to numerical values.

[0025] This training phase achieves efficient alignment between language input and numerical computation, providing accurate computational input for subsequent reasoning.

[0026] Phase 3: Training the Token Router Module like Figure 2 As shown in (3), after the first two modules are trained, the token router module is trained to implement the dynamic decision-making mechanism.

[0027] The input is the hidden state representation of all tokens at the current time step; The output is the probability distribution between the set of expert modules and the large language model, indicating which module should generate the next token; Optimization is performed using the Cross-Entropy (CE) loss function; The training data comes from language-computation task sequences in real-world scenarios, such as: "Battery health is" → requires calling an expert, "Daily usage status is good" → can be generated by LLM.

[0028] Through this training phase, the token router can automatically determine whether to call the expert module based on the context semantics, thereby achieving intelligent scheduling.

[0029] III. Reasoning Process and Paradigm like Figure 3 As shown, the specific process of the collaborative reasoning method provided in this embodiment during the reasoning phase is as follows: Step S101: Input text token sequence It receives natural language text input from the user and encodes it into a token sequence using a word segmenter.

[0030] Step S102: Invoke the token router Input the current token sequence into the token router to obtain the probability distribution of calling the expert module or LLM.

[0031] Steps S103 and S104: Branch Execution If the router's output tends to invoke the expert module, then: Call the text-to-calculation alignment module to parse the current semantics into numerical input; Call the corresponding expert model to perform high-precision calculations and output numerical results; Convert the calculation result into a token (such as "0.95") and add it to the output sequence.

[0032] If the router output tends to call LLM, then: Use the large language model to generate the next token; Add the generated token to the output sequence.

[0033] Steps S105 and S106: Update the sequence and continue reasoning. Regardless of which module is called, the newly generated token will be added to the output sequence and enter the inference loop of the next time step.

[0034] Step S107: Continue reasoning until the end. Repeat the above process until a complete statement is generated or a terminator is encountered.

[0035] IV. Typical Application Scenarios Taking "battery health assessment" as an example, the practical application effect of the embodiments of this disclosure is illustrated: Suppose the input is: "the Battery health is".

[0036] Step 1: The token router analyzes the context, identifies "health", and then outputs the specific value. Step 2: Call the text to the calculation module to extract implicit information such as "battery lifespan, temperature, and cycle count"; Step 3: Call the battery health expert model and calculate the health score as 0.95; Step 4: Insert "0.95" as the token into the output sequence to get "the Battery health is 0.95"; Step 5: The subsequent LLM is generated based on this result: "The daily usage status is relatively good". The entire process achieves a closed-loop reasoning of "semantics → computation → semantics", which ensures both computational accuracy and linguistic fluency.

[0037] V. Technological Advantages and Beneficial Effects The scientific computing and language-based collaborative reasoning method and system based on physically separated expert routing networks provided in this disclosure have the following significant advantages: High-precision calculation guarantee: By freezing the expert model parameters, we ensure that the calculation process is not affected by LLM training, thus maintaining high accuracy and interpretability; Dynamic intelligent scheduling: The token router makes dynamic decisions based on semantic context, avoiding unnecessary calls and improving inference efficiency; Highly scalable: Expert modules can be trained independently and hot-swapped, supporting rapid deployment of scientific computing tasks across multiple domains; Good semantic continuity: The calculation results are naturally integrated into the language generation process, forming a closed-loop reasoning chain and improving the user experience; Stable and efficient training: The three-stage step-by-step training strategy effectively isolates heterogeneous optimization targets and improves training convergence speed and stability; High resource utilization: Expert modules are invoked only when needed, reducing computing power consumption and making them suitable for edge device deployment. Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present disclosure.

[0038] The following is a detailed reference. Figure 3 The diagram illustrates a structural block diagram suitable for implementing an electronic device according to embodiments of the present disclosure. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 301, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 302 or a program loaded from memory 308 into random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device. The processor 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0039] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0040] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from memory 308, or installed from ROM 302. When the computer program is executed by processor 301, it performs the functions defined in the reward model training method and / or problem evaluation method of embodiments of this disclosure.

[0041] Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0042] This disclosure also provides a computer-readable storage medium in which the methods described in this disclosure can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the reward model training method and / or problem evaluation method shown in the above embodiments are implemented.

[0043] A portion of this disclosure can be applied to computer program products, such as computer program instructions, which, when executed by a computer, can invoke or provide methods and / or technical solutions according to this disclosure through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, and installation package files. Accordingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions; the computer compiling the instructions and then executing the corresponding compiled program; the computer reading and executing the instructions; or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0044] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A scientific computing and language generation collaborative reasoning method based on a physically separated hybrid expert architecture, characterized in that, The method comprises: constructing a PiMoE architecture, which comprises a set of high-precision scientific computing expert modules with frozen parameters, a text-to-computation alignment module, and a token router module; In the training phase, a step-by-step decoupling training strategy is adopted: In the first stage, based on the numerical input and output pairs of scientific or industrial fields, the scientific computing expert modules are independently trained, and their parameters are frozen after convergence; In the second stage, the text-to-computation alignment module is trained to map natural language input to numerical representation compatible with the scientific computing expert modules; In the third stage, the token router module is trained to output the calling probability distribution between the expert module and the large language model (LLM) based on the hidden representation of the current Token; In the inference phase, according to the current semantic context, the token router module dynamically decides whether each Token is generated by the scientific computing expert module or the LLM, realizing the alternating calling and collaborative reasoning of language generation and scientific computing at the Token granularity.

2. The collaborative reasoning method of claim 1, wherein, The scientific computing expert modules are multiple, pre-trained for different scientific computing subfields, and their parameters remain frozen during PiMoE training and inference to maintain their computing accuracy, interpretability, and dynamic scalability.

3. The collaborative reasoning method of claim 1 or 2, wherein, The training target of the text-to-computation alignment module is to minimize the mean square error (MSE), and its input is the calculation task described in natural language, and the output is a standardized numerical input vector suitable for the scientific computing expert modules.

4. The collaborative reasoning method of any one of claims 1 to 3, wherein, The token router module is optimized by a cross-entropy loss function, and its input is the hidden state of all Tokens at the current time step, and its output is the probability distribution of the expert module set and the LLM set, which is used to guide the selection of the generation source of the next Token.

5. The collaborative reasoning method of any one of claims 1 to 4, wherein, In the inference process: When the current semantic context does not require numerical calculation, the token router module routes the Token generation task to the LLM to perform standard language modeling for the next Token prediction; When high-precision calculation is detected, the token router module triggers the text-to-computation alignment module to convert the language input into numerical input, then calls the scientific computing expert module to generate the calculation result, and inserts the result as a new Token into the output sequence for subsequent LLM semantic reasoning and generation.

6. The collaborative reasoning method of claim 5, wherein, The calculation result is regarded by the LLM as part of the context for the semantic reasoning, interpretation, or planning of subsequent sentences, realizing the "computation-language" closed-loop generation.

7. The collaborative reasoning method of any one of claims 1 to 6, wherein, All modules in the PiMoE architecture are based on the same basic large language model architecture (such as Qwen2.5-0.5B), and the total parameter quantity is the sum of the parameters of each module. In the current embodiment, the total parameter quantity is 1.5B.

8. A scientific computing and language collaborative reasoning system, comprising: It comprises: scientific computing expert modules for performing high-precision numerical calculations, with parameters frozen after training; a text-to-computation alignment module for converting natural language input into numerical format that can be processed by experts; a token router module for dynamically deciding whether to call the expert module or the LLM at the Token granularity; An integrated inference engine is used to coordinate the generation of tokens by the expert modules and the LLM alternately according to the router decision during inference, realizing seamless integration of language and calculation. The system is trained and deployed by the method of any one of claims 1 to 7.

9. An electronic device, comprising: Comprise: A memory and a processor, which are connected in communication with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the collaborative inference method of any one of claims 1 to 7, and / or runs the collaborative inference system of claim 8.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for making a computer execute the collaborative inference method of any one of claims 1 to 7, and / or deploy the collaborative inference system of claim 8.

Images