A personalized science knowledge question-answering method and system

Abstract

The application discloses a personalized science and technology knowledge question answering method and system. The method firstly acquires and processes science and technology literature data, constructs a field basic model training corpus for further pre-training, and forms a field basic model. Then, based on the field basic model, the parameters of a sampling model are initialized, initial training is performed by using seed data, and the sampling model is trained through multi-round iteration reinforcement learning to generate sample data. Then, based on the field basic model, the parameters of a question answering model are initialized, a distillation training data set is constructed by using the sample data generated by the sampling model, and supervised fine-tuning training is performed until the model converges. Finally, the trained question answering model is encapsulated into an online inference interface service to construct a science and technology knowledge question answering system. The application realizes science and technology knowledge question answering services for personalized user needs, and significantly improves the efficiency and accuracy of problem solving.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to the fields of natural language processing and deep learning, and in particular to a personalized scientific and technological knowledge question-answering method and system. Background Technology

[0002] With the continuous progress of science and technology in my country, the demand for acquiring scientific and technological knowledge is increasing daily. Natural language processing technology plays an increasingly important role in scientific and technological text retrieval for professionals in these fields. Achieving accurate and efficient access to scientific and technological intelligence has become a crucial task in the scientific and technological community.

[0003] The highly specialized nature, complex text structures, and rich semantic content of the science and technology field present users with certain difficulties in accessing scientific and technological literature. Therefore, establishing a science and technology knowledge question-and-answer system based on textual corpora from professional books, journals, papers, patents, etc., integrating scientific and technological knowledge, catering to the access preferences of different user groups, and focusing on domain-specific scientific and technological information retrieval has become an important research topic.

[0004] Common personalized science and technology knowledge Q&A systems are generally based on recommendation methods. By establishing a relationship between users and knowledge, answers are proactively delivered to users through recommendations.

[0005] In public-facing scientific and technological knowledge question systems, which involve numerous professional fields and a large number of users, traditional knowledge-answering systems struggle to effectively match domain knowledge with user characteristics and search preferences. During the scientific and technological knowledge retrieval process, users face challenges due to the sheer volume, complexity, and difficulty in reading various types of scientific and technological knowledge. Furthermore, users often have specific professional backgrounds and clear task objectives. Search results in specialized fields are typically highly relevant to the user's technical area and research interests, while general search results suffer from poor alignment with user interests. This results in generally low relevance and accuracy of professional text comprehension and search results. Consequently, existing recommendation-based search systems often struggle to find and acquire scientific and technological knowledge that meets user needs, lack effective means of acquiring such knowledge, and the interaction process often requires significant time and effort, failing to meet users' needs for effective access to scientific and technological knowledge in the face of new problems, cross-disciplinary issues, and cutting-edge technological scenarios. Summary of the Invention

[0006] This application provides a personalized scientific and technological knowledge question-answering method and system. Based on training a domain-specific model using scientific literature data, it uses retrieval history as seed data and trains a sampling model through reinforcement learning in a multi-round iterative manner. Then, it generates sample data through sampling and fine-tunes the large-scale question-answering model through data distillation. During use, it fully utilizes user characteristics and conversation history as input to the question-answering model with few-sample prompts, dynamically generating personalized answers through thought chain reasoning, thus significantly improving the efficiency and accuracy of question answering. This application overcomes the difficulties of low retrieval efficiency in existing recommendation-based knowledge question-answering systems and solves the technical problem of low user satisfaction.

[0007] Firstly, a personalized scientific and technological knowledge question-answering method is provided, comprising four steps: domain-based model construction, sampling model training, question-answering model training, and question-answering system construction. Specifically, it includes:

[0008] Domain foundation model construction: Acquire and process scientific and technological literature data, construct a training corpus for the domain foundation model, and continue pre-training based on the open-source pre-trained model using the training corpus to construct the domain foundation model;

[0009] Sampling model training: Based on the domain base model, initialize the parameters of the sampling model, use seed data to initially train the sampling model, train the sampling model through multiple rounds of iterative reinforcement learning, and generate sample data;

[0010] Question answering model training: Based on the domain-based model, initialize the parameters of the question answering model, construct a distillation training dataset using sample data generated by the sampling model, and perform supervised fine-tuning training on the question answering model until convergence;

[0011] Question Answering System Construction: The trained question answering model is encapsulated into an online inference interface service to build a scientific and technological knowledge question answering system. During the use of the question answering system, user characteristics and conversation history are input into the question answering model to dynamically generate personalized answers.

[0012] Optionally, in the domain foundation model construction step, processing scientific and technological literature data includes deduplicating the data, removing HTML / XML tags and non-text content, and splitting the text data into title, abstract, background, problem, technical solution, and effect sections.

[0013] Optionally, in the sampling model training step, the seed data includes user information and the user's historical retrieval data. The user information includes the user's name, age, occupation, company, and industry information registered when the user opened the account. The historical retrieval data includes user queries, returned results, and user feedback information data through clicks or browsing.

[0014] Optionally, the sampling model training step, which involves training the sampling model through multiple rounds of reinforcement learning, includes: initializing the model with domain-based model parameters at the beginning of each iteration, generating multiple candidate answers to the user question using the current sampling model, scoring the candidate answers by combining rule validation and large model evaluation, and updating the parameters of the sampling model using the online GRPO reinforcement learning algorithm.

[0015] Optionally, the parameters of the sampling model are updated using the online GRPO reinforcement learning algorithm, specifically through the formula:

[0016]

[0017] Training is performed, where θ represents the sampling model parameters; q represents the query input, which follows a P(Q) distribution; J GRPO (θ) represents the optimization objective; G is the number of sample groups, with a value of 16; π θ A new strategy for sampling models; For the old strategy of the sampling model; π ref The reference model strategy initially uses the domain-based model, and subsequent iterations use the model sampled from the previous round; `clip` is the clipping function; o i Output for the i-th sample; o i，t Output for the i-th sample at time t; i，＜t The output of the i-th sample before time t; ∈ is the shearing range hyperparameter; β is the KL divergence weight coefficient; D KL KL divergence distance:

[0018]

[0019] Calculated using relative reward values ​​within the group: To normalize the reward, r i Let r be the reward for the i-th sample, r represent the reward variable across all samples, mean(r) be the mean of variable r, and std(r) be the standard deviation of variable r.

[0020] Optionally, in the question-answering model training step, constructing a distillation training dataset using sample data generated by the sampling model includes: performing rejection sampling filtering on the sample data to remove samples with incorrect format or scores below a threshold, combining multiple historical dialogues of the same user in chronological order of sampling time into synthetic sample data, and concatenating it with the current question to generate prompt words as model input.

[0021] Secondly, a personalized scientific and technological knowledge question-answering system, comprising a domain-based basic model construction module, a sampling model training module, a question-answering model training module, and a question-answering system construction module, specifically including:

[0022] The domain foundation model construction module is used to acquire and process scientific and technological literature data, construct domain foundation model training corpus, and further pre-train based on open-source pre-trained models using the training corpus to construct the domain foundation model.

[0023] The sampling model training module is used to initialize the parameters of the sampling model based on the domain base model, perform initial training on the sampling model using seed data, train the sampling model through multiple rounds of iterative reinforcement learning, and generate sample data.

[0024] The question-answering model training module is used to initialize the parameters of the question-answering model based on the domain base model, construct the distillation training dataset using sample data generated by the sampling model, and perform supervised fine-tuning training on the question-answering model until convergence.

[0025] The question-answering system building module is used to encapsulate the trained question-answering model into an online inference interface service to build a scientific and technological knowledge question-answering system. During the use of the question-answering system, user characteristics and conversation history are input into the question-answering model to dynamically generate personalized answers.

[0026] Thirdly, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the personalized scientific and technological knowledge question-and-answer method described in any of the first aspects above.

[0027] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the personalized scientific and technological knowledge question-and-answer method described in any of the first aspects above.

[0028] Fifthly, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the personalized scientific and technological knowledge question-and-answer method described in any of the first aspects above.

[0029] This invention provides a personalized knowledge question-answering system based on a large model, comprehensively applying various training datasets and model training methods. A domain-based foundational model is trained using scientific literature data, enabling the model to possess domain knowledge recognition capabilities. Using retrieval history as seed data solves the cold-start problem caused by a lack of labeled data in the sampling model. After training the sampling model through reinforcement learning in a multi-round iterative mode, sample data is generated, producing a large amount of training data for training deep thinking models, thus enabling the sampling model to possess domain logical reasoning capabilities. The sampling model, through reinforcement learning using the online GRPO algorithm, avoids model degradation caused by overfitting of the value function, efficiently learning discrete reward signals in single-round interaction data, and focusing on the current user's current intent recognition model optimization scenario. This question-answering model, fine-tuned using data distillation, inputs user characteristics and intentions into the model via a few-shot approach. Through few-shot reasoning, it can perceive the static characteristics and dynamic intentions of the current user, dynamically generating user question-answering context using thought chain reasoning to produce personalized answers. It can dynamically adapt to the diverse user characteristics and changing usage intentions in the question-answering system. The generalization ability of the question-answering model meets the personalized preferences of various types of users in real-world usage scenarios, satisfies users' personalized scientific and technological knowledge acquisition needs, and improves the efficiency and accuracy of question answering in a targeted manner. Attached Figure Description

[0030] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0031] Figure 1 The main flowchart provided for the embodiments of this application;

[0032] Figure 2 This is a schematic diagram of the sampling model training data format provided in the embodiments of this application;

[0033] Figure 3 A schematic diagram illustrating the composition of training data provided in the embodiments of this application;

[0034] Figure 4 This is a schematic diagram of inter-model dependencies provided in the embodiments of this application;

[0035] Figure 5 A flowchart for a personalized science and technology knowledge question-and-answer system;

[0036] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0038] In the description of this application, the terms “comprising,” “having,” and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are expressly listed, but may also include other steps or units that are not expressly listed but are inherent to these processes, methods, products, or devices, or steps or units added based on further optimizations of the inventive concept.

[0039] To overcome the low retrieval efficiency and low user satisfaction of existing recommendation-based knowledge question-answering systems, this invention provides a personalized knowledge question-answering system based on a large model. It trains a domain-based model using scientific literature data, employs retrieval history as seed data, and trains a sampling model through reinforcement learning in a multi-round iterative manner. Sample data is then generated through sampling, and the large question-answering model is fine-tuned through data distillation. During use, user characteristics and conversation history are fully incorporated into the question-answering model through few-sample prompts, dynamically generating personalized answers through thought chain reasoning, thus effectively improving the efficiency and accuracy of question answering.

[0040] In one embodiment, such as Figure 1 As shown, a personalized scientific and technological knowledge question-answering method and a personalized knowledge question-answering system based on a large model are provided. This system employs data distillation to construct various training data and implements training through a multi-stage, multi-model cumulative approach. The system consists of a domain-based basic model, a sampling model, a question-answering model, and a question-answering system.

[0041] The training corpus for the domain-based model is generated from public and private scientific literature texts and is built upon a large open-source model through further pre-training.

[0042] The sampling model is based on the domain-specific model, uses retrieval history as seed data, and is trained through reinforcement learning in a multi-round iterative self-improvement mode.

[0043] The question-answering model is based on a domain-specific foundational model, uses samples generated by a sampling model as training data, and is fine-tuned through data distillation.

[0044] The question system encapsulates the question-answering model into an online reasoning interface service to build a scientific and technological knowledge question-answering system.

[0045] The method includes: domain foundation model construction: acquiring and processing scientific and technological literature data, constructing a domain foundation model training corpus, and using the training corpus to continue pre-training based on an open-source pre-trained model to construct the domain foundation model;

[0046] Sampling model training: Based on the domain base model, initialize the parameters of the sampling model, use seed data to initially train the sampling model, train the sampling model through multiple rounds of iterative reinforcement learning, and generate sample data;

[0047] Question answering model training: Based on the domain-based model, initialize the parameters of the question answering model, construct a distillation training dataset using sample data generated by the sampling model, and perform supervised fine-tuning training on the question answering model until convergence;

[0048] Question Answering System Construction: The trained question answering model is encapsulated into an online inference interface service to build a scientific and technological knowledge question answering system. During the use of the question answering system, user characteristics and conversation history are input into the question answering model to dynamically generate personalized answers.

[0049] The specific process is as follows: Figure 1 As shown:

[0050] The domain foundation model is constructed by pre-training on an open-source foundation model using scientific literature data. The training corpus for the domain foundation model is built upon the acquisition, cleaning, and segmentation of scientific literature. Acquisition involves classifying and organizing publicly available professional books, journal articles, blog posts, and private journal articles and patent data. Cleaning involves deduplication, removing HTML / XML tags, and non-textual content. Segmentation further divides the original text into parts such as title, abstract, background, problem, technical solution, and effects. The above text data is then formatted to form the training corpus for the domain foundation model. The domain model is trained on the domain corpus data using a further pre-training method based on the open-source pre-trained model. The open-source pre-trained model can be a QWen 2.5 series model.

[0051] The sampling model is obtained by training the domain-based model through iterative reinforcement learning.

[0052] The sampling model is trained using a multi-round iterative approach. Each iteration begins with initialization using parameters from the domain-specific foundational model. The initial training data is seeded from data collected from online retrieval systems. In subsequent iterations, the sampling model generates training data for the next stage through a self-improving process. Specifically, the sampling model generates multiple candidate answers for the same user question, and the system scores these candidate answers using a combination of rule-based validation and large-scale model evaluation. The large-scale model can be evaluated using Deepseek R1, accessible via an online API.

[0053] The cold-start seed data used during model initialization training is a preference training dataset constructed based on historical data from an online retrieval system. This seed data is obtained by collecting historical search data from users within the existing online retrieval system and linking it to user information. User information includes the user's name, age, occupation, company, and industry registered when they opened their account. Historical search data includes user queries, returned results, and user feedback information such as clicks or browsing. Users input free text into the online retrieval system, which returns a fixed number of search results per page. Users obtain answers by clicking and browsing. The preference training dataset is constructed by ranking and scoring the system's feedback results based on user feedback.

[0054] The training data for the sampling model consists of question-and-answer pairs from a complete user conversation. By designing prompts, the model is guided to first generate user characteristic descriptions in a thought-chain manner, and then combine these descriptions with the current question to generate an answer. Prompts include requiring the model to output content in a specified format and placing deep thought processes and answers within different specific labels. For example, the model can be asked to place the thought process within... <think>< / think> Inside the tag, place the answer. <answer>< / answer> Within the label section, the generated sample data is evaluated and ranked using a scoring function. This scoring function includes rule-based output format validation and scoring the sample data using a large model.

[0055] The parameters of the sampling model are updated using the online GRPO reinforcement learning algorithm, specifically through the formula:

[0056]

[0057] Training is performed, where θ represents the sampling model parameters; q represents the query input, which follows a P(Q) distribution; J GRPO (θ) represents the optimization objective; G is the number of sample groups, with a value of 16; π θ A new strategy for sampling models; For the old strategy of the sampling model; π refThe reference model strategy initially uses the domain-based model, and subsequent iterations use the model sampled from the previous round; `clip` is the clipping function; o i Output for the i-th sample; o i，t Output for the i-th sample at time t; i，＜t The output of the i-th sample before time t; ∈ is the shearing range hyperparameter; β is the KL divergence weight coefficient; D KL KL divergence distance:

[0058]

[0059] Calculated using relative reward values ​​within the group: To normalize the reward, r i Let r be the reward for the i-th sample, r represent the reward variable across all samples, mean(r) be the mean of variable r, and std(r) be the standard deviation of variable r.

[0060] Figure 2 This describes the training data format for the sampling model. The data consists of prompts, user information, questions, and answers. The prompts include role settings and instructions. The answer section comprises multiple candidate answers to the same question, along with their scores; each group contains N answers and scores. N can be chosen as 16.

[0061] The question-answering model is trained based on a domain-specific foundational model. A distillation training dataset is constructed using sample data generated by a sampling model, and then trained through supervised fine-tuning. The sample data generated by the sampling model is filtered through rejection sampling. Multiple historical dialogues from the same user are grouped in chronological order of sampling time to generate synthetic sample data, which is then concatenated with the current question to generate prompts as model input. Rejection sampling employs rule-based format validation and result evaluation methods, discarding incorrectly formatted outputs and samples with scores below a specified threshold.

[0062] The data distillation training process for the sampling model and the question-answering model is shown in Algorithm 1:

[0063] Algorithm 1: Training process for sampling model and question answering model:

[0064] Input: Seed training dataset {Mtrain}, observation o∈O, action a∈A, reward r∈R;

[0065] Neural network parameters of the sampling model These are the model parameters for the i-th iteration;

[0066] The neural network parameters θ of the question-answering model;

[0067] The neural network parameters Ψ of the domain-based model;

[0068] Reward scoring function F;

[0069] The number of training data samples N for the question-answering model.

[0070] Output: The neural network parameters θ of the question-answering model.

[0071] step:

[0072] Initialize the training dataset D as the seed training dataset {Mtrain}.

[0073] Looping i from 1 to N, iteratively perform sampling model training and update neural network parameters. Until convergence:

[0074] (1) Initialize the sampling model parameters to the domain-based model parameters:

[0075] (2) Randomly sample data from the training dataset D.

[0076] hT(i) = (o0,a0,r0,o1,a1,r1,…,oT,aT,rT)i, construct the training dataset Mitrain.

[0077] (3) Apply the reinforcement learning algorithm GRPO to train the sampling model on the training dataset Mitrain until... Converging to the optimal strategy.

[0078] (4) Clear the training dataset D.

[0079] (5) Using sampling models Perform sampling to generate sample data, score the samples using a reward scoring function, and add the sample data to dataset D.

[0080] Train the question-answering model, updating the neural network parameters θ until convergence:

[0081] (1) Initialize the question-answering model parameters to the domain basic model parameters: θ = Ψ.

[0082] (2) Iterate through the training dataset D, randomly sampling N sample data hT(i) according to user group each time, sorting them by time and combining them into sample data:

[0083] HT(j) = (hT(1),hT(2),…,hT(N-1),hT(N))j, and store it in the training dataset Mtrain.

[0084] (3) Apply the supervised fine-tuning algorithm to train the question answering model on the training dataset Mtrain and update the parameter θ until θ converges to the optimal value.

[0085] (4) Save the neural network parameters θ of the question-and-answer model.

[0086] Figure 3 This diagram illustrates the data relationship between the sampling model and the question-answering model. The sampling model uses reinforcement learning to train based on a single user question-answer data point, while the question-answering model combines multiple data points from the same user instance before training. In the diagram, a, o, and r represent the action, observation, and reward in reinforcement learning, respectively, corresponding to the token output in the larger model. The question system encapsulates the question-answering model into an online inference interface, providing users with scientific and technological knowledge question-answering services.

[0087] Figure 4 This describes the logical relationships between the domain foundation model, sampling model, question-answering model, their training data, and the question-answering system within the system. The domain foundation model is built upon a corpus of scientific and technological literature and serves as the basis for the sampling model (in each iteration) and the question-answering model, initialized through parameter replication. The sampling model employs a multi-round iterative training pattern. Initialization (stage 1) uses seed data, and subsequent iterations (stage i) use data sampled from the previous iteration (stage i-1). The training data for the large question-answering model is generated from the sampling data of the last iteration (N) version of the sampling model. The question-answering system is implemented based on the inference services provided by the large question-answering model.

[0088] In summary, this invention provides a personalized knowledge question-answering system based on a large model, employing various training data construction methods and implementing multi-stage, multi-model cumulative training. It uses a domain-based basic model trained with scientific literature data as the foundation for subsequent sampling and question-answering models. Retrieval history is used as seed data, and sample data is generated after training the sampling model through reinforcement learning in a multi-round iterative manner. The large question-answering model is then fine-tuned through data distillation. During use, user characteristics and objectives are fully incorporated into the question-answering model through few-sample prompts. Personalized question-answering answers are generated dynamically using a thought chain reasoning approach, thus realizing a personalized scientific knowledge question-answering system.

[0089] In one embodiment, such as Figure 5 As shown, a personalized scientific and technological knowledge question-answering system is provided. The system includes a domain basic model construction module, a sampling model training module, a question-answering model training module, and a question-answering system construction module, which specifically includes:

[0090] The domain foundation model construction module is used to acquire and process scientific and technological literature data, construct domain foundation model training corpus, and further pre-train based on open-source pre-trained models using the training corpus to construct the domain foundation model.

[0091] The sampling model training module is used to initialize the parameters of the sampling model based on the domain base model, perform initial training on the sampling model using seed data, train the sampling model through multiple rounds of iterative reinforcement learning, and generate sample data.

[0092] The question-answering model training module is used to initialize the parameters of the question-answering model based on the domain base model, construct the distillation training dataset using sample data generated by the sampling model, and perform supervised fine-tuning training on the question-answering model until convergence.

[0093] The question-answering system building module is used to encapsulate the trained question-answering model into an online inference interface service to build a scientific and technological knowledge question-answering system. During the use of the question-answering system, user characteristics and conversation history are input into the question-answering model to dynamically generate personalized answers.

[0094] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, and network interface connected via a system bus. The processor provides computing and control capabilities, the network interface enables communication with external terminals via a network connection, and the computer device implements the aforementioned personalized scientific knowledge question-and-answer method by loading and running computer programs.

[0095] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0096] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored relating to all or part of the processes in the methods of the above embodiments.

[0097] In one embodiment, a computer program product is also provided, including a computer program / instruction relating to all or part of the processes in the methods of the above embodiments.

[0098] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in M ​​forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0099] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A personalized scientific knowledge question-and-answer method, characterized in that, The method comprises four steps: constructing a domain-based basic model, training a sampling model, training a question-answering model, and constructing a question-answering system. Specifically, it includes: Domain foundation model construction: Acquire and process scientific and technological literature data, construct a training corpus for the domain foundation model, and continue pre-training based on the open-source pre-trained model using the training corpus to construct the domain foundation model; Sampling model training: Based on the domain base model, initialize the parameters of the sampling model, use seed data to initially train the sampling model, train the sampling model through multiple rounds of iterative reinforcement learning, and generate sample data; Question answering model training: Based on the domain-based model, initialize the parameters of the question answering model, construct a distillation training dataset using sample data generated by the sampling model, and perform supervised fine-tuning training on the question answering model until convergence; Question answering system construction: The trained question answering model is encapsulated into an online inference interface service to build a scientific and technological knowledge question answering system. During the use of the question answering system, user characteristics and conversation history are input into the question answering model to dynamically generate personalized answers. In the sampling model training step, the seed data includes user information and the user's historical retrieval data. The user information includes the user's name, age, occupation, company, and industry information registered when the user opened the account. The historical retrieval data includes user queries, returned results, and user feedback information data through clicks or browsing. The sampling model training step, which involves training the sampling model through multiple rounds of reinforcement learning, includes: initializing the parameters of the domain-based model at the beginning of each iteration, generating multiple candidate answers to the user question using the current sampling model, scoring the candidate answers by combining rule verification and large model evaluation, and updating the parameters of the sampling model using the online GRPO reinforcement learning algorithm. The parameters of the sampling model are updated using the online GRPO reinforcement learning algorithm, specifically through the formula: Training is performed, where θ represents the sampling model parameters; q represents the query input, which follows a P(Q) distribution; J GRPO (θ) represents the optimization objective; G is the number of sample groups, with a value of 16; π θ A new strategy for sampling models; For the old strategy of the sampling model; π ref The reference model strategy initially uses the domain-based model, and subsequent iterations use the model sampled from the previous round; `clip` is the clipping function; o i Output for the i-th sample; o i，t Output for the i-th sample at time t; i，＜t The output of the i-th sample before time t; ∈ is the shearing range hyperparameter; β is the KL divergence weight coefficient; D KL KL divergence distance: Calculated using relative reward values ​​within the group: To normalize the reward, r i Let r be the reward for the i-th sample, r represent the reward variable across all samples, mean(r) be the mean of variable r, and std(r) be the standard deviation of variable r.

2. The method according to claim 1, characterized in that, In the process of building the domain basic model, processing scientific and technological literature data includes deduplication of the data, removal of HTML / XML tags and non-text content, and splitting the text data into title, abstract, background, problem, technical solution, and effect sections.

3. The method according to claim 1, characterized in that, In the question-answering model training step, the process of constructing a distillation training dataset using sample data generated by the sampling model includes: performing rejection sampling filtering on the sample data to remove samples with incorrect format or scores below the threshold; combining multiple historical dialogues of the same user in chronological order of sampling time to form synthetic sample data; and concatenating the synthetic sample data with the current question to generate prompt words as model input.

4. A personalized science and technology knowledge question-and-answer system, characterized in that, The system includes a domain-based basic model building module, a sampling model training module, a question-answering model training module, and a question-answering system building module, which specifically include: The domain foundation model construction module is used to acquire and process scientific and technological literature data, construct domain foundation model training corpus, and further pre-train based on open-source pre-trained models using the training corpus to construct the domain foundation model. The sampling model training module is used to initialize the parameters of the sampling model based on the domain base model, perform initial training on the sampling model using seed data, train the sampling model through multiple rounds of iterative reinforcement learning, and generate sample data. The question-answering model training module is used to initialize the parameters of the question-answering model based on the domain base model, construct the distillation training dataset using sample data generated by the sampling model, and perform supervised fine-tuning training on the question-answering model until convergence. The question-answering system building module is used to encapsulate the trained question-answering model into an online inference interface service to build a scientific and technological knowledge question-answering system. During the use of the question-answering system, user characteristics and conversation history are input into the question-answering model to dynamically generate personalized answers. In the sampling model training module, the seed data includes user information and the user's historical search data. The user information includes the user's name, age, occupation, company, and industry information registered when the user opened the account. The historical search data includes user queries, returned results, and user feedback information data through clicks or browsing. In the sampling model training module, the sampling model is trained through multiple rounds of reinforcement learning, including: initializing the parameters of the domain base model at the beginning of each round of iteration, generating multiple candidate answers to the user question using the current sampling model, scoring the candidate answers by combining rule verification and large model evaluation, and updating the parameters of the sampling model using the online GRPO reinforcement learning algorithm; The parameters of the sampling model are updated using the online GRPO reinforcement learning algorithm, specifically through the formula: Training is performed, where θ represents the sampling model parameters; q represents the query input, which follows a P(Q) distribution; J GRPO (θ) represents the optimization objective; G is the number of sample groups, with a value of 16; π θ A new strategy for sampling models; For the old strategy of the sampling model; π ref The reference model strategy initially uses the domain-based model, and subsequent iterations use the model sampled from the previous round; `clip` is the clipping function; o i Output for the i-th sample; o i，t Output for the i-th sample at time t; i，＜t The output of the i-th sample before time t; ∈ is the shearing range hyperparameter; β is the KL divergence weight coefficient; D KL KL divergence distance: Calculated using relative reward values ​​within the group: To normalize the reward, r i Let r be the reward for the i-th sample, r represent the reward variable across all samples, mean(r) be the mean of variable r, and std(r) be the standard deviation of variable r.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.

7. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 3.

Images