Private domain question and answer method and system based on automatic scheduling

Abstract

The invention discloses a private domain question and answer method and system based on automatic scheduling, and mainly relates to the technical field of natural language processing. Comprising the following steps: loading private domain data, and performing text division on a loaded text; on the basis of the LLM and the divided text blocks, automatically generating fine tuning data sets of the embedding model and the reordering model, and respectively performing fine tuning on the sum; carrying out manual classification on the private domain data files; performing automatic query distribution on user queries based on LLM; taking query as flow input pre-arranged by the knowledge base, and outputting reference information; and after the reference information is embedded into the cue word template, inputting the reference information into the LLM to generate an answer corresponding to the user query. The method has the beneficial effects that the private domain knowledge is classified, the query tasks are automatically shunted, and the private domain knowledge questions and answers are more accurate, convenient and flexible in combination with the standardized process of query rewriting, semantic retrieval and reordering.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of natural language processing, and particularly relates to a private domain question and answer method and system based on automatic scheduling. BACKGROUND

[0002] The development of generative artificial intelligence is rapid, and large language models (LLM) have shown great advantages in natural language understanding, analysis and dialogue, making knowledge acquisition and understanding more efficient. However, the pre-training phase of LLM knowledge injection is extremely costly, such as a large amount of high-quality data, powerful computing power and a long training time, which makes it difficult to train a customized LLM for users according to the private domain data of the users. In addition, the "illusion" of LLM is difficult to avoid, resulting in insufficient credibility of the endogenous knowledge and poor timeliness. Retrieval augmented generation (RAG) is one of the preferred solutions for training LLM, effectively solving the problems of private domain knowledge defects and low timeliness of LLM, and to a certain extent, reducing the risk of generating incorrect information.

[0003] The main process of RAG is to load, divide and vectorize the file to the knowledge base, retrieve a number of text blocks most relevant to the user input from the knowledge base by calculating semantic similarity, and thus provide reliable relevant reference information for the answer of LLM to solve the knowledge defects of LLM. However, due to the input length of LLM and the understanding ability of long text, the number of relevant text blocks recalled from the knowledge base is usually single-digit, and it is difficult for the retriever to accurately hit the relevant information in the small amount of text blocks retrieved; especially for information comparison problems, the retriever needs to accurately hit all the information to be compared, and RAG usually fails.

[0004] Therefore, an effective solution is needed to improve the accuracy, convenience and applicability of the private domain knowledge question and answer system. SUMMARY

[0005] The purpose of the present application is to provide a private domain question and answer method and system based on automatic scheduling, which classifies private domain knowledge, automatically distributes query tasks, and combines query rewriting, semantic retrieval and reordering standardized processes to make private domain knowledge question and answer more accurate, convenient and flexible.

[0006] To achieve the above purpose, the present application realizes the following technical solutions: On the one hand, the present application provides a private domain question and answer method based on automatic scheduling, comprising the following steps: Step S1: loading private domain data and performing text division on the loaded text; Step S2: based on LLM and the divided text blocks, automatically generating an embedding model and a reordering model fine-tuning dataset , and adopting , and fine-tuning; Step S3: manually classifying the private domain data files, dividing all files into categories, corresponding to knowledge bases , and arranging the flow of the knowledge bases based on the fine-tuned and ; Step S4: based on the LLM, automatically querying and shunting the user query to determine the corresponding knowledge base , ; and Step S5: inputting the query into the pre-arranged flow of the knowledge base , and outputting the reference information ; Step S6: embedding the reference information into the prompt template and inputting it into the LLM to generate the answer corresponding to the user query .

[0007] Preferably, the step S2 specifically comprises: Step S21: loading and deploying the LLM, which includes but is not limited to: ChatGLM3, Tongyi Qianwen, and Baichuan; Step S22: randomly selecting text blocks from the divided text block set to obtain a subset of , and embedding the text blocks into the prompt template one by one to obtain , ; inputting into the LLM to obtain corresponding questions , ; Step S23: based on the text blocks and the corresponding questions , preliminarily constructing a dataset , specifically, taking each piece of data in as a query , as a positive example of , and the query and positive example binary tuple preliminarily constitutes a dataset A single piece of data in ,Will Re-expressed as , ; Step S24: Loading text embedding model ,based on For the dataset Generate a counterexample for each data in The Chinese version of the text embedding model BGE-Large is used. BGE stands for BAAI General Embedding. BGE-Large is based on the BERT architecture and consists of an embedding layer and 24 stacked Transformer encoders. The embedding layer includes word embedding, position embedding, and token type embedding. Step S25: Using the data set Text Embedding Model Make fine adjustments; Step S26: Using the data set Reordering Model Make fine adjustments.

[0008] Preferably, step S24 is specifically as follows: Step S241: Assemble the text blocks Each text block in Enter one by one to ,get Semantic embedding of , The reasoning process for generating semantic embeddings is as follows: (1) in , represents the real number space, , , Indicates that the input is hour, The full output of , Right now The vector at the 0th position of Semantic embedding of ; Represents the cascade operation of 24 Transformer encoders; 、 and Represent the embedding operations of word embedding, position embedding, and token embedding respectively; Represents a mapping function that converts words in a string into corresponding digital ids; 、 respectively are the absolute position encoding and token type encoding of , respectively are , , is a full zero vector with dimension ; denotes string concatenation, and the text block set can be obtained by formula (1) ; Step S242: Calculate the similarity between the semantic embedding of each single query in the data set and each embedding vector in the semantic embedding set , , specifically, the semantic embedding of is obtained by formula (1) , , and the semantic embedding set is regarded as a matrix , , , and the similarity matrix between and is calculated by formula (2) : (2) Each element in is the inner product of and , which also represents the similarity between and . The element values in are sorted from high to low, and the embedding vectors between the th and the th are taken, and of them are randomly selected. The text blocks corresponding to the embedding vectors selected correspond to the counterexample of the query ; Step S243: The query, positive example, and counterexample triplets constitute the final data set .

[0009] Preferably, the step S25 specifically comprises: Step S251: Label a single sample in the data set as , First, concatenate it with the query instruction , that is , = "Generate a representation for this sentence to retrieve related articles:", Input to Get semantic embedding ; Input to get ;from Randomly select counterexamples and input them into get , ; Step S252: Define training text embedding model The objective function is, specifically, calculated by formula (3) and and The similarity of , : (3) represents the horizontal concatenation of column vectors, , The training loss uses cross entropy loss. Since the positive example is always the 0th column of the similarity matrix, the one-hot label of the training data All are [1, 0, 0, 0], training The objective function is shown in formula (4): (4) in, yes The learnable parameters of Represents a reshape operation that converts a matrix into a vector; Step S253: Based on the data set And the objective function shown in formula (4), with a lower learning rate right conduct Rounds of full parameter fine-tuning, where , .

[0010] Preferably, the step S26 specifically includes: Step S261: Load the reordering model , namely BGE-Reranker-Large, will ( Input to Post-output and Similarity value , the specific reasoning process is shown in formula (5): (5) in, represents global pooling; and They are The learnable weights and biases of the last classification layer in , , ; 、 They are ( Absolute position encoding and token type encoding, , The dimension is The all-zero vector and dimension are The vector after concatenation of all 1 vectors, ; Step S262: Define training re-ranking model The objective function, labeled dataset A single sample in , Input to Inference and Similarity value ;from Randomly select Counterexample , and Enter into , get and Similarity ; The objective function of is shown in formula (6): (6) in, yes The learnable parameters of That is a single piece of training data The one-hot label of , the 0th position is always 1, that is ; Step S263: Based on the data set And the objective function shown in formula (6), with a lower learning rate right conduct Rounds of full parameter fine-tuning, where , .

[0011] Preferably, the step S3 specifically includes: Step S31: Manually classify the private domain data files and divide all the files into categories, The files of the categories correspond to Knowledge Base ; Step S32: For a single knowledge base The text blocks after division are input into the fine-tuned text embedding model one by one. , get the semantic embedding of the text block, and then save all the semantic embeddings to the Faiss vector library ; Step S33: The knowledge base The calling process is arranged in sequence as user query, query rewriting, semantic embedding, vector library retrieval, reranking, and related text; Specifically, the user query Embed prompt word template ,get ,Will Input into LLM to get the rewritten query list ,like For non-comparative questions, list There is only one element in the Basically the same; if If information comparison is involved, the list Contains multiple elements, each element is After splitting, the text embedding model is used Rewritten user query The elements in are semantically embedded one by one to obtain the vector ;calculate With Faiss vector library The inner product of all vectors in The most similar The text block corresponding to the vector is used as the user query The relevant text block ; Using re-ranking model For relevant text Re-sort and take the highest score after re-sorting text blocks As a reference .

[0012] Preferably, the step S4 specifically includes: Step S41: User query Embedded in prompt word template In, get ; The name of the whole knowledge base and a brief summary of each knowledge base are included; Step S42: inputting the reference information into the LLM to obtain an output , i.e., the answer to the user query .

[0013] Preferably, the step S5 specifically comprises: if , the user query is inputted into the knowledge base with the name as the input, and the reference information is obtained as the output of the knowledge base with the name through the pre-arranged knowledge base flow; otherwise, the user query is not inputted into any knowledge base, and the reference information is empty.

[0014] Preferably, the step S6 specifically comprises: Step S61: if the reference information is not empty, the prompt word and the user query are embedded into the prompt word template to obtain ; if the reference information is empty, only the user query is embedded into the prompt word template to obtain , i.e., the user query is answered by the LLM itself; Step S62: inputting the prompt word or into the LLM to obtain an output , i.e., the answer to the user query .

[0015] On the other hand, a system based on the automated scheduling-based private domain question and answer method is provided, comprising: an LLM module configured to deploy an LLM service and generate text based on a prompt word; a sample generation module configured to generate fine-tuning data for a semantic embedding model and a reordering model based on the LLM module and private domain data of a user; a model training module configured to fine-tune the embedding model and the reordering model based on the data generated by the sample generation module; a knowledge base module configured to store text blocks and semantic vectors of a plurality of knowledge bases and arrange a running flow of the knowledge bases; a query shunting module configured to divide a user query into corresponding knowledge bases.

[0016] Compared with the prior art, the beneficial effects of the present invention are: The present invention classifies private domain data and automatically diverts query tasks, and then combines the standardized process of query rewriting, semantic retrieval, and re-ranking to make private domain knowledge question and answer more accurate, convenient, and flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart of the method provided by the present invention; Figure 2 A diagram showing the architecture of the method provided by the present invention; Figure 3 A schematic diagram of the vector library search provided by the present invention; Figure 4 A schematic diagram of fine-tuning the text embedding model provided by the present invention; Figure 5 Schematic diagram of fine-tuning the reordering model provided by the present invention. DETAILED DESCRIPTION

[0018] Below in conjunction with specific embodiment, further set forth the present invention.Should be understood that these embodiments are only used to illustrate the present invention and are not used in limiting the scope of the present invention.In addition, should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the application equally.

[0019] In the present invention, terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom", etc. indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. They are relational words determined only for the convenience of describing the structural relationships of the various parts or elements of the present invention, and do not specifically refer to any part or element in the present invention, and should not be understood as limiting the present invention.

[0020] Example 1: The embodiment of the present invention provides a private domain question-answering method based on automated scheduling, such as Figure 1 、 Figure 2 As shown, the following steps are included: Step S1: Load private domain data and perform text segmentation on the loaded text; Step S2: Automatically generate an embedding model based on the LLM and the divided text blocks and re-ranking models Fine-tuning dataset , and adopt Respectively and Make fine adjustments; Step S3: Manually classify the private domain data files and divide all the files into one category, corresponding one knowledge base , and based on the fine-tuned and process orchestration is performed on the knowledge base; Step S4: based on the LLM, the user query is automatically diverted, and the corresponding knowledge base is determined , ; Step S5: input the query into the pre-orchestrated process of the knowledge base , and output the reference information ; Step S6: after embedding the reference information into the prompt template, input it into the LLM to generate the corresponding answer to the user query .

[0021] In embodiment 1 of the present application, step S1 includes the following steps: Step S1-1: load private domain data, load DOCX format documents using python-docx library, modify table data to Markdown format after loading, remove line breaks in cells, and fill empty cells with the character "empty" to ensure clear table structure; use the soffice command of LibreOffice to automatically convert DOC format documents to DOCX format and then use the python-docx library to load; use the rapidocr_onnxruntime library to load image PNG, JPG, JPEG, etc. format images; use openpyxl to load XLSX, XLSM, XLS, etc. format Excel tables, also in Markdown format, remove line breaks and "empty" character padding empty cells; use the rapidocr_onnxruntime library to load PPT, PPTX format files; use the fitz library and the rapidocr_onnxruntime library to load PDF files.

[0022] Step S1-2: use regular expressions to clean the read file content, for example, use "re.sub(r"(?:[\.…] *){3,}", '...', text)" to replace redundant ellipses, use "re.sub(r"\n{3,}", '\n',text)" to replace consecutive line breaks, and use "re.sub(r"\ {3,}", '', text)" to replace consecutive spaces.

[0023] Step S1-3: text division is performed on the loaded text data, and the text data is divided into text blocks with a length less than 400. The divided text data has a total of text blocks, and the text block set .

[0024] In Embodiment 1 of the present application, step S2 includes the following steps: Step S2-1: load and deploy LLM, and the LLM used in this embodiment is Int4 quantized Qwen1.5-72B.

[0025] Step S2-2: randomly extract text blocks from the text block set , obtain a subset of text blocks, and embed the text blocks into the prompt template one by one to obtain , ; input into the LLM to obtain corresponding questions , . For example, “<instruction> You are a data engineer in the field of natural language processing, and your task is to ask as many questions as possible based on the known text. Note: 1. The answer to the question must be in the known text; 2. The question is not allowed to be repeated; 3. The question is given in the form of a list and does not have redundant answers. <instruction> \n <known text>{{ content}}< / known text>”, where “{{ content}}” is the text block to be embedded .

[0026] Step S2-3: based on the text block and the corresponding question , a dataset is preliminarily constructed . Specifically, each data in is taken as a query, , is taken as a positive example of , and a (query, positive example) binary tuple preliminarily constitutes a single data in the dataset , i.e. , and is represented as , .

[0027] Step S2-4: load the text embedding model , and based on ​ Each piece of data generates a counterexample. In this embodiment The Chinese version of the text embedding model BGE-Large is used, and BGE is BAAI general embedding. The base of BGE-Large is the BERT (Transformer-based bidirectional encoder representation) architecture, which is specifically composed of an embedding layer and 24 Transformer encoder stacks, wherein the embedding layer includes word embedding, position embedding and token type embedding. Step S2-4 specifically includes: Step S2-4-1: input each text block in the text block set to to obtain the semantic embedding of . . The inference process of generating semantic embedding is specifically as follows: taking a sample with a length of as an example, input to to obtain an embedding vector , as shown in formula (I), wherein , represents a real number space, , .

[0028] (I) represents the output of when the input is , , that is, the vector of the 0th position of , which is also the semantic embedding of ; represents the cascade operation of the 24 Transformer encoders; , and represent the embedding operations of the word embedding, the position embedding and the token embedding, respectively; represents a mapping function that converts words in a string to corresponding digital ids; , are strings “[CLS]” and “[SEP]”, respectively; , are the absolute position encoding and token type encoding of , , is a full zero vector with a dimension of ; ​Represents string concatenation. The text block set can be obtained by formula (I) The corresponding semantic embedding set .

[0029] Step S2-4-2: Calculate the semantic embedding of each query in the data set and the semantic embedding of each embedding vector in the semantic embedding set . The similarity between them, . Specifically, the semantic embedding of is obtained by formula (I) , , and the semantic embedding set is regarded as a matrix , , and the similarity matrix between is calculated by formula (II) , .

[0030] (II) Each element in is the inner product of , which also represents the similarity between and . The element values in are sorted from high to low, and the embedding vectors between the th and the th are taken, and are randomly selected from them. The text blocks corresponding to the embedding vectors selected are taken as counterexamples of the query , , , . Here, the text blocks with higher similarity are taken as counterexamples, rather than the ones with the lowest similarity. The purpose is to enable the semantic embedding to distinguish information with high initial similarity but no association, for example, in the "function description of product A" and "function description of product B", the influence of the names of products A and B on semantic similarity should be greater than that of the function description.

[0031] Step S2-4-3: The (query, positive example, counterexample) triplet constitutes the final data set .

[0032] Step S2-5: Fine-tune the text embedding model using the data set , such as​Figure 3 are shown, specifically including: Step S2-5-1: Taking a single sample in the data set as an example, concatenate it with the query instruction , that is , , = "Generate a representation for this sentence for retrieving related articles:", and input it into to obtain semantic embedding ; input it into to obtain ; , randomly sample negative examples from , and input them into to obtain , .

[0033] Step S2-5-2: Define the objective function of the training text embedding model . Specifically, the similarity between and and is calculated by formula (III) to obtain the similarity matrix , .

[0034] (III) represents the horizontal concatenation of column vectors, . The training loss of uses cross-entropy loss. Since the positive example is always the 0th column of the similarity matrix, the one-hot label of the training data is all [1, 0, 0, 0], and the objective function of is shown in formula (IV).

[0035] (IV) where is the learnable parameter of , represents the reshaping operation of converting a matrix to a vector.

[0036] Step S2-5-3: Based on the data set and the objective function shown in formula (IV), perform full-parameter fine-tuning of for epochs with a lower learning rate , where , .

[0037] Step S2-6: Using the dataset Reordering Model Make fine adjustments, such as Figure 4 As shown, specifically including: Step S2-6-1: Load the reordering model In this embodiment, the reordering model The BGE-Reranker-Large architecture is the same as BERT, with the difference that a mean pooling layer and a single neuron classification layer are added at the end. For example, Input to Post-output and Similarity value , the specific reasoning process is shown in formula (V).

[0038] (V) in, represents global pooling; and They are The learnable weights and biases of the last classification layer in , , ; 、 They are ( Absolute position encoding and token type encoding, , The dimension is The all-zero vector and dimension are The vector after concatenation of all 1 vectors, ; The meanings of other functions and parameters are consistent with those of formula (Ⅰ).

[0039] Step S2-6-2: Define training reranking model The objective function of the dataset A single sample in For example, Input to Inference and Similarity value ;from Randomly select Counterexample , and Enter into get and Similarity . The objective function is shown in formula (VI).

[0040] (VI) wherein, is a learnable parameter of the model, i.e. the one-hot label of the single training data , the 0th position is always 1, i.e. .

[0041] Step S2-6-3: based on the data set and the objective function shown in formula (VI), the full parameter fine-tuning of is performed for a number of rounds with a lower learning rate . . , .

[0042] In embodiment 1 of the present application, step S3 includes the following steps: Step S3-1: manually classify the private domain data files, divide all files into categories according to the file content, for example, "equipment maintenance record form" in different periods belong to the same category, but "training materials" belong to different categories; files in each category correspond to knowledge base .

[0043] Step S3-2: taking a single knowledge base as an example, the files in are divided into text blocks, and the divided text blocks are input into the fine-tuned text embedding model one by one to obtain the semantic embedding of the text blocks, and then all semantic embeddings are saved to the Faiss vector library .

[0044] Step S3-3: the calling flow of the knowledge base is arranged as user query→query rewriting→semantic embedding→vector library retrieval→reordering→related text, as shown in Figure 5 . Specifically, the user query is embedded into the prompt word template to obtain , and is input into the LLM to obtain the rewritten query list . If is a non-contrastive problem, there is only one element in the list , and the semantics is basically the same as ; if involves information comparison, the list Contains multiple elements, each element is After splitting, the sub-questions, such as "Which one among A, B and C has the most functions?" are rewritten into a query list of "['What functions does A include?', 'What functions does B include?', 'What functions does C include']". For example, "<instruction> You are a professional language analyst. Your task is to polish or split the question and present it in the form of a single list. When the question does not involve information comparison, polish the question for easy retrieval. For example, 'The function of A' can be polished to ['What are the functions of A?']; if the question involves information comparison, split the question. For example, 'Which of A, B, and C has the highest performance?' can be split into ['What is the performance of A?', 'What is the performance of B?', 'What is the performance of C?']. Note: Do not have redundant answers.< / 指令> \n<Question>{{ query}}< / 问题> \nPlease convert the question according to the instructions. ", where "{{ query}}" is the query to be embedded . Using text embedding model Rewritten user query The elements in are semantically embedded one by one to obtain the vector ;calculate With Faiss vector library The inner product of all vectors in The most similar The text block corresponding to the vector is used as the user query The relevant text block ; Using re-ranking model For relevant text Re-sort and take the highest score after re-sorting text blocks As a reference , .

[0045] In embodiment 1 of the present invention, step S4 includes the following steps: Step S4-1: User query Embedded in prompt word template In, get , For example, "<Instruction> You are a professional task splitter. Your task is to select a knowledge base that is most relevant to the problem. The knowledge base includes .< / 指令> \nThe brief introduction of each knowledge base is as follows:\n : \n : \n......\n : \n<Question>{{ query}}< / 问题> \nPlease select the knowledge base that is most relevant to your question and list it in a list format without redundant answers. ”, where “{{ query}}” is the query to be embedded , for The name of the knowledge base, Knowledge Base A brief overview of .

[0046] Step S4-2: Input into LLM and get output , that is, user query The corresponding knowledge base.

[0047] In the embodiment 1 of the present invention, step S5 is specifically, if , then the user query As the name The knowledge base input is obtained through the knowledge base process pre-arranged in step S3-3. Reference information output from the knowledge base ; Otherwise user query Not used as input to any knowledge base, reference information Empty.

[0048] In embodiment 1 of the present invention, step S6 includes the following steps: Step S6-1: If the reference information If it is not empty, and user inquiries Embedded in prompt word template get ; If the reference information is empty, only the user query Embedded in prompt word template get . For example, "<Instruction> Please answer the question based on the known information. If the known information does not contain the answer to the question, please answer 'The question cannot be answered based on the known information'."< / 指令> \n<Known Information>{{ content}}< / 已知信息> \n<Question>{{ query}}< / 问题> ”, where “{{ content}}” and “{{ query}}” are the reference information to be embedded. and inquiries . LLM's own capabilities are used to query users To answer, specific examples are "Please answer the following question: {{ query}}", where {{ query}} is the query to be embedded .

[0049] Step S6-2: input the prompt word or to the LLM to obtain the output , which is the corresponding answer to the user query .

[0050] Embodiment 2: A private domain question and answer system based on automatic scheduling, used to run the private domain question and answer method based on automatic scheduling described in embodiment 1, including an LLM module, a sample generation module, a model training module, a knowledge base module, and a query shunting module.

[0051] The LLM module is used to deploy an LLM service and generate text based on prompt words; the sample generation module is used to generate fine-tuning data for semantic embedding models and reordering models based on the LLM module and the user's private domain data; the model training module is used to fine-tune the embedding models and reordering models based on the data generated by the sample generation module; the knowledge base module is used to store text blocks and their semantic vectors of multiple knowledge bases, and to arrange the running process of the knowledge bases; the query shunting module is used to divide user queries into corresponding knowledge bases.

[0052] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the described embodiments. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A private domain question-answering method based on automated scheduling, characterized in that: The following steps are involved: Step S1: Load private domain data and perform text segmentation on the loaded text; Step S2: Automatically generate an embedding model based on the LLM and the divided text blocks and re-ranking models Fine-tuning dataset , and adopt respectively and Make fine adjustments; Step S3: Manually classify the private domain data files and divide all the files into categories, corresponding to Knowledge Base , and based on the fine-tuned and Conduct process orchestration for the knowledge base; Step S4: Query the user based on LLM Perform automatic query diversion to confirm Corresponding knowledge base , ; Step S5: Query As a knowledge base Pre-arranged process input and output reference information ; Step S6: Reference information After being embedded into the prompt word template, it is input into LLM to generate user queries The corresponding answer.

2. A private domain question-answering method based on automated scheduling according to claim 1, characterized in that: The step S2 specifically includes: Step S21: Load and deploy LLM, including but not limited to: ChatGLM3, Tongyi Qianwen, Baichuan; Step S22: From the divided text block set Randomly select A block of text, Subset of , and the text block Embed one by one into the prompt word template In, get , ;Will Input into LLM to get the text block Corresponding Question , ; Step S23: Based on the text block and corresponding issues Preliminary construction of the dataset , specifically, Each piece of data in As a query, , As The query and positive example tuples initially constitute the data set A single piece of data in ,Will Re-expressed as , ; Step S24: Loading text embedding model ,based on For the dataset Generate a counterexample for each data in The Chinese version of the text embedding model BGE-Large is used. BGE stands for BAAI General Embedding. BGE-Large is based on the BERT architecture and consists of an embedding layer and 24 stacked Transformer encoders. The embedding layer includes word embedding, position embedding, and token type embedding. Step S25: Using the data set Text Embedding Model Make fine adjustments; Step S26: Using the data set Reordering Model Make fine adjustments.

3. The private domain question-answering method based on automated scheduling according to claim 2, characterized in that: Step S24 is specifically as follows: Step S241: Assemble the text blocks Each text block in Enter one by one to ,get Semantic embedding of , The reasoning process for generating semantic embeddings is as follows: （1） in , represents the real number space, , , Indicates that the input is hour, The full output of , Right now The vector at the 0th position of Semantic embedding of ; Represents the cascade operation of 24 Transformer encoders; 、 and Represent the embedding operations of word embedding, position embedding, and token embedding respectively; Represents a mapping function that converts words in a string into corresponding digital ids; 、 They are the strings "[CLS]" and "[SEP]" respectively; 、 They are Absolute position encoding and token type encoding, , The dimension is An all-zero vector of ; Indicates string concatenation, and the text block set can be obtained through formula (1) The corresponding semantic embedding set ; Step S242: Calculate data sets one by one Single query Semantic embedding and semantic embedding collection Each embedding vector The similarity of , specifically, through formula (1) we get Semantic embedding of , , and then embed the semantics into the set Treated as a matrix , , and calculated by formula (2) and Similarity matrix , : （2） Each element in is and The inner product of and The similarity of Sort the element values ​​in from high to low, and take the to The embedding vector between them is randomly extracted , extracted The text block corresponding to the embedding vector As a query Counterexamples Step S243: The query, positive example, and negative example triples constitute the final data set .

4. The private domain question-answering method based on automated scheduling according to claim 2, characterized in that: The step S25 specifically includes: Step S251: Labeling the dataset A single sample in , First, with the query command Splicing, that is , = "Generate a representation for this sentence to retrieve related articles:", Input to Get semantic embedding ; Input to get ;from Randomly select counterexamples and input them into get , ; Step S252: Define training text embedding model The objective function is specifically calculated by formula (3): and and The similarity of , : （3） represents the horizontal concatenation of column vectors, , The training loss uses cross entropy loss. Since the positive example is always the 0th column of the similarity matrix, the one-hot label of the training data All are [1, 0, 0, 0], training The objective function is shown in formula (4): （4） in, yes The learnable parameters of Represents a reshape operation that converts a matrix into a vector; Step S253: Based on the data set And the objective function shown in formula (4), with a lower learning rate right conduct Rounds of full parameter fine-tuning, where , .

5. The private domain question-answering method based on automated scheduling according to claim 2, characterized in that: The step S26 specifically includes: Step S261: Load the reordering model , namely BGE-Reranker-Large, will ( Input to Post-output and Similarity value , the specific reasoning process is shown in formula (5): （5） in, represents global pooling; and They are The learnable weights and biases of the last classification layer in , , ; 、 They are ( Absolute position encoding and token type encoding, , The dimension is The all-zero vector and dimension are The vector after concatenation of all 1 vectors, ; Step S262: Define training re-ranking model The objective function, labeled dataset A single sample in , Input to Inference and Similarity value ;from Randomly select Counterexample , and Enter into , get and Similarity ; The objective function of is shown in formula (6): （6） in, yes The learnable parameters of That is a single training data The one-hot label of , the 0th position is always 1, that is ; Step S263: Based on the data set And the objective function shown in formula (6), with a lower learning rate right conduct Rounds of full parameter fine-tuning, where , .

6. The private domain question-answering method based on automated scheduling according to claim 1, characterized in that: The step S3 specifically includes: Step S31: Manually classify the private domain data files and divide all the files into categories, The files of the categories correspond to Knowledge Base ; Step S32: For a single knowledge base The text blocks after division are input into the fine-tuned text embedding model one by one. , get the semantic embedding of the text block, and then save all the semantic embeddings to the Faiss vector library ; Step S33: The knowledge base The calling process is arranged in sequence as user query, query rewriting, semantic embedding, vector library retrieval, reranking, and related text; Specifically, the user query Embed prompt word template ,get ,Will Input into LLM to get the rewritten query list ,like For non-comparative questions, list There is only one element in the Basically the same; if If information comparison is involved, the list Contains multiple elements, each element is After splitting, the text embedding model is used Rewritten user query The elements in are semantically embedded one by one to obtain the vector ;calculate With Faiss vector library The inner product of all vectors in The most similar The text block corresponding to the vector is used as the user query The relevant text block ; Using re-ranking model For relevant text Re-sort and take the highest score after re-sorting text blocks As a reference .

7. The private domain question-answering method based on automated scheduling according to claim 1, characterized in that: The step S4 specifically includes: Step S41: User query Embedded in prompt word template In, get ; Contains the names of all knowledge bases and a brief overview of each knowledge base; Step S42: Input into LLM and get output , that is, user query The corresponding knowledge base.

8. The private domain question-answering method based on automated scheduling according to claim 1, characterized in that: The step S5 specifically includes: , then the user query As the name The input of the knowledge base is obtained through the pre-arranged knowledge base process named Reference information output from the knowledge base ; Otherwise user query Not used as input to any knowledge base, reference information Empty.

9. The private domain question-answering method based on automated scheduling according to claim 1, characterized in that: The step S6 specifically includes: Step S61: If the reference information If it is not empty, and user inquiries Embedded in prompt word template get ; If the reference information is empty, only the user query Embedded in prompt word template get , that is, using LLM's own capabilities to query users Provide answers; Step S62: The prompt word or Input to LLM, get output , User query The corresponding answer.

10. A system based on the private domain question-answering method based on automated scheduling as claimed in claim 1, characterized in that: include: The LLM module is used to deploy LLM services and generate text based on prompt words. The sample generation module is used to generate fine-tuning data for the semantic embedding model and re-ranking model based on the LLM module and the user's private domain data; The model training module is used to fine-tune the embedding model and reranking model based on the data generated by the sample generation module; The knowledge base module is used to store text blocks and semantic vectors of multiple knowledge bases and to orchestrate the operation process of the knowledge base; The query diversion module is used to divide user queries into corresponding knowledge bases.

Images