Large language model domain knowledge collaboration method and device, equipment and medium

Abstract

The invention provides a big language model domain knowledge collaboration method, device and equipment and a medium, and relates to the technical field of artificial intelligence, the method comprises the steps that a user question is input into a knowledge collaboration big language model, a preliminary hypothesis answer is generated, and the knowledge collaboration big language model is a big language model subjected to knowledge collaboration fine tuning; based on a self-checking mechanism, determining the rationality of the preliminary hypothesis answer, and when the preliminary hypothesis answer is unreasonable, retrieving domain knowledge evidence related to a domain subject term in the user question; performing semantic and domain filtering on the domain knowledge evidence in sequence to generate high-confidence evidence; and generating a final answer according to the user question and the high-confidence evidence. According to the invention, more professional and accurate answers can be obtained.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a method, device, equipment and medium for collaborative domain knowledge of large language models. Background Art

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) have demonstrated powerful semantic understanding and generation capabilities in natural language processing tasks. However, when dealing with tasks in specific domains, general large language models often lack sufficient domain knowledge, resulting in inaccurate or unprofessional generated responses. This problem is particularly prominent in professional fields such as agriculture, medicine, and law. For example, in the agricultural field, users may ask questions about crop planting, pest control, variety recommendation, etc., and general large language models often cannot provide high-quality responses due to the lack of relevant domain knowledge.

[0003] In related technologies, in order to enable large language models to provide high-quality responses, domain knowledge injection fine-tuning technology and retrieval-enhanced generation technology are generally adopted. These two technologies can provide a certain degree of high-quality responses. However, the current domain knowledge injection fine-tuning technology pays more attention to the fitting of general vocabulary during the training process and ignores important keywords in the domain. Moreover, the current retrieval-enhanced generation technology has problems in retrieval efficiency and retrieval timing, and the retrieval method also has limitations. Especially when facing domain knowledge with similar semantics, the retrieval effect is not ideal. Therefore, the responses generated by current large language models still lack professionalism and accuracy. Summary of the Invention

[0004] The problem solved by the present invention is how to improve the professionalism and accuracy of the responses of large language models.

[0005] To solve the above problems, the present invention provides a method, device, equipment and medium for collaborative domain knowledge of large language models.

[0006] In a first aspect, the present invention provides a method for collaborative domain knowledge of large language models, including:

[0007] Inputting a user question into a knowledge collaborative large language model to generate a preliminary hypothesized answer, where the knowledge collaborative large language model is a large language model fine-tuned through knowledge collaboration;

[0008] Based on a self-check mechanism, determining the rationality of the preliminary hypothesized answer. When the preliminary hypothesized answer is unreasonable, retrieving domain knowledge evidence related to the domain topic words in the user question;

[0009] Sequentially performing semantic and domain filtering on the domain knowledge evidence to generate high-confidence evidence;

[0010] Generate a final answer based on the user question and the high-confidence evidence.

[0011] Optionally, before inputting the user question into the knowledge collaborative large language model to generate a preliminary hypothesis answer, it further includes:

[0012] Determine the first semantic similarity between the training Q&A data and the corresponding domain knowledge data according to the obtained training Q&A data and the corresponding domain knowledge data;

[0013] Determine the loss parameter of the initial large language model according to the first semantic similarity and the initial prediction probability of the initial large language model, where the initial prediction probability is determined by the initial large language model based on the training Q&A data;

[0014] Fine-tune the initial large language model according to the loss parameter to obtain the knowledge collaborative large language model.

[0015] Optionally, the determining the first semantic similarity between the training Q&A data and the corresponding domain knowledge data according to the obtained training Q&A data and the corresponding domain knowledge data includes:

[0016] Segment the standard answer in the training Q&A data to obtain multiple words;

[0017] Determine the first semantic similarity between each word and the domain knowledge data.

[0018] Optionally, the determining the loss parameter of the initial large language model according to the first semantic similarity and the initial prediction probability of the initial large language model includes:

[0019] Use a non-linear function to convert the first semantic similarity into a similarity weight;

[0020] Generate a scaled prediction probability according to the similarity weight and the initial prediction probability;

[0021] Determine the loss parameter according to the scaled prediction probability and the cross-entropy loss.

[0022] Optionally, the retrieving domain knowledge evidence related to the domain topic words in the user question includes:

[0023] Use a topic word extraction model to extract the domain topic words in the user question;

[0024] Based on the domain topic words, perform fuzzy matching in the domain knowledge base to generate the domain knowledge evidence.

[0025] Optionally, the sequential semantic and domain filtering of the domain knowledge evidence to generate high-confidence evidence includes:

[0026] Extract the second semantic similarity between the domain knowledge evidence and the user question, and based on the second semantic similarity, rank the domain knowledge evidence, and retain the domain knowledge evidence with a preset ranking according to the ranking result to obtain highly relevant evidence;

[0027] Use a domain evaluation model to perform domain screening on the highly relevant evidence to obtain the high-confidence evidence.

[0028] Optionally, the large language model domain knowledge collaboration method further includes:

[0029] When the preliminary hypothesis answer is reasonable, output the preliminary hypothesis answer.

[0030] In a second aspect, the present invention provides a large language model domain knowledge collaboration device, including:

[0031] An input module for inputting a user question into a knowledge collaboration large language model to generate a preliminary hypothesis answer, where the knowledge collaboration large language model is a large language model fine-tuned through knowledge collaboration;

[0032] A retrieval module for determining the reasonableness of the preliminary hypothesis answer based on a self-check mechanism, and when the preliminary hypothesis answer is unreasonable, retrieving domain knowledge evidence related to the domain topic words in the user question;

[0033] A filtering module for sequentially performing semantic and domain filtering on the domain knowledge evidence to generate high-confidence evidence;

[0034] A generation module for generating a final answer according to the user question and the high-confidence evidence.

[0035] In a third aspect, the present invention provides an electronic device, including a memory and a processor;

[0036] The memory is used to store a computer program;

[0037] The processor is used to implement the large language model domain knowledge collaboration method as described in the first aspect when executing the computer program.

[0038] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the large language model domain knowledge collaboration method as described in the first aspect is implemented.

[0039] The beneficial effects of the large language model domain knowledge collaboration method, device, equipment and medium of the present invention are:

[0040] Collaborate with the large language model through the knowledge of the target field, such as the knowledge collaborative large language model fine-tuned with the knowledge of the agricultural field. First, generate a preliminary hypothetical answer according to the user's question, and then evaluate the rationality of the answer through a self-check mechanism. If the preliminary answer is unreasonable, retrieve the domain knowledge evidence related to the domain theme words in the user's question, and can efficiently and accurately retrieve the domain knowledge evidence of the relevant field to ensure that the finally provided answer has higher accuracy and reliability. Then, sequentially perform semantic and domain filtering on the domain knowledge evidence to ensure the high relevance and high professionalism of the evidence, and then use the high-confidence evidence with high relevance and high professionalism as support to answer again based on the user's question, and a more professional and accurate answer can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flowchart showing the method for collaborating domain knowledge of the large language model provided by an embodiment of the present invention;

[0042] Figure 2 It is a schematic structural diagram of the device for collaborating domain knowledge of the large language model provided by an embodiment of the present invention;

[0043] Figure 3 It is a schematic structural diagram of the electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe in detail the specific embodiments of the present invention with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments described herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not used to limit the protection scope of the present invention.

[0045] It should be understood that the various steps recorded in the method embodiments of the present invention can be executed in different orders and / or executed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this regard.

[0046] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to"; the term "based on" means "at least partially based on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts such as "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0047] It should be noted that the modifications of "one" and "multiple" mentioned in the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0048] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0049] In view of the problems existing in the above related technologies, this embodiment provides a method, apparatus, device and medium for knowledge collaboration in the field of large language models.

[0050] As Figure 1 shown, a method for knowledge collaboration in the field of large language models provided by an embodiment of the present invention includes:

[0051] Input the user's question into the knowledge collaboration large language model to generate a preliminary hypothesized answer, where the knowledge collaboration large language model is a large language model fine-tuned through knowledge collaboration.

[0052] Specifically, input the user's question into the knowledge collaboration large language model to generate a preliminary hypothesized answer for subsequent self-checking. The knowledge collaboration large language model is a large language model fine-tuned through knowledge collaboration. The large language model fine-tuned through knowledge collaboration has professional knowledge in a specific field, such as the agricultural field.

[0053] Based on the self-checking mechanism, determine the rationality of the preliminary hypothesized answer. When the preliminary hypothesized answer is unreasonable, retrieve domain knowledge evidence related to the domain topic words in the user's question.

[0054] Specifically, to avoid unnecessary access to the knowledge base, after obtaining the preliminary hypothesized answer, based on the self-checking mechanism, i.e., the self-reflection mechanism, determine the rationality of the preliminary hypothesized answer. Evaluation model C can be used. 1Determine rationality. When the initially hypothesized answer is unreasonable, retrieve domain knowledge evidence related to the domain topic words in the user's question, thereby highlighting the domain-related characters in the question, that is, the importance of the domain topic words, and then improving the accuracy of retrieval and subsequent model responses, obtaining retrieval results that meet the user's needs, and facilitating subsequent processing. When the initially hypothesized answer is reasonable, directly output the initially hypothesized answer as the final answer to avoid unnecessary knowledge base access and save computing power.

[0055] Semantically and domain-filter the domain knowledge evidence sequentially to generate high-confidence evidence.

[0056] Specifically, perform double filtering of semantics and domain on the domain knowledge evidence sequentially to ensure that the finally used evidence has both semantic relevance and meets the requirements of the relevant domain, such as the professional requirements of the agricultural domain, improving the quality and credibility of the answer. High-confidence evidence can be domain knowledge evidence with a confidence level greater than a preset threshold.

[0057] Generate a final answer based on the user's question and the high-confidence evidence.

[0058] Specifically, based on the user's question, combined with high-confidence evidence, regenerate a response through the knowledge collaborative large language model to obtain a more professional and accurate final answer. Moreover, the generated final answer is not only accurate but also has a rich domain knowledge background, capable of providing valuable decision-making guidance for users. The final answer is as follows:

[0059]

[0060] Among them, y is the final answer, M * is the knowledge collaborative large language model, Q is the user's question, is the high-confidence evidence.

[0061] Exemplarily, for the agricultural domain, for the user's question "What are the most common pests and diseases of corn and how to control them?", the final answer obtained through the large language model domain knowledge collaborative method of the present invention is:

[0062] Common diseases of corn include northern leaf blight, southern leaf blight, and corn rust, etc.; major pests mainly include corn borer, cotton bollworm, etc. Northern Leaf Blight: It occurs in the middle and late growth stages of corn, mainly damaging the corn leaves. In severe cases, it can cause a 20% - 30% reduction in corn yield. Control methods: (1) Select disease-resistant varieties; (2) Strengthen field management, increase the application of phosphorus and potassium fertilizers to improve the disease resistance of plants; (3) Timely remove weeds in the field to reduce the source of bacteria; (4) At the initial stage of disease occurrence, spray a mixture of 800-fold solution of 50% carbendazim wettable powder or 600-fold solution of 75% chlorothalonil wettable powder, or a mixture of 800-fold solution of 50% thiophanate-methyl wettable powder and 600-fold solution of 75% chlorothalonil wettable powder for control. Southern Leaf Blight: It occurs in the middle and late growth stages of corn, mainly damaging the corn leaves. In severe cases, it can cause a 20% - 30% reduction in corn yield. Control methods: (1) Select disease-resistant varieties... Corn Rust: It occurs in the middle and late growth stages of corn, mainly damaging the corn leaves... (4) At the initial stage of disease occurrence, spray 1000-fold solution of 15% triadimefon wettable powder or a mixture of 3000-fold solution of 25% propiconazole EC and 400-fold solution of 25% propiconazole EC plus 1000-fold solution of 15% triadimefon wettable powder for control. Corn Borer: It occurs in the middle and late growth stages of corn, mainly damaging the female ears of corn. In severe cases, it can cause a 20% - 30% reduction in corn yield. Control methods: (1) Select insect-resistant varieties; (2) Strengthen field management, increase the application of phosphorus and potassium fertilizers to improve the disease resistance of plants; (3) Timely remove weeds in the field to reduce the source of insects; (4) During the adult oviposition period, use a sugar-vinegar solution to trap and kill adults; (5) At the peak of larval hatching, irrigate the heart with 1000-fold solution of 90% trichlorfon crystal or spray with 800-fold solution of 90% trichlorfon crystal for control. Cotton Bollworm: It occurs in the middle and late growth stages of corn... (Since the subsequent content is too long, it is not shown here)

[0063] And the response from GPT4 is:

[0064] There are several common pests and diseases of corn. Here are some of them and their control methods: 1. Corn Borer: Description: The corn borer is one of the most common pests during the growth of corn. The larvae mainly cause damage to the corn ears. Control methods: Use biological control such as releasing natural enemy insects, and use the sex pheromone of the corn borer to trap adults; when using chemical control, select pesticides that are relatively safe for humans and livestock for spraying. 2. Southern Rust: Description: Southern rust is a disease caused by fungi, mainly forming orange-red rust spores on the leaves. Control methods: Timely spray fungicides containing triazoles and thiophanate-methyl, etc. 3. Northern Leaf Blight: Description: Northern leaf blight is also a fungal disease, manifested as round or irregular brown lesions on the leaves. Control methods: Strengthen field management, irrigate and fertilize in a timely manner; spray effective fungicides. 4. Gray Leaf Spot: Description: Gray leaf spot forms gray or off-white lesions on the leaves. Control methods: Remove crop residues and use fungicides for early control.

[0065] CLAUDE's reply is as follows:

[0066] The following are several of the most common diseases, pests and their control methods of corn: 1. Corn borer. Control: Timely remove weeds and crop residues in the field and use sex pheromones to trap and kill adults; Sprinkle insecticides in a timely manner, such as chlorantraniliprole, indoxacarb, etc. 2. Northern corn leaf blight. Control: Select disease-resistant varieties: Reasonably close planting to ensure ventilation: Sprinkle fungicides at the initial stage of the disease, such as prochloraz, azoxystrobin, etc. 3. Corn root rot. Control: Crop rotation; Seed treatment: Reasonable fertilization to enhance the resistance of plants; In severe cases, fungicides can be applied to the roots. 4. Corn aphids. Control: Biological control, such as using natural enemies like ladybugs, green lacewings; Sprinkle plant-derived insecticides or chemical insecticides. 5. Corn stalk rot. Control: Select disease-resistant varieties; Avoid continuous cropping; Appropriate close planting to increase ventilation; Timely remove diseased plants.

[0067] TianGong's reply is as follows:

[0068] Corn may encounter various diseases and pests during its growth process. Some of the more common ones include: Northern corn leaf blight: A leaf disease caused by fungi, mainly damaging the leaves, and can also infect the leaf sheaths and husks severely [web page]. Southern corn leaf blight is a disease caused by fungi. The lesions are oval, rectangular or fusiform and expand along the leaf veins [webpage]. Corn sheath blight: A disease caused by fungi, mainly damaging the leaf sheaths, and can also infect the leaves, husks and ears [web page]..... The control measures for the above diseases and pests mainly include: Agricultural control: Including selecting disease-resistant varieties, reasonable crop rotation, sowing at the right time, reasonable fertilization, etc. [webpage]. Biological control: Using natural enemy insects, microbial agents, etc. for control. Chemical control: Using pesticides for control, but attention should be paid to scientific use of pesticides to avoid harm to the environment and human body.

[0069] Expert's reply is as follows:

[0070] The common diseases on corn include northern corn leaf blight, southern corn leaf blight, corn brown spot, corn rust, corn curvularia leaf spot, corn stalk rot, corn sheath blight, corn head smut, corn common smut, maize dwarf mosaic virus, maize rough dwarf disease. In terms of control, taking northern corn leaf blight as an example: (1) Planting disease-resistant varieties (is the most economical and effective measure for controlling northern corn leaf blight), but when breeding and promoting disease-resistant varieties, attention must be paid to reasonable layout, regular rotation, and combined with excellent cultivation and management measures. (2) Strengthen cultivation and management measures and pay attention to removing the source of bacteria. (3) Use some agents, 70% mancozeb, 50% diclomezine, but the specific period and dosage should be used according to the instructions of the product manual or the information registration website of pesticides, and the dosage of pesticides cannot be increased randomly...

[0071] In summary, the final answer obtained by the method for collaborative domain knowledge of the large language model of the present invention is more accurate and professional, has a rich domain knowledge background, and can provide valuable decision-making guidance for users.

[0072] In this embodiment, the knowledge collaborative large language model in the target domain, such as the knowledge collaborative large language model fine-tuned with agricultural domain knowledge, first generates a preliminary hypothesized answer according to the user's question, and then evaluates the rationality of the answer through a self-check mechanism. If the preliminary answer is unreasonable, it retrieves domain knowledge evidence related to the domain topic words in the user's question, and can efficiently and accurately retrieve the domain knowledge evidence in the relevant domain to ensure that the finally provided answer has higher accuracy and reliability. Then, the domain knowledge evidence is sequentially subjected to semantic and domain filtering to ensure the high relevance and high professionalism of the evidence, and then, supported by the high-confidence evidence with high relevance and high professionalism, it answers again based on the user's question, and a more professional and accurate answer can be obtained.

[0073] Optionally, before inputting the user's question into the knowledge collaborative large language model to generate a preliminary hypothesized answer, it further includes:

[0074] Determine the first semantic similarity between the training Q&A data and the domain knowledge data according to the obtained training Q&A data and the corresponding domain knowledge data;

[0075] Determine the loss parameter of the initial large language model according to the first semantic similarity and the initial prediction probability of the initial large language model, where the initial prediction probability is determined by the initial large language model based on the training Q&A data;

[0076] Fine-tune the initial large language model according to the loss parameter to obtain the knowledge collaborative large language model.

[0077] Specifically, the initial prediction probability refers to the probability distribution of the words that may appear at each position in the generated text by the initial large language model without any weight adjustment. The training Q&A data includes training question data and standard answer data. First, determine the first semantic similarity between the data in the standard answer data and the domain knowledge data, and then determine the data with high relevance to the domain according to the first semantic similarity, that is, the data with a relevance greater than the preset relevance threshold, namely the domain highly relevant words, and determine the positions of the domain highly relevant words in the standard answer data. Then, according to the first semantic similarity, adjust the influence of the domain highly relevant words in the initial prediction probability of the initial large language model, enhance the contribution of the domain highly relevant words in the loss function, so that the model pays more attention to these words during the training process, thereby realizing the effective injection of domain knowledge. For example, the effective injection of agricultural domain knowledge, improving the domain consistency and accuracy of the generation result. Then, determine the loss parameter of the initial large language model, and fine-tune the initial large language model based on the loss function to obtain the knowledge collaborative large language model.

[0078] Exemplarily, the large language model can also be used to determine the domain highly relevant words and their positions.

[0079] Optionally, the determining the first semantic similarity between the obtained training Q&A data and the corresponding domain knowledge data includes:

[0080] Segment the standard answer in the training Q&A data to obtain multiple words;

[0081] Determine the first semantic similarity between each of the words and the domain knowledge data.

[0082] Specifically, segment the standard answer in the training Q&A data to obtain multiple words, and use the first semantic similarity formula to determine the first semantic similarity between each word and the domain knowledge data. The first semantic similarity formula includes:

[0083]

[0084] where S i (y i is the first semantic similarity between the i-th word y i and the domain knowledge data K, and are the semantic representations of y i and K respectively.

[0085] Optionally, the determining the loss parameter of the initial large language model according to the first semantic similarity and the initial prediction probability of the initial large language model includes:

[0086] Using a non - linear function, convert the first semantic similarity into a similarity weight;

[0087] Generate a scaled prediction probability according to the similarity weight and the initial prediction probability;

[0088] Determine the loss parameter according to the scaled prediction probability and the cross - entropy loss.

[0089] Specifically, to further enhance the influence of domain - highly relevant words, a non - linear function is used to adjust the semantic similarity matrix, convert the first semantic similarity into a similarity weight, and then according to the similarity weight and the initial prediction probability, use the scaling prediction formula to combine the similarity weight and the initial prediction probability to obtain the adjusted probability, that is, the scaled prediction probability. Finally, according to the scaled prediction probability and the cross - entropy loss, use the cross - entropy formula to determine the loss parameter, so as to update the parameters of the model according to the loss parameter, make the model pay more attention to the domain, improve the model's understanding ability of the content in the agricultural field, and generate more accurate answers.

[0090] The non - linear function includes:

[0091] w i =-ln(1 - S i (y i , K)+λ)+1;

[0092] where w i is the similarity weight, and λ is a non - zero constraint factor.

[0093] The scaling prediction formula includes:

[0094] P w =softmax(w i ·a i );

[0095] where P w is the scaled prediction probability, softmax is a function, and a i is the initial prediction probability,

[0096] The cross - entropy formula includes:

[0097]

[0098] where is the loss parameter, and m is the sequence length.

[0099] Optionally, retrieving the domain knowledge evidence related to the domain - topic words in the user question includes:

[0100] Using a topic - word extraction model to extract the domain - topic words in the user question;

[0101] Based on the domain topic words, perform fuzzy matching in the domain knowledge base to generate the domain knowledge evidence.

[0102] Specifically, use a topic word extraction model to extract the domain topic words in the user's question. The list of extracted domain topic words is as follows:

[0103]

[0104] Among them, T is the list of domain topic words, and t k is the k-th domain topic word, is the topic word extraction model, Q is the user's question. Each domain topic word appears in the user's question. Then, based on each domain topic word, perform fuzzy matching on the knowledge entries in the domain knowledge base, and then use the evidence formula to take the union of the obtained knowledge entries, that is, the relevant domain knowledge, to generate the domain knowledge evidence. The evidence formula includes:

[0105]

[0106] Among them, D Q is the domain knowledge evidence, is the relevant domain knowledge.

[0107] Exemplarily, during retrieval, a pre-trained named entity recognition algorithm can also be used, or a more accurate encoding model can be trained to directly perform retrieval using semantic representations.

[0108] Optionally, the sequential semantic and domain filtering of the domain knowledge evidence to generate high-confidence evidence includes:

[0109] Extract the second semantic similarity between the domain knowledge evidence and the user's question, and based on the second semantic similarity, sort the domain knowledge evidence. According to the sorting result, retain the domain knowledge evidence with a preset ranking to obtain highly relevant evidence;

[0110] Use a domain evaluation model to perform domain screening on the highly relevant evidence to obtain the high-confidence evidence.

[0111] Specifically, use the second semantic similarity formula to perform semantic encoding on the domain knowledge evidence and the user's question, extract the second semantic similarity of the domain knowledge evidence, and based on the second semantic similarity, sort the domain knowledge evidence. Retain the domain knowledge evidence with a preset ranking. The preset ranking can be set according to the actual situation, such as the top ten, to obtain highly relevant evidence. Then use the domain evaluation model C 2 to perform domain screening on the highly relevant evidence to obtain more professional high-confidence evidence.

[0112] The second semantic similarity formula includes:

[0113]

[0114]

[0115]

[0116] Among them, is the second semantic similarity, is the vector representation of the i-th domain knowledge evidence, and Q is the user's question. DPR is a vector semantic encoding method.

[0117] Exemplarily, since ChatGPT has good understanding ability for prompts and contains a vast amount of general knowledge itself, it can be used as a domain evaluation model and a subject term extraction model.

[0118] Optionally, the large language model domain knowledge collaboration method further includes:

[0119] When the preliminary hypothesized answer is reasonable, output the preliminary hypothesized answer.

[0120] Specifically, when the preliminary hypothesized answer is reasonable, to avoid unnecessary knowledge base access and save computing power, directly output the preliminary hypothesized answer.

[0121] Such as Figure 2 As shown, a large language model domain knowledge collaboration device 200 provided by an embodiment of the present invention includes:

[0122] An input module 210, which is used to input the user's question into the knowledge collaboration large language model to generate a preliminary hypothesized answer, where the knowledge collaboration large language model is a large language model fine-tuned through knowledge collaboration;

[0123] A retrieval module 220, which is used to determine the reasonableness of the preliminary hypothesized answer based on a self-check mechanism. When the preliminary hypothesized answer is unreasonable, retrieve domain knowledge evidence related to the domain subject term in the user's question;

[0124] A filtering module 230, which is used to sequentially perform semantic and domain filtering on the domain knowledge evidence to generate high-confidence evidence;

[0125] A generation module 240, which is used to generate a final answer according to the user's question and the high-confidence evidence.

[0126] Such as Figure 3As shown in the figure, an electronic device 300 provided by an embodiment of the present invention includes a memory 310 and a processor 320; the memory 310 is used to store a computer program; the processor 320 is used to implement the above-mentioned method for collaborative knowledge in the field of large language models when executing the computer program.

[0127] Or, an electronic device 300 includes a memory 310 and a processor 320 coupled to the memory 310; the memory 310 is configured to store a computer program; the processor 320 is configured to perform the following operations when executing the computer program:

[0128] Input the user's question into the knowledge collaborative large language model to generate a preliminary hypothesized answer, where the knowledge collaborative large language model is a large language model fine-tuned through knowledge collaboration;

[0129] Based on a self-check mechanism, determine the rationality of the preliminary hypothesized answer. When the preliminary hypothesized answer is unreasonable, retrieve domain knowledge evidence related to the domain topic words in the user's question;

[0130] Sequentially perform semantic and domain filtering on the domain knowledge evidence to generate high-confidence evidence;

[0131] Generate a final answer according to the user's question and the high-confidence evidence.

[0132] A computer-readable storage medium provided by an embodiment of the present invention has a computer program stored thereon. When the computer program is executed by a processor, the above-mentioned method for collaborative knowledge in the field of large language models is implemented.

[0133] Or, a non-volatile computer-readable storage medium has a computer program stored thereon. When the computer program is executed by a processor, the processor is caused to perform the following operations:

[0134] Input the user's question into the knowledge collaborative large language model to generate a preliminary hypothesized answer, where the knowledge collaborative large language model is a large language model fine-tuned through knowledge collaboration;

[0135] Based on a self-check mechanism, determine the rationality of the preliminary hypothesized answer. When the preliminary hypothesized answer is unreasonable, retrieve domain knowledge evidence related to the domain topic words in the user's question;

[0136] Sequentially perform semantic and domain filtering on the domain knowledge evidence to generate high-confidence evidence;

[0137] Generate a final answer according to the user's question and the high-confidence evidence.

[0138] Now, the electronic device 300 that can be a server or a client of the present invention will be described. It is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device 300 is intended to represent various forms of digital electronic computer devices, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 300 can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0139] The electronic device 300 includes a computing unit that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The computing unit, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0140] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc. In the present application, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present invention. In addition, the functional units in various embodiments of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0141] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will all fall within the protection scope of the present invention.

Claims

1. A large language model domain knowledge collaboration method, characterized in that: include: Inputting the user question into the knowledge collaborative large language model to generate a preliminary hypothetical answer, wherein the knowledge collaborative large language model is a large language model that has been fine-tuned by knowledge collaboration; Based on the self-checking mechanism, determine the rationality of the preliminary hypothesized answer, and when the preliminary hypothesized answer is unreasonable, retrieve domain knowledge evidence related to the domain subject words in the user's question; Sequentially performing semantic and domain filtering on the domain knowledge evidence to generate high-confidence evidence; A final answer is generated according to the user question and the high-confidence evidence.

2. The large language model domain knowledge collaboration method according to claim 1, characterized in that: Before inputting the user question into the knowledge collaborative large language model to generate a preliminary hypothetical answer, the method further includes: Determine, based on the acquired training question and answer data and the corresponding domain knowledge data, a first semantic similarity between the training question and answer data and the domain knowledge data; Determining a loss parameter of the initial large language model according to the first semantic similarity and an initial prediction probability of the initial large language model, wherein the initial prediction probability is determined by the initial large language model based on the training question and answer data; According to the loss parameter, the initial large language model is fine-tuned to obtain the knowledge collaborative large language model.

3. The large language model domain knowledge collaboration method according to claim 2, characterized in that: The determining, based on the acquired training question and answer data and the corresponding domain knowledge data, a first semantic similarity between the training question and answer data and the domain knowledge data comprises: Segmenting the standard answers in the training question-answering data to obtain a plurality of words; The first semantic similarity between each of the words and the domain knowledge data is determined.

4. The large language model domain knowledge collaboration method according to claim 2, characterized in that: The step of determining the loss parameter of the initial large language model according to the first semantic similarity and the initial prediction probability of the initial large language model includes: Using a nonlinear function, converting the first semantic similarity into a similarity weight; generating a scaled prediction probability according to the similarity weight and the initial prediction probability; The loss parameter is determined based on the scaled prediction probability and the cross entropy loss.

5. The large language model domain knowledge collaboration method according to claim 1, characterized in that: The retrieving domain knowledge evidence related to the domain subject words in the user question includes: Using a subject word extraction model to extract the domain subject words in the user question; Based on the domain keywords, fuzzy matching is performed in the domain knowledge base to generate the domain knowledge evidence.

6. The large language model domain knowledge collaboration method according to claim 1, characterized in that: The sequentially performing semantic and domain filtering on the domain knowledge evidence to generate high-confidence evidence includes: Extracting a second semantic similarity between the domain knowledge evidence and the user question, and sorting the domain knowledge evidence based on the second semantic similarity, retaining the domain knowledge evidence with a preset ranking according to the sorting result, and obtaining highly relevant evidence; A domain assessment model is used to perform domain screening on the highly relevant evidence to obtain the high confidence evidence.

7. The large language model domain knowledge collaboration method according to claim 1, characterized in that: Also includes: When the preliminary hypothesis answer is reasonable, the preliminary hypothesis answer is output.

8. A large language model domain knowledge collaboration device, characterized in that: include: An input module, used to input user questions into the knowledge collaborative large language model to generate a preliminary hypothetical answer, wherein the knowledge collaborative large language model is a large language model that has been fine-tuned by knowledge collaboration; A retrieval module, used to determine the rationality of the preliminary hypothesized answer based on a self-checking mechanism, and when the preliminary hypothesized answer is unreasonable, retrieve domain knowledge evidence related to the domain subject words in the user's question; A filtering module, used for sequentially performing semantic and domain filtering on the domain knowledge evidence to generate high-confidence evidence; A generation module is used to generate a final answer based on the user question and the high-confidence evidence.

9. An electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the large language model domain knowledge collaboration method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the large language model domain knowledge collaboration method as described in any one of claims 1 to 7 is implemented.

Images