Large model illusion optimization method and device, equipment and medium

Abstract

The invention provides an optimization method and device for large model illusion, equipment and a medium, and relates to the technical field of computers. The method comprises the steps of obtaining a query text; querying in a knowledge vector database based on the query text to obtain a first context candidate set of external knowledge; performing information extraction on the query text to obtain a query text tree; based on the query text tree and the first context candidate set, the visibility of knowledge in the first context candidate set is evaluated, and the visibility of each piece of knowledge is used for representing the degree of correlation between each piece of knowledge in the first context candidate set and the query text; based on the visibility of the knowledge in the first context candidate set, reordering the knowledge in the first context candidate set to obtain a target context candidate set; and inputting the target context candidate set and the query text into the large model for reasoning to obtain a reply text of the query text. According to the embodiment of the invention, the illusion problem of a large model can be inhibited.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method, device, equipment and medium for optimizing large model illusion. Background Art

[0002] In recent years, with the continuous advancement of deep learning technology and the significant improvement of computing resources, large model technology has made unprecedented breakthroughs in the field of natural language processing. By learning massive data sets, these models can not only understand complex semantic structures, but also generate high-quality text content, which has greatly promoted the development of applications such as machine translation, sentiment analysis, and dialogue systems. However, as the scale of the model continues to expand, a phenomenon known as the "hallucination problem" has gradually emerged, becoming a key obstacle to the further development of large models. The so-called "hallucination" means that when the model generates or answers questions, it may generate information that is inconsistent with the facts, or fabricate non-existent facts, which not only affects the accuracy and reliability of the model output, but also has a negative impact on user trust. In addition, due to the complexity of the internal mechanism of the large model, such errors are often difficult to predict and control, which brings uncertainty to the application of the model. Therefore, solving the hallucination problem of large models and improving the credibility and stability of their output has become an important direction of current research.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The present disclosure provides a method, device, equipment and medium for optimizing large model hallucination, which correct the hallucination problem of large models at least to a certain extent.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to one aspect of the present disclosure, a method for optimizing large model hallucination is provided, comprising:

[0007] Obtain the query text, which records the content of the question to be processed;

[0008] Based on the query text, query is performed in the knowledge vector database to obtain a first context candidate set of external knowledge;

[0009] Extract information from the query text to obtain a query text tree;

[0010] Based on the query text tree and the first context candidate set, the visibility of the knowledge in the first context candidate set is evaluated, wherein the visibility of each piece of knowledge is used to indicate the relevance of each piece of knowledge in the first context candidate set to the query text;

[0011] Based on the visibility of the knowledge in the first context candidate set, reorder the knowledge in the first context candidate set to obtain a target context candidate set;

[0012] The target context candidate set and the query text are input into the large model for reasoning to obtain the response text of the query text.

[0013] In one embodiment of the present disclosure, after performing a query in a knowledge vector database based on the query text to obtain a first context candidate set of external knowledge, the method further includes:

[0014] Calculate the similarity value between each piece of knowledge in the first context candidate set and the query text;

[0015] Based on the similarity value, reorder the knowledge in the first context candidate set to obtain a second context candidate set;

[0016] Based on the visibility of knowledge in the first context candidate set, the knowledge in the first context candidate set is reordered to obtain the target context candidate set, including: based on the visibility of knowledge in the second context candidate set, the knowledge in the second context candidate set is reordered to obtain the target context candidate set.

[0017] In one embodiment of the present disclosure, based on the query text tree and the first context candidate set, evaluating the visibility of knowledge in the first context candidate set includes:

[0018] Input the query text tree and each piece of knowledge in the first context candidate set into the deep learning model to evaluate the correlation between the query text tree and each piece of knowledge, and obtain a correlation score;

[0019] Based on the relevance score, visibility of the knowledge in the first context candidate set is determined.

[0020] In one embodiment of the present disclosure, based on the visibility of the knowledge in the second context candidate set, the knowledge in the second context candidate set is reordered to obtain the target context candidate set, including:

[0021] Convert the query text into a vector form to obtain a query vector;

[0022] Input the query vector, visibility, key vector and value vector in the second context candidate set into the self-attention mechanism model to the target context candidate set;

[0023] Among them, the self-attention mechanism model adds visibility as a penalty factor when calculating the output of each head.

[0024] In one embodiment of the present disclosure, a query is performed in a knowledge vector database based on a query text to obtain an external knowledge first context candidate set, including:

[0025] Convert the query text into a vector form to obtain a query vector;

[0026] Based on the query vector, the structure vector in the knowledge vector database is indexed and recalled to obtain the first context candidate set of external knowledge.

[0027] In one embodiment of the present disclosure, before indexing and recalling the structure vector in the knowledge vector database based on the query vector to obtain the first context candidate set of external knowledge, the following steps are included:

[0028] Based on the original text data related to the judicial field, the legal ontology is constructed, which is structured legal knowledge;

[0029] The legal knowledge in the legal ontology is converted into structural vectors and stored in the knowledge vector database.

[0030] In one embodiment of the present disclosure, the original text data related to the judicial field includes multi-source heterogeneous data.

[0031] According to another aspect of the present disclosure, there is provided a device for optimizing large model illusion, comprising:

[0032] A query acquisition module is used to acquire a query text, which records the content of the question to be processed;

[0033] A knowledge query module, used to query the knowledge vector database based on the query text to obtain a first context candidate set of external knowledge;

[0034] An information extraction module is used to extract information from the query text to obtain a query text tree;

[0035] A visibility evaluation module, used to evaluate the visibility of knowledge in the first context candidate set based on the query text tree and the first context candidate set, wherein the visibility of each piece of knowledge in the first context candidate set is used to indicate the relevance of each piece of knowledge in the first context candidate set to the query text;

[0036] A first re-arrangement module, configured to re-arrange the knowledge in the first context candidate set based on the visibility of the knowledge in the first context candidate set to obtain a target context candidate set;

[0037] The model reasoning module is used to input the target context candidate set and the query text into the large model for reasoning to obtain the response text of the query text.

[0038] According to another aspect of the present disclosure, an electronic device is provided, including: a memory for storing instructions; and a processor for calling the instructions stored in the memory to implement the above-mentioned method for optimizing large model illusion.

[0039] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed by a processor, the above-mentioned method for optimizing large model illusion is implemented.

[0040] According to yet another aspect of the present disclosure, a computer program product is provided. The computer program product stores instructions, and when the instructions are executed by a computer, the computer implements the above-mentioned method for optimizing large model illusion.

[0041] According to yet another aspect of the present disclosure, there is provided a chip, comprising at least one processor and an interface;

[0042] An interface for providing program instructions or data to at least one processor;

[0043] At least one processor is used to execute program instructions to implement the above-mentioned optimization method for large model hallucination.

[0044] The optimization method, device, equipment and medium for the large model hallucination provided by the embodiments of the present disclosure re-sort the knowledge in the first context candidate set based on the visibility of the knowledge to form a target context candidate set. This mechanism can not only filter out the most relevant information, but also exclude those knowledge points that may cause misleading or inaccurate answers, effectively reducing the probability of hallucination problems. Finally, the optimized target context candidate set is input into the large model together with the original query text, so that the model can have a more accurate and comprehensive basis when generating replies, thereby improving the quality of the replies.

[0045] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0047] Obviously, the drawings described below are only some embodiments of the present disclosure, and a person skilled in the art can obtain other drawings based on these drawings without any creative work.

[0048] Figure 1A flow chart of a method for optimizing large model hallucination in an embodiment of the present disclosure is shown;

[0049] Figure 2 Another flow chart of a method for optimizing large model illusion in an embodiment of the present disclosure is shown;

[0050] Figure 3 A flow chart of another method for optimizing large model illusion in an embodiment of the present disclosure is shown;

[0051] Figure 4 A flowchart of constructing a legal ontology based on legal data in an embodiment of the present disclosure is shown;

[0052] Figure 5 A flowchart of a visible module structure in an embodiment of the present disclosure is shown;

[0053] Figure 6 A schematic diagram of a device for optimizing large model illusion in an embodiment of the present disclosure is shown;

[0054] Figure 7 A structural block diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0056] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0057] The present exemplary implementation is described in detail below with reference to the accompanying drawings and embodiments.

[0058] First, an embodiment of the present disclosure provides a method for optimizing large model hallucinations, which can be executed by any electronic device with computing and processing capabilities.

[0059] Figure 1A flow chart of an optimization method for large model hallucination in an embodiment of the present disclosure is shown, Figure 1 As shown, the optimization method for large model hallucination provided in the embodiment of the present disclosure includes S101-S106.

[0060] In S101, a query text is obtained, and the query text records the content of the question to be processed.

[0061] The "query text" can be the specific content of the question or request that the user asks the big model. It is the information carrier that the user hopes to get answered or processed. Specifically, the query text can be any form of text input, such as "Is the amount of infringement compensation for invention patents higher than that for utility models", "Please recommend me a good book on intellectual property" and "Principles of infringement determination".

[0062] In S102, a query is performed in a knowledge vector database based on the query text to obtain a first context candidate set of external knowledge.

[0063] The first context candidate set includes multiple pieces of knowledge, each piece of knowledge may be a knowledge text, that is, the first context candidate set includes multiple knowledge texts.

[0064] In S103, information is extracted from the query text to obtain a query text tree.

[0065] Query Text Tree is a method of structured representation of query text. By decomposing query text into entities, relations and attributes, the query content can be understood and processed more clearly.

[0066] The above-mentioned information extraction of the query text may include entity recognition, relationship extraction and attribute extraction. As an example, named entity recognition (NER) technology may be used to extract all entities from the query text. For example, BERT and other models may be used for entity recognition. As an example, the grammatical and semantic relationships between entities may be analyzed to extract the relationships. For example, dependency parsing technology may be used. As an example, modifiers or descriptors of entities may be identified to extract attributes. For example, part-of-speech tagging (POS Tagging) technology may be used.

[0067] In one embodiment, the query text is "find the latest traffic regulations in Beijing". The above entity recognition can identify "Beijing" and "traffic regulations", where Beijing is a location and traffic regulations are a type of regulations. The above relationship extraction can extract "find" and "of", where "find" represents a query action and "of" represents a relationship. The above attribute extraction can extract "latest", which describes the time attribute of the regulations.

[0068] In S104, based on the query text tree and the first context candidate set, the visibility of the knowledge in the first context candidate set is evaluated, wherein the visibility of each piece of knowledge is used to indicate the relevance of each piece of knowledge in the first context candidate set to the query text.

[0069] In some embodiments, the degree of relevance can be expressed by calculating the similarity between the query text tree and each piece of knowledge in the first context candidate set. In some embodiments, visibility can be expressed by a numerical value. In some embodiments, visibility can also be expressed by a visibility category, such as directly relevant, indirectly relevant, and irrelevant.

[0070] In S105 , based on the visibility of the knowledge in the first context candidate set, the knowledge in the first context candidate set is reordered to obtain a target context candidate set.

[0071] In S106, the target context candidate set and the query text are input into the large model for reasoning to obtain the response text of the query text.

[0072] The disclosed embodiment reorders the knowledge in the first context candidate set based on the visibility of the knowledge to form a target context candidate set. This mechanism can not only filter out the most relevant information, but also exclude those knowledge points that may cause misleading or inaccurate answers, effectively reducing the probability of hallucination problems. Finally, the optimized target context candidate set is input into the large model together with the original query text, so that the model can have a more accurate and comprehensive basis when generating replies, thereby improving the quality of the replies.

[0073] Figure 2 A flow chart of an optimization method for large model hallucination in an embodiment of the present disclosure is shown, Figure 2 As shown, the optimization method for large model illusion provided in the embodiment of the present disclosure includes S201-S208, wherein S201-S202 and Figure 1 S101-S102 of the embodiment are the same, and S208 is the same as Figure 1 S106 of the embodiment is the same and will not be described again here.

[0074] In S203, the similarity value between each piece of knowledge in the first context candidate set and the query text is calculated.

[0075] In S204, based on the similarity values, the knowledge in the first context candidate set is reordered to obtain a second context candidate set.

[0076] When reordering the knowledge in the first context candidate set, the knowledge with higher similarity to the query text is ranked higher.

[0077] In S205, information is extracted from the query text to obtain a query text tree.

[0078] In S206 , based on the query text tree and the second context candidate set, the visibility of the knowledge in the second context candidate set is evaluated, wherein the visibility of each piece of knowledge is used to indicate the relevance of each piece of knowledge in the second context candidate set to the query text.

[0079] In S207 , based on the visibility of the knowledge in the second context candidate set, the knowledge in the second context candidate set is reordered to obtain a target context candidate set.

[0080] When reordering the knowledge in the second context candidate set, knowledge with higher visibility is ranked higher.

[0081] In the disclosed embodiment, the knowledge in the first context candidate set is reordered based on the similarity value between each piece of knowledge in the first context candidate set and the query text, so that the knowledge with higher relevance to the query text is ranked higher in the second context candidate set, effectively reducing the probability of occurrence of hallucination problems.

[0082] In some embodiments, the above-mentioned evaluation of the visibility of knowledge in the first context candidate set based on the query text tree and the first context candidate set may be that the query text tree and each piece of knowledge in the first context candidate set are input into a deep learning model to evaluate the correlation between the query text tree and each piece of knowledge to obtain a correlation score; based on the correlation score, the visibility of the knowledge in the first context candidate set is determined.

[0083] In some embodiments, based on the visibility of the knowledge in the second context candidate set, the knowledge in the second context candidate set is reordered to obtain the target context candidate set, including: converting the query text into a vector form to obtain a query vector; inputting the query vector, the visibility, the key vector and the value vector in the second context candidate set into a self-attention mechanism model to obtain the target context candidate set;

[0084] Among them, the self-attention mechanism model adds visibility as a penalty factor when calculating the output of each head.

[0085] It should be noted that each key vector in the second context candidate set may represent feature information of each piece of knowledge, that is, a feature vector of the knowledge; and the value vector in the second context candidate set may be a content vector of each piece of knowledge.

[0086] In some embodiments, the above-mentioned querying in the knowledge vector database based on the query text to obtain the first context candidate set of external knowledge includes: converting the query text into a vector form to obtain a query vector; indexing and recalling the structural vector in the knowledge vector database based on the query vector to obtain the first context candidate set of external knowledge.

[0087] In some embodiments, before the above-mentioned indexing and recalling of structural vectors in the knowledge vector database based on the query vector to obtain the first context candidate set of external knowledge, it includes: constructing a legal ontology based on original text data related to the judicial field, and the legal ontology is structured legal knowledge; converting the legal knowledge in the legal ontology into a structural vector and storing it in the knowledge vector database.

[0088] In some embodiments, the original text data related to the judicial field includes multi-source heterogeneous data.

[0089] The disclosed embodiments can be used for hallucination suppression of large models in the judicial field. Taking multi-source heterogeneous data, such as laws and regulations, legal documents, case books, Wikipedia, etc., abstracting the ontology structure can further reduce the redundancy, repetitiveness and irrelevant noise of knowledge.

[0090] The disclosed embodiments can quickly process and analyze large amounts of judicial data, such as case documents, legal provisions, etc., to help judicial personnel obtain the required information more quickly, thereby improving judicial efficiency and alleviating the contradiction between too many cases and too few judges. It provides judicial personnel with intelligent assistance, such as case prediction and risk assessment. This helps judicial personnel make more informed decisions and improves judicial quality. Large models can reduce the impact of human factors on judicial decision-making and improve judicial fairness. For example, large models can help judges understand and apply laws and regulations more accurately, and avoid unfair judgments due to personal bias or errors.

[0091] It should be noted that, in addition to the judicial field, the embodiments of the present disclosure can also be used to solve common problems in other vertical business fields.

[0092] Figure 3 A flow chart of an optimization method for large model hallucination in an embodiment of the present disclosure is shown, Figure 3As shown, in the disclosed embodiment, a dynamic and updateable structural knowledge system (the legal ontology in the previous text) is first constructed based on a large amount of legal and regulatory data. Compared with the original text data, the ontology structure is clearer in structure and semantically clear, more logical and factual, more abstract than the knowledge graph, and more suitable for the judicial field with a large number of domain terms and rigorous requirements. Then the structural knowledge is vectorized and stored in the vector database (the knowledge vector database in the previous text), and the external knowledge is retrieved and recalled based on the query to obtain the context; here, an additional structural visibility module is introduced, in which a visibility mechanism is introduced to linearize and selectively implant knowledge to reduce the impact of knowledge noise and heterogeneity, improve model performance, and suppress the illusion of a large model; finally, questions are asked to the large model based on the obtained context and query to obtain the result.

[0093] Reference below Figure 4 The process of constructing a legal ontology based on legal data in the above-mentioned embodiment is described in detail. The disclosed embodiment constructs an ontology structure knowledge system, namely a legal ontology, based on massive legal and regulatory data. By associating the entity and relationship information in the ontology with the task, external knowledge can be introduced into the large model to increase the interpretability and factuality of the information. On the other hand, by reasoning about the information in the ontology through the ontology reasoning engine, more valid information can be derived based on the information in the external knowledge base, and the content of the ontology can be enriched to mine the implicit information in the data. At the same time, by comparing and analyzing the ontology with the external knowledge base in real time, inaccurate, incomplete or outdated information in the external knowledge base can be discovered, and it can be corrected and supplemented, which can then provide more accurate and complete knowledge, improve factuality, and reduce the rate of hallucinations. This part of the process is as follows:

[0094] (1) Data acquisition: First, obtain external knowledge graph sources. Due to the characteristics of the ontology data structure, multi-source heterogeneous data can be used, such as laws and regulations, legal documents, case files, encyclopedia data, etc. Abstracting the ontology structure can further reduce the redundancy, repetition and irrelevant noise of knowledge.

[0095] (2) Data extraction: For downstream applications in the legal field, extract ontology text from external knowledge. This may include the extraction of entities, relationships, attributes, etc. The extraction method is defined based on the structure and semantic rules of the ontology.

[0096] (4) Data processing: Clean and preprocess the extracted ontology text, including removing noise, processing missing values, and standardizing text formats.

[0097] (5) Ontology construction: Ontology is constructed using the cleaned and preprocessed ontology text. This can be done by using an ontology modeling language or an ontology construction tool. During the ontology construction process, ontology data can be inferred to enrich the ontology content, and by comparing and analyzing with external knowledge bases, inaccurate, incomplete or outdated information in the external knowledge base can be corrected and supplemented, and the corrected information can be used to provide more accurate and complete knowledge. Compared with the original text data, the ontology structure is clearer and more semantically explicit, more logical and factual, and more abstract than the knowledge graph. It is more suitable for the judicial field with a large number of domain terms and rigorous requirements.

[0098] (6) Ontology verification and evaluation: Verify and evaluate the constructed ontology. This can include checking the consistency, completeness, and correctness of the ontology to ensure that it conforms to the facts, further assist in improving the accuracy of knowledge injection, improving the accuracy of model answers, and suppressing hallucinations.

[0099] (7) Ontology data model update: Iteratively update the ontology data model based on external data links to ensure the timeliness of the data.

[0100] After constructing the legal ontology, the vector knowledge base is generated for the structural knowledge obtained above, that is, the legal knowledge in the legal ontology is converted into structural vectors and stored in the knowledge vector database; and information retrieval, screening and sorting are performed on the user input questions to obtain retrieval information. The process of this stage is as follows:

[0101] (1) Determine a vector model, and perform vectorization conversion on the ontology data based on the selected vector model to obtain ontology vector data.

[0102] (2) Determine a vector database and store the converted vector into the vector database.

[0103] (3) Determine the retrieval method, parse the user input query, vectorize the text based on the parsed content, index and recall the structure vector in the database based on the query vector, and use different indexing algorithms to quickly obtain the first context candidate set of external knowledge.

[0104] (4) Determine a re-ranking algorithm, re-rank the obtained external knowledge context candidate set, and obtain an external knowledge second context candidate set. An algorithm such as a semantic model to calculate similarity can be used.

[0105] The disclosed embodiment proposes a mechanism for automatically screening the recalled ontology structure knowledge, which can make effective knowledge visible and shield noise knowledge, thereby effectively solving the impact of noise knowledge and improving the effect of knowledge introduction. According to the needs of different downstream tasks in the legal field, the injection of the formed external structure knowledge is guided to reduce the impact of knowledge noise and avoid the large model illusion problem caused by low-quality external knowledge. On the other hand, compared with the original text data, the visibility calculation of the abstract ontology structure has a clearer structure and clearer semantics, stronger logic and facts, and is more suitable for the judicial field with a large number of domain terms and rigorous requirements. Figure 5 As shown, the module process is as follows:

[0106] (1) Obtain the user's query text and extract information from the query text, including entities, relationships, attributes, etc., to form a query text tree.

[0107] (2) The query text tree and the knowledge in the second context candidate set are respectively input into the visibility evaluator to perform data visibility calculation and evaluate the visibility of the knowledge. The recalled knowledge is divided into three types: directly related to the query, indirectly related, and unrelated.

[0108] Directly related: Data that is directly related to the user input and the answer to the question. The retrieved data directly contains the answer to the question. This type of data injection usually has a beneficial effect on the output of large models, and the evaluation module has the highest visibility of directly related data.

[0109] Indirectly related: Data that is semantically related to user input and the answer to the question, but the retrieved data does not contain the answer to the question. The injection of such data may interfere with the output of the large model, making the model effect unstable. The evaluation module has reduced visibility of indirectly related data and is only visible to valid data.

[0110] Irrelevant: The retrieved data has no relevance to the user’s data and the answer to the question. This type of data is usually noise data and will have a negative impact on the model’s answer. Irrelevant data has the lowest visibility.

[0111] The model of the visible module can be modeled by different deep learning models. By calculating the correlation between the query data and the recalled knowledge, the corresponding visibility is obtained by linear transformation fitting. Given the input query data x and the second context candidate knowledge set K = {k1, k2, k3, k4...kN}, the visible evaluator outputs visibility y, and the model predicts P(y|x, k). The results obtained by the model are standardized to obtain visibility distributed between (0, 1).

[0112] (3) The visibility of the knowledge in the second context candidate set inferred by the visibility evaluator is input into the visibility module to obtain the final context.

[0113] Specifically, given the input query x and the second context candidate knowledge set K = {k1, k2, ... kN}, the visibility S = {s1, s2, ... sN} is obtained. A visibility network is established based on the self-attention mechanism, and the visibility S of the context candidate set is input into the weight network as prior knowledge to achieve the ability to make effective knowledge visible and shield noise data. The specific process is as follows:

[0114] Linear transformation: First, the query Q, key K (from the context), and value V (also from the context) are linearly transformed. This is usually done by multiplying different weight matrices to obtain multiple linearly transformed Q, K, and V.

[0115] Calculate the score: For each head, calculate the dot product of the query and the key, and then divide it by a scaling factor (usually the square root of the key dimension) to scale it to get the attention score.

[0116] Apply the Softmax function: Normalize the attention score through the Softmax function to get the attention weight.

[0117] Weighted summation: The attention weights are weighted summed with the values, and the visibility S is added as a penalty factor to obtain the output of each head.

[0118] Concatenation and linear transformation: The outputs of multiple heads are concatenated and a final linear transformation is performed to obtain the fusion result of the multi-head attention mechanism.

[0119] The vector after the above attention transformation is reversely mapped back to the text to obtain the final injected external knowledge context data.

[0120] Finally, the obtained context information is integrated with the user query and prompt words, and input into the large model for reasoning to obtain the task result, including the following steps:

[0121] (1) Determine the inference model: Determine the inference model. The inference model needs to take into account multiple factors such as downstream task effects and resource consumption.

[0122] (2) Large model query: The final context is used as the input and guidance signal of the generative model, and is input into the large model together with the original input query to produce the final answer or other forms of text.

[0123] (3) Post-processing: Post-process the generated results, such as removing repetitions, adjusting sentence structure, etc., to obtain better generation effects.

[0124] The disclosed embodiment can not only solve the data limitation problem caused by knowledge heterogeneity by constructing a dynamically updated structural knowledge system based on ontology and facts, but also ensure the reliability, real-time and traceability of knowledge injected into the large model. On the other hand, a structural visibility mechanism is proposed, which can further improve the accuracy of the large model and suppress the hallucination problem of the large model based on knowledge linearization and selective implantation to reduce the impact of knowledge noise.

[0125] In the embodiments of the present disclosure, the terms “first”, “second” and “third” are used for descriptive purposes only and should not be understood as indicating or implying relative importance.

[0126] The term "and / or" in this disclosure is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0127] Furthermore, although the steps of the method in the present disclosure are described in a particular order in the drawings, this does not require or imply that the steps must be performed in this particular order or that all the steps shown must be performed to achieve the desired results.

[0128] In some embodiments, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0129] Based on the same inventive concept, the present disclosure also provides an optimization device for large model illusion, as described in the following embodiments. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0130] Figure 6 A schematic diagram of an optimization device for large model illusion in an embodiment of the present disclosure is shown, Figure 6 As shown, the optimization device for large model hallucination includes a query acquisition module 601, a knowledge query module 602, an information extraction module 603, a visibility evaluation module 604, a first rearrangement module 605 and a model reasoning module 606.

[0131] The query acquisition module 601 is used to acquire a query text, which records the content of the question to be processed;

[0132] A knowledge query module 602 is used to query the knowledge vector database based on the query text to obtain a first context candidate set of external knowledge;

[0133] An information extraction module 603 is used to extract information from the query text to obtain a query text tree;

[0134] A visibility evaluation module 604 is used to evaluate the visibility of the knowledge in the first context candidate set based on the query text tree and the first context candidate set, wherein the visibility of each piece of knowledge in the first context candidate set is used to indicate the relevance of each piece of knowledge in the first context candidate set to the query text;

[0135] A first reordering module 605, configured to reorder the knowledge in the first context candidate set based on the visibility of the knowledge in the first context candidate set to obtain a target context candidate set;

[0136] The model reasoning module 606 is used to input the target context candidate set and the query text into the large model for reasoning to obtain the response text of the query text.

[0137] In some embodiments, the device for optimizing large model illusion further includes a second rearrangement module.

[0138] The second re-ranking module is used to perform a query in the knowledge vector database based on the query text, and after obtaining the first context candidate set of external knowledge, calculate the similarity value between each piece of knowledge in the first context candidate set and the query text; based on the similarity value, re-rank the knowledge in the first context candidate set to obtain the second context candidate set;

[0139] The first reordering module 605 is used to reorder the knowledge in the second context candidate set based on the visibility of the knowledge in the second context candidate set to obtain a target context candidate set.

[0140] In some embodiments, the visibility evaluation module 604 is used to input the query text tree and each piece of knowledge in the first context candidate set into a deep learning model to evaluate the correlation between the query text tree and each piece of knowledge to obtain a correlation score; based on the correlation score, the visibility of the knowledge in the first context candidate set is determined.

[0141] In some embodiments, the first rearrangement module 605 is used to convert the query text into a vector form to obtain a query vector; the query vector, visibility, key vector and value vector in the second context candidate set are input into the self-attention mechanism model to the target context candidate set; wherein the self-attention mechanism model adds visibility as a penalty factor when calculating the output of each head.

[0142] In some embodiments, the knowledge query module 602 is used to convert the query text into a vector form to obtain a query vector; and to index and recall the structure vector in the knowledge vector database based on the query vector to obtain an external knowledge first context candidate set.

[0143] In some embodiments, before indexing and recalling the structural vectors in the knowledge vector database based on the query vector to obtain the first context candidate set of external knowledge, it includes: constructing a legal ontology based on original text data related to the judicial field, and the legal ontology is structured legal knowledge; converting the legal knowledge in the legal ontology into a structural vector and storing it in the knowledge vector database.

[0144] In some embodiments, the original text data related to the judicial field includes multi-source heterogeneous data.

[0145] The concepts of “first”, “second”, etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0146] Regarding the device for optimizing the large model illusion in the above-mentioned embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method for optimizing the large model illusion, and will not be elaborated here.

[0147] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory.

[0148] In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0149] Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0150] Refer to the following Figure 7 To describe the electronic device provided by the embodiment of the present disclosure. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0151] Figure 7 FIG. 1 is a schematic diagram showing the architecture of an electronic device 700 provided by an embodiment of the present disclosure. Figure 7 As shown, the electronic device 700 includes but is not limited to: at least one processor 710 and at least one memory 720.

[0152] The memory 720 is used to store instructions.

[0153] In some embodiments, the memory 720 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 7201 and / or a cache memory unit 7202 , and may further include a read-only memory unit (ROM) 7203 .

[0154] In some embodiments, the memory 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0155] In some embodiments, the memory 720 may store an operating system, which may be a real-time operating system (RTX), LINUX, UNIX, WINDOWS, or OS X.

[0156] In some embodiments, data may also be stored in the memory 720 .

[0157] As an example, the processor 710 may read data stored in the memory 720 , where the data may be stored at the same storage address as the instruction, or the data may be stored at a different storage address than the instruction.

[0158] The processor 710 is used to call the instructions stored in the memory 720 to implement the steps of various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processor 710 can execute the steps of the above-mentioned optimization method embodiment for large model hallucination.

[0159] It should be noted that the processor 710 may be a general-purpose processor or a special-purpose processor. The processor 710 may include one or more processing cores, and the processor 710 executes various functional applications and data processing by running instructions.

[0160] In some embodiments, the processor 710 may include a central processing unit (CPU) and / or a baseband processor.

[0161] In some embodiments, the processor 710 may determine an instruction according to the priority identifier and / or function category information carried in each control instruction.

[0162] In the present disclosure, the processor 710 and the memory 720 may be provided separately or integrated together.

[0163] As an example, the processor 710 and the memory 720 may be integrated on a single board or a system on chip (SOC).

[0164] like Figure 7 As shown, the electronic device 700 is in the form of a general-purpose computing device. The electronic device 700 may further include a bus 730 .

[0165] Bus 730 may be a bus representing one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0166] The electronic device 700 may also communicate with one or more external devices 740 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 700, and / or any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 750.

[0167] Furthermore, the electronic device 700 can also communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through the network adapter 760 .

[0168] like Figure 7 As shown, the network adapter 760 communicates with other modules of the electronic device 700 via the bus 730 .

[0169] It should be understood that although not shown in the figures, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0170] It is to be understood that the structure shown in the embodiment of the present disclosure does not constitute a specific limitation on the electronic device 700. In other embodiments of the present disclosure, the electronic device 700 may include: Figure 7 More or fewer components may be shown, or some components may be combined or separated, or the components may be arranged differently. Figure 7 The components shown may be implemented in hardware, software or a combination of software and hardware.

[0171] The present disclosure also provides a computer-readable storage medium having computer instructions stored thereon. When the computer instructions are executed by a processor, the method for optimizing the large model illusion described in the above method embodiment is implemented.

[0172] The computer-readable storage medium in the embodiments of the present disclosure is a computer instruction that can be sent, propagated or transmitted for use by or in conjunction with an instruction execution system, apparatus or device.

[0173] As one example, computer readable storage media are non-volatile storage media.

[0174] In some embodiments, more specific examples of computer-readable storage media in the present disclosure may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, a USB flash drive, a mobile hard disk, or any suitable combination of the foregoing.

[0175] In the embodiments of the present disclosure, the computer-readable storage medium may include a data signal propagated in a baseband or as a part of a carrier wave, in which computer instructions (readable program codes) are carried.

[0176] Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.

[0177] In some examples, computing instructions contained on a computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0178] The embodiments of the present disclosure also provide a computer program product, which stores instructions. When the instructions are executed by a computer, the computer implements the method for optimizing the large model illusion described in the above method embodiment.

[0179] The above instructions may be program codes. In specific implementation, the program codes may be written in any combination of one or more programming languages.

[0180] The programming language includes object-oriented programming languages, such as Java, C++, etc., and also includes conventional procedural programming languages, such as "C" language or similar programming languages.

[0181] The program code may execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server.

[0182] Where a remote computing device is involved, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0183] The embodiment of the present disclosure also provides a chip, including at least one processor and an interface;

[0184] An interface for providing program instructions or data to at least one processor;

[0185] At least one processor is used to execute program instructions to implement the optimization method for large model hallucination described in the above method embodiment.

[0186] In some embodiments, the chip may further include a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0187] Those skilled in the art will appreciate that all or part of the steps for implementing the above embodiments may be implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0188] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein.

[0189] The present disclosure is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not disclosed in the present disclosure. The description and examples are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A method for optimizing large model hallucination, characterized in that: include: Obtaining a query text, wherein the query text records the content of the question to be processed; Based on the query text, query is performed in the knowledge vector database to obtain a first context candidate set of external knowledge; Extracting information from the query text to obtain a query text tree; Based on the query text tree and the first context candidate set, the visibility of the knowledge in the first context candidate set is evaluated, wherein the visibility of each piece of knowledge is used to indicate the relevance of each piece of knowledge in the first context candidate set to the query text; Based on the visibility of the knowledge in the first context candidate set, reorder the knowledge in the first context candidate set to obtain a target context candidate set; The target context candidate set and the query text are input into a large model for reasoning to obtain a response text for the query text.

2. The method according to claim 1, characterized in that After performing a query in the knowledge vector database based on the query text to obtain the external knowledge first context candidate set, the method further includes: Calculate the similarity value between each piece of knowledge in the first context candidate set and the query text; Based on the similarity value, reorder the knowledge in the first context candidate set to obtain a second context candidate set; The method of reordering the knowledge in the first context candidate set based on the visibility of the knowledge in the first context candidate set to obtain the target context candidate set includes: reordering the knowledge in the second context candidate set based on the visibility of the knowledge in the second context candidate set to obtain the target context candidate set.

3. The method according to claim 1, characterized in that The step of evaluating visibility of knowledge in the first context candidate set based on the query text tree and the first context candidate set includes: Input the query text tree and each piece of knowledge in the first context candidate set into a deep learning model to evaluate the correlation between the query text tree and each piece of knowledge, and obtain a correlation score; Based on the relevance score, visibility of knowledge in the first context candidate set is determined.

4. The method according to claim 2, characterized in that: The step of reordering the knowledge in the second context candidate set based on the visibility of the knowledge in the second context candidate set to obtain a target context candidate set includes: Convert the query text into a vector form to obtain a query vector; Input the query vector, the visibility, the key vector and the value vector in the second context candidate set into the self-attention mechanism model to the target context candidate set; Among them, the self-attention mechanism model adds the visibility as a penalty factor when calculating the output of each head.

5. The method according to claim 1, characterized in that The querying in the knowledge vector database based on the query text to obtain the external knowledge first context candidate set includes: Convert the query text into a vector form to obtain a query vector; Based on the query vector, the structure vector in the knowledge vector database is indexed and recalled to obtain a first context candidate set of external knowledge.

6. The method according to claim 5, characterized in that Before indexing and recalling the structure vector in the knowledge vector database based on the query vector to obtain the first context candidate set of external knowledge, the method includes: Based on the original text data related to the judicial field, construct a legal ontology, wherein the legal ontology is structured legal knowledge; The legal knowledge in the legal ontology is converted into a structural vector and stored in the knowledge vector database.

7. The method according to claim 6, characterized in that The original text data related to the judicial field includes multi-source heterogeneous data.

8. An optimization device for large model illusion, characterized in that: include: A query acquisition module, used to acquire a query text, wherein the query text records the content of the question to be processed; A knowledge query module, used to query the knowledge vector database based on the query text to obtain a first context candidate set of external knowledge; An information extraction module, used to extract information from the query text to obtain a query text tree; A visibility evaluation module, used to evaluate the visibility of knowledge in the first context candidate set based on the query text tree and the first context candidate set, wherein the visibility of each piece of knowledge in the first context candidate set is used to indicate the relevance of each piece of knowledge in the first context candidate set to the query text; A first re-arrangement module, configured to re-arrange the knowledge in the first context candidate set based on the visibility of the knowledge in the first context candidate set to obtain a target context candidate set; The model reasoning module is used to input the target context candidate set and the query text into the large model for reasoning to obtain the response text of the query text.

9. An electronic device, characterized in that: include: A memory for storing instructions; A processor is used to call the instructions stored in the memory to implement the method for optimizing the large model illusion as described in any one of claims 1-7.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the method for optimizing large model illusion described in any one of claims 1 to 7 is implemented.

Images