Method for reducing hallusion of large model in vertical field

Abstract

The invention discloses a method for reducing hallusion of a large model in a vertical field, which comprises the following steps of: performing multi-path recall according to input original data, and fusing the recalled data according to a recall strategy to form a recall result; calculating the correlation between the recall results and the original data, sorting the recall results according to a sorting strategy, and deleting the recall results with low correlation; recombining the original data according to a recall result; inputting the recombined original data into a word embedding model to obtain processing result data; a word embedding model of vertical domain data fining is used, a recall strategy and a sorting strategy are adopted for original data, the original data are recombined and then input into the word embedding model, and meanwhile personalized features of a user are increased according to user identity information. And then knowledge recall in the vertical field is carried out according to the data recombined by the user and personalized characteristics, so that the illusion phenomenon of a large model in the vertical field is reduced, and meanwhile, the overall accuracy of the word embedding model during question answering is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vertical field large models, and in particular to a method for reducing the illusion of vertical field large models. Background Art

[0002] The application of vertical big models in various industries is increasing, and they show great potential in some emerging industries. However, the problem of vertical big model illusion gradually emerges in the application of big models in various industries.

[0003] At present, the conventional method to solve the illusion of large models in vertical fields is to use local knowledge base technology. The main solution to the problem of large model illusion is to use Chain-of-Thought COT technology. COT is a method to solve the problem of large model illusion. It improves the accuracy and reliability of large models generating answers to questions by simulating the human thinking process. COT simulates logical reasoning and gradually demonstrates the thinking process, helping large models to perform logical reasoning before generating answers to questions. By limiting the creativity of the model, it avoids over-generalization of the model without sufficient basis, thereby reducing the model's hallucination ability. The use of the chain of thought method requires users to provide a lot of reasoning processes, which will be a certain burden for users. At the same time, this solution is not suitable for all types of problems, especially some problems that require professional knowledge or common sense. Summary of the invention

[0004] In view of the shortcomings of the prior art, the object of the present invention is to provide a method for reducing the illusion of large models in vertical fields.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for reducing the illusion of a large model in a vertical domain, comprising the steps of:

[0007] Perform multi-channel recall based on the input raw data, and integrate the recalled data according to the recall strategy to form the recall result;

[0008] Calculate the correlation between the recall results and the original data, sort the recall results according to the sorting strategy, and delete the recall results with low correlation;

[0009] Reorganize the original data based on the recall results;

[0010] The reorganized original data is input into the word embedding model to obtain the processed result data.

[0011] In the present invention, preferably, the multi-channel recall is respectively vector library recall, DSSM model recall and keyword co-occurrence recall.

[0012] In the present invention, preferably, the DSSM double-tower model is used to perform the multi-way recall.

[0013] In the present invention, preferably, it also includes obtaining user data corresponding to the original data, generating personalized features based on the user data, and inputting the personalized features into the word embedding model to train the word embedding model.

[0014] In the present invention, preferably, in the process of recalling the vector library, the recalled data is filtered by using keyword co-occurrence.

[0015] In the present invention, preferably, the recombining specifically comprises the steps of:

[0016] Insert the recall results with high relevance into the header or end of the original data.

[0017] In the present invention, preferably, the NSP task of the BERT model is used to calculate the CLS scores of the recall result and the original data to obtain the correlation between the recall result and the original data.

[0018] In the present invention, preferably, the sorting strategy is to sort the recall results from large to small according to the CLS scores, and retain the top 10 data with higher scores.

[0019] In the present invention, preferably, the step of: training a word embedding model capable of understanding vertical domain knowledge, wherein the word embedding model adopts a BERT model.

[0020] In the present invention, preferably, industry data in a vertical field is obtained, first split according to paragraphs, chapters, and knowledge points, and question-answer data is formed for the paragraphs, chapters, and knowledge points. Then, data cleaning is performed to construct a training task data set, and the training task data set is used to train the word embedding model.

[0021] In the present invention, preferably, the MASK strategy of the BERT model is adopted when training the word embedding model.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] The method of the present invention uses a word embedding model fine-tuned by vertical field data, adopts a recall strategy and a sorting strategy for the original data, reorganizes the original data and then inputs it into the word embedding model, and at the same time adds the user's personalized features according to the user's identity information as a part of training the word embedding model, and then recalls the vertical field knowledge according to the user's reorganized data, thereby reducing the hallucination phenomenon of the large model in the vertical field, enabling the word embedding model to better utilize the information in these documents to answer user questions when answering questions, thereby improving the overall accuracy of the word embedding model in answering questions. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The figure is a flow chart of a method for reducing the illusion of a large model in a vertical field according to the present invention.

[0025] Figure 2 A schematic diagram of the process for generating the recall results described in the invention.

[0026] Figure 3 Schematic diagram of the word embedding model training process described in the invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0029] Please also see Figures 1 to 3 A preferred embodiment of the present invention provides a method for reducing the hallucination of a large model in a vertical field, by using a fine-tuning embedding model of vertical field data to train a recall model and a sorting model. According to the user identity information, the personalized features of the user are added as part of the training word embedding model, and then the knowledge of the vertical field is recalled according to the user text data and the personalized features, and the recalled knowledge sorting and reorganized knowledge process are added to reduce the hallucination phenomenon of the large model in the vertical field, including the steps of:

[0030] Perform multi-channel recall based on the input raw data, and integrate the recalled data according to the recall strategy to form the recall result;

[0031] Calculate the correlation between the recall results and the original data, sort the recall results according to the sorting strategy, and delete the recall results with low correlation;

[0032] Reorganize the original data based on the recall results;

[0033] The reorganized original data is input into the word embedding model to obtain the processed result data.

[0034] In this embodiment, for the input original data, that is, the prompt input by the user, the DSSM twin-tower model is used to perform the multi-way recall, and the multi-way recall is respectively vector library recall, DSSM model recall and keyword co-occurrence recall, and then the data of the three-way recall is integrated with the recall strategy to form the final recall result. The DSSM twin-tower model can not only obtain the low-dimensional semantic vector expression sentence embedding of the sentence, but also predict the semantic similarity of the two sentences. The ones with high semantic similarity will be recalled. Among them, it is possible that each word in the vector library is different, the semantics are similar, and it is still recalled, so the keyword co-occurrence is used for filtering later. At the same time, in order to improve the generalization ability and accuracy, we rewrite the original data submitted by the user and recall the rewritten original data.

[0035] Specifically, vector library recall calculates the vectors between the questions asked by users, i.e., the original data, and the knowledge in the knowledge base, and recalls the knowledge with high similarity. DSSM model recall uses the trained DSSM model to predict the relevance between the user's questions (original data) and the knowledge in the knowledge base, and recalls the knowledge with high relevance. Keyword co-occurrence recall extracts the keywords in the user's questions, and then recalls the knowledge related to the keywords in the knowledge base. The knowledge recalled from multiple channels is integrated with the recall strategy. First, duplicates are removed, and then the knowledge with low relevance in each algorithm is removed, and the knowledge with high relevance is retained and integrated together.

[0036] In this embodiment, the DSSM twin-tower model can embed existing question-answer pairs in advance to improve efficiency during recall. The DSSM twin-tower model constructs original data embedding of features on the original data side and knowledge embedding of features on the vertical domain knowledge side. These two embeddings are trained through complex deep learning networks. During the training phase, the DSSM twin-tower model uses user search raw data and massive vertical domain knowledge data to maximize the cosine similarity between the original data and the vertical domain knowledge semantic vectors. During the recall phase, the semantic similarity is represented by calculating the cosine similarity distance between the two semantic vectors, and those with high semantic similarity will be recalled.

[0037] In this embodiment, the NSP task of the BERT model is used to calculate the CLS scores of the recall results and the original data to obtain the correlation between the recall results and the original data. The sorting strategy is to sort the recall results from large to small according to the CLS scores, retain the top 10 data with higher scores, and thus obtain the final sorted recall results. By sorting and removing the recall results with low correlation, the generalization ability of the word embedding model is reduced.

[0038] In this embodiment, due to the Lost in the middle phenomenon of the word embedding model, when processing tasks that require identifying and understanding relevant contextual information, the word embedding model is very sensitive to the location of relevant information. Therefore, the recall results are used to reorganize the original data. Specifically, the recall results with high relevance are inserted into the header or end of the original data, so that the word embedding model is effective in processing tasks with contextual information.

[0039] In this embodiment, it also includes obtaining user data corresponding to the original data, generating personalized features based on the user data, and using the personalized features to train the word embedding model. Adding personalized features can enable the word embedding model to understand more detailed user information, and ultimately be able to answer user questions more specifically, that is, to provide more accurate processing result data for the original data.

[0040] In this embodiment, the step of training a word embedding model capable of understanding vertical domain knowledge is also included, and the word embedding model adopts a BERT model.

[0041] In this embodiment, the industry data in the vertical field is obtained, and first split according to paragraphs, chapters, and knowledge points, and question-answer data is formed for paragraphs, chapters, and knowledge points. The question-answer data is manually reviewed, so that the data is processed into the required training data, and a training task data set is constructed. The training task data set is used to train the word embedding model. It provides basic data support for the training of the word embedding model, and also forms a question-answer knowledge base with the split knowledge.

[0042] In this embodiment, the word embedding model adopts the BERT model, and uses the training task data set for fine-tuning to form a word embedding model with the ability to understand vertical domain knowledge. The MASK strategy of the BERT model is adopted during training. The Mask strategy of BERT is adopted to randomly cover or replace part of the characters or words in the input text, and then let the model predict the covered or replaced part through context understanding, so as to train the model's ability to capture semantic information. Through this strategy, the word embedding model needs to learn contextual information to predict the covered or replaced part, thereby improving the word embedding model's ability to understand semantics. The loss function only calculates the loss of the covered or replaced part to prompt the model to focus on these modified parts, so as to better understand the semantics of the text.

[0043] In some other preferred embodiments of the present invention, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the method described in the above embodiment.

[0044] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0045] The above description is a detailed description of the preferred feasible embodiments of the present invention, but the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modified changes completed under the technical spirit suggested by the present invention should fall within the patent scope covered by the present invention.

Claims

1. A method for reducing the illusion of large models in vertical fields, characterized in that: Includes steps: Perform multi-channel recall based on the input raw data, and integrate the recalled data according to the recall strategy to form the recall result; Calculate the correlation between the recall results and the original data, sort the recall results according to the sorting strategy, and delete the recall results with low correlation; Reorganize the original data based on the recall results; The reorganized original data is input into the word embedding model to obtain the processed result data.

2. A method for reducing the illusion of large models in vertical fields according to claim 1, characterized in that: The DSSM dual-tower model is used to perform the multi-way recall, and the multi-way recall includes vector library recall, DSSM model recall and keyword co-occurrence recall.

3. A method for reducing the illusion of large models in vertical domains according to claim 1, characterized in that: It also includes obtaining user data corresponding to the original data, generating personalized features based on the user data, and inputting the personalized features into the word embedding model to train the word embedding model.

4. A method for reducing the illusion of large models in vertical fields according to claim 2, characterized in that: In the process of recalling the vector library, the recalled data is filtered by using keyword co-occurrence.

5. The method of reducing the illusion of large models in vertical domains according to claim 1, characterized in that: The reorganization specifically comprises the following steps: Insert the recall results with high relevance into the header or end of the original data.

6. A method for reducing the illusion of large models in vertical domains according to claim 1, characterized in that: The NSP task of the BERT model is used to calculate the CLS scores of the recall result and the original data, and the correlation between the recall result and the original data is obtained.

7. A method for reducing the illusion of large models in vertical domains according to claim 6, characterized in that: The sorting strategy is to sort the recall results from large to small according to the CLS scores, and retain the top 10 data with higher scores.

8. A method for reducing the illusion of large models in vertical domains according to claim 1, characterized in that: The method also includes the steps of: training a word embedding model capable of understanding vertical domain knowledge, wherein the word embedding model adopts a BERT model.

9. A method for reducing the illusion of large models in vertical domains according to claim 8, characterized in that: Obtain industry data in a vertical field, first split it according to paragraphs, chapters, and knowledge points, and form question-answer data for paragraphs, chapters, and knowledge points, then perform data cleaning, and construct a training task data set, which is used to train the word embedding model.

10. A method for reducing the illusion of large models in vertical domains according to claim 9, characterized in that: The MASK strategy of the BERT model is adopted when training the word embedding model.

Images