Method and device for generating questions and answers based on retrieval enhancement
Through vectorized processing and multimodal analysis, combined with CLIP and SimCSE models, the inaccurate answers in the generated question-and-answer system are solved, ensuring the accuracy and consistency of the reply information, and improving the accuracy of the question-and-answer system.
Patent Information
- Application Number
- CN202411088063.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2025-01-03
AI Technical Summary
In the prior art, the generation of question and answer systems may lead to inaccurate answers or factual errors, especially inconsistent with user input, inconsistent with common sense content, or contradictory with historically generated content.
Using a search-enhanced generation method, vectorized processing is performed by receiving query information and inputting it into the vectorized model that has been trained, including multimodal pre-trained neural network CLIP and preset comparison learning model SimCSE; Perform analysis, combine text and image information; determine the similarity between vectorized query information and analytical information, generate consistent reply information, and avoid the hallucination problem of large language models.
It realizes the generation of reply information that is consistent with user input, consistent with common sense content, and not contradicts the historical generated content, and improves the accuracy of question and answer.
Smart Images
Figure CN119248912A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for generating questions and answers based on retrieval enhancement. Background Art
[0002] With the development and advancement of technology, document retrieval is an important part of the resource library. It can be used to help users find required data more quickly, and can obtain required data through conversational retrieval, thereby improving user experience and efficiency.
[0003] In the prior art, a common approach is to clean up user features, eliminate noise and redundant information, perform necessary feature conversion according to business needs, ensure the quality and applicability of user features, use large language models such as ChatGPT to build a prediction model, and evaluate the performance of the prediction model through preset indicators.
[0004] However, the above method may result in the generation of content that is inconsistent with user input, inconsistent with common sense content, or contradictory to historically generated content, resulting in inaccurate answers or factually incorrect questions. Summary of the invention
[0005] In view of this, the embodiments of the present disclosure provide a method and device for generating questions and answers based on retrieval enhancement to solve the problem of inaccurate answers or factual errors in the prior art.
[0006] A first aspect of an embodiment of the present disclosure provides a method for generating questions and answers based on retrieval enhancement, comprising:
[0007] Receive query information, input the query information into the trained vectorized model, and obtain vectorized query information corresponding to the query information, wherein the vectorized model includes a multimodal pre-trained neural network CLIP and a preset comparative learning model SimCSE;
[0008] Parsing the document information corresponding to the filtered document to obtain parsed information, the document information including text information and image information, the parsed information including a text block corresponding to the text information and an image feature corresponding to the image information;
[0009] Inputting the parsing information into the vectorization model to obtain vectorized parsing information corresponding to the parsing information, wherein the vectorized parsing information includes vectorized text blocks corresponding to the text information and vectorized image features corresponding to the image information;
[0010] Determine the information similarity between the vectorized query information and each vectorized parsing information, and use the vectorized parsing information with information similarity greater than a preset threshold as the target parsing information;
[0011] The target parsed information is input into a preset large language model to obtain response information generated based on the target parsed information.
[0012] A second aspect of the embodiments of the present disclosure provides a device for generating questions and answers based on retrieval enhancement, including:
[0013] A first processing module is configured to receive query information, input the query information into a trained vectorization model, and obtain vectorized query information corresponding to the query information, wherein the vectorization model includes a multimodal pre-trained neural network CLIP and a preset contrast learning model SimCSE;
[0014] A parsing module is configured to parse the document information corresponding to the filtered document to obtain parsed information, wherein the document information includes text information and image information, and the parsed information includes a text block corresponding to the text information and an image feature corresponding to the image information;
[0015] A second processing module is configured to input the parsing information into a vectorization model to obtain vectorized parsing information corresponding to the parsing information, wherein the vectorized parsing information includes a vectorized text block corresponding to the text information and a vectorized image feature corresponding to the image information;
[0016] A third processing module is configured to determine the information similarity between the vectorized query information and each vectorized parsing information, and use the vectorized parsing information with information similarity greater than a preset threshold as the target parsing information;
[0017] The reply module is configured to input the target parsed information into a preset large language model to obtain reply information generated based on the target parsed information.
[0018] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.
[0019] According to a fourth aspect of an embodiment of the present disclosure, a readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.
[0020] Compared with the prior art, the beneficial effects of the disclosed embodiments are as follows: query information is refined through a trained vectorization model, and the query information is vectorized to obtain vectorized query information, and document information corresponding to the filtered documents is vectorized to obtain vectorized document information, the information similarity between the vectorized query information and each vectorized parsed information is determined to obtain target parsed information, and the target parsed information is input into a preset large language model, so as to polish and summarize the reply information to obtain reply information generated according to the target parsed information and corresponding to the query information, thereby avoiding the hallucination problem of the large language model, achieving reply information that is consistent with user input, consistent with common sense content, and not inconsistent with historical generated content, and improving the accuracy of question and answer. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0022] Figure 1 is a schematic diagram of an application scenario of an embodiment of the present disclosure;
[0023] Figure 2 is a flowchart of a method for generating questions and answers based on retrieval enhancement provided by an embodiment of the present disclosure;
[0024] Figure 3 is a schematic diagram of a preset search system architecture for searching for filtered documents corresponding to query information provided by an embodiment of the present disclosure;
[0025] Figure 4 is a schematic diagram of a device for generating questions and answers based on retrieval enhancement provided by an embodiment of the present disclosure;
[0026] Figure 5 is a schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0027] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.
[0028] A method and device for generating questions and answers based on retrieval enhancement according to an embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings.
[0029] Figure 1 1 is a schematic diagram of an application scenario of an embodiment of the present disclosure. The application scenario may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a server 104, and a network 105.
[0030] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be hardware or software. When the first terminal device 101, the second terminal device 102, and the third terminal device 103 are hardware, they can be various electronic devices with display screens and supporting communication with the server 104, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers; when the first terminal device 101, the second terminal device 102, and the third terminal device 103 are software, they can be installed in the electronic devices described above. The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be implemented as multiple software or software modules, or as a single software or software module, which is not limited in the embodiments of the present disclosure. Further, various applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.
[0031] The server 104 may be a server that provides various services, for example, a background server that receives a request sent by a terminal device that establishes a communication connection with the server, and the background server may receive and analyze the request sent by the terminal device, and generate a processing result. The server 104 may be a single server, a server cluster consisting of several servers, or a cloud computing service center, which is not limited in the embodiments of the present disclosure.
[0032] It should be noted that the server 104 can be hardware or software. When the server 104 is hardware, it can be various electronic devices that provide various services for the first terminal device 101, the second terminal device 102, and the third terminal device 103. When the server 104 is software, it can be multiple software or software modules that provide various services for the first terminal device 101, the second terminal device 102, and the third terminal device 103, or it can be a single software or software module that provides various services for the first terminal device 101, the second terminal device 102, and the third terminal device 103, and the embodiments of the present disclosure are not limited to this.
[0033] The network 105 may be a wired network connected by coaxial cables, twisted pairs, and optical fibers, or a wireless network that can interconnect various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), infrared, etc., which is not limited in the embodiments of the present disclosure.
[0034] The user can establish a communication connection with the server 104 via the network 105 through the first terminal device 101, the second terminal device 102 and the third terminal device 103 to receive or send information, etc. Specifically, after the user imports the collected data of the points of interest into the server 104, the server 104 obtains the first data of the points of interest to be processed, which includes the first longitude and latitude and the first classification of the points of interest to be processed, and performs a conflict check on the points of interest to be processed according to the first longitude and latitude and the first classification; further, in the case of determining a conflict, the server 104 performs conflict processing on the points of interest to be processed to avoid a large amount of duplicate data and unavailable data in the database.
[0035] It should be noted that the specific types, quantities and combinations of the first terminal device 101, the second terminal device 102 and the third terminal device 103, the server 104 and the network 105 can be adjusted according to the actual needs of the application scenario, and the embodiments of the present disclosure do not limit this.
[0036] Figure 2 FIG. 1 is a flow chart of a method for generating questions and answers based on retrieval enhancement provided by an embodiment of the present disclosure. Figure 2 As shown, the method for generating question and answer based on retrieval enhancement includes:
[0037] S201, receiving query information, inputting the query information into a trained vectorized model, and obtaining vectorized query information corresponding to the query information, wherein the vectorized model includes a multimodal pre-trained neural network CLIP and a preset contrastive learning model SimCSE;
[0038] S202, parsing the document information corresponding to the filtered document to obtain parsed information;
[0039] S203, inputting the parsing information into a vectorization model to obtain vectorized parsing information corresponding to the parsing information, where the vectorized parsing information includes vectorized text blocks corresponding to the text information and vectorized image features corresponding to the image information;
[0040] S204, determining the information similarity between the vectorized query information and each vectorized parsing information, and taking the vectorized parsing information with information similarity greater than a preset threshold as the target parsing information;
[0041] S205, inputting the target parsed information into a preset large language model to obtain reply information generated based on the target parsed information.
[0042] The document information includes text information and image information, and the parsed information includes text blocks corresponding to the text information and image features corresponding to the image information.
[0043] Figure 2 The retrieval-enhanced question-answering method can be composed of Figure 1 The first terminal device, the second terminal device or the third terminal device is executed, and the preset vectorization model and the preset large language model can be stored in Figure 1 server.
[0044] In an exemplary embodiment of the present application, query information is received, wherein the query information is sent by a user through a user terminal, and the query information can be in various forms, including text or images.
[0045] The received query information is input into the trained vectorization model, where the vectorization model includes a multimodal neural network CLIP (Contrastive Language-Image Pre-training) and a preset contrastive learning model SimCSE (Simple Contrastive Learning of Sentence Embeddings). The CLIP model is used to process the association between text and images, and can map text and images into the same vector space, so that similar text and images have similar representations in the vector space. The SimCSE model focuses on text vectorization and improves the quality of sentence embedding through contrastive learning.
[0046] The document information corresponding to the filtered documents is parsed, the filtered documents include text information and image information, the parsed information includes text blocks corresponding to the text information and image features corresponding to the image information, wherein each text block includes content of a specific topic or context, and each image feature corresponding to the image information can represent the main content and characteristics of the image information.
[0047] The parsing information, i.e., text blocks and image features, is input into the vectorization model. Through the powerful learning ability of the vectorization model, the parsing information is converted into vectorized parsing information, so that the text blocks and image features can be compared and matched in the corresponding format in the vector space.
[0048] Determining the information similarity between the vectorized query information and each vectorized parsing information can be achieved by calculating the distance or angle between the two vectors. The closer the distance or the smaller the angle, the higher the similarity of the information represented by the two vectors. When the information similarity between a certain vectorized parsing information and the vectorized query information is higher than a preset threshold, the vectorized parsing information is used as the target parsing information.
[0049] The target parsed information is input into a preset large language model, so that the preset large language model generates reply information according to the content in the target parsed information, wherein the preset large language model can generate corresponding reply information according to the input information, and the reply information can meet the needs corresponding to the query information as much as possible, and provide accurate, useful and easy-to-understand information.
[0050] Furthermore, before inputting the target parsing information into the preset large language model, the text block or image feature corresponding to the target parsing information can be used to obtain the corresponding prompt through a preset prompt template, wherein the prompt is a fixed text structure, and the target parsing information is input into the preset large language model in a specific form, so that the target parsing information received by the preset large language model is standardized and structured, and the consistency between different target parsing information can be ensured, thereby better guiding the preset large language model to generate coherent, accurate or creative response information.
[0051] According to the technical solution provided by the embodiments of the present disclosure, the reply information can be polished and summarized by using a preset large language model to obtain reply information generated according to the target parsed information and corresponding to the query information, thereby avoiding the hallucination problem of the large language model, achieving reply information that is consistent with the user input, consistent with common sense content, and not inconsistent with historically generated content, thereby improving the accuracy of question and answering.
[0052] In some embodiments, parsing the document information corresponding to the filtered document to obtain parsed information includes:
[0053] Segmenting the corresponding text information in the document information corresponding to the filtered document according to a preset parsing method to obtain text blocks corresponding to the text information. The preset parsing method includes segmentation according to paragraphs or segmentation according to string length;
[0054] The image information corresponding to the document information is input into a preset image description model to obtain image features corresponding to the image information.
[0055] The text information of the filtered documents is segmented according to a preset parsing method so as to convert continuous text content into text blocks that are easy to process. The preset parsing method can be flexibly set according to actual needs, and may include segmentation by paragraph or by string length.
[0056] Among them, segmentation by paragraphs is suitable for text content with clear structure and obvious separators between paragraphs, such as line breaks, blank lines, etc. During the parsing process, the text content is divided into several paragraphs by identifying the separators. Each paragraph is regarded as a text block, which can maintain the structure and coherence of the text content and is conducive to the analysis and processing of the text content.
[0057] Splitting by string length can be used when the text content does not have obvious paragraph separators, or when it needs to be split according to a specific length. Specifically, the text content is split into several text blocks according to the preset string length (for example, the string length of each text block is 500 characters). This is suitable for scenarios where the size of the text block needs to be controlled, but there is a possibility of destroying the structure and coherence of the text content.
[0058] In addition to text information, the filtered documents may also include image information. The image description model can be used to parse the image information. The image description model is a machine learning model that can convert image content into text descriptions. It extracts and encodes features of the image to generate feature vectors related to the image content. The feature vector contains key information about the image, such as objects, scenes, colors, etc. The image description model generates text descriptions related to the image content based on the above feature vectors, that is, image features corresponding to the image information.
[0059] According to the technical solution provided in the embodiment of the present disclosure, the text information and image information of the filtered documents can be parsed to obtain parsed information including text blocks segmented according to a preset parsing method and image features corresponding to the image information, which can provide a basis for subsequent analysis and processing of the document content.
[0060] Furthermore, the filtered documents are obtained by inputting the query information into a preset search system for searching. The following is a relevant description of the preset search system:
[0061] Figure 3 Schematic diagram of a preset search system architecture for searching for filtered documents corresponding to query information provided by an embodiment of the present disclosure. Figure 3 As shown, the search system includes a data stream unit, a search unit, an online storage unit and a platform-dependent unit.
[0062] The data flow unit includes event monitoring, event receiving, data receiving and index writing, wherein event monitoring is used to capture the "query" action, and the data corresponding to the "query" action is the query information. Event receiving is used to transmit the "query" action to the online storage unit to trigger the query, thereby filtering the unfiltered documents to obtain the filtered documents. Data receiving is used to receive specific query information, and index writing is used to determine the index corresponding to the query information to determine the filtered documents corresponding to the query information.
[0063] The search unit includes a search central control service, which operates four key sub-modules or engines. Among them, the intent engine is responsible for analyzing and identifying the query or input intent corresponding to the received query information. The recall engine recalls documents or information related to the query information by label recall and vector recall according to the intent. The sorting engine sorts the above documents or information according to preset algorithms or rules to ensure that the most important information is displayed first. The re-arrangement engine is used to improve user experience and content diversity and improve traffic efficiency. The results obtained by the sorting engine are adjusted according to certain rules and strategies (i.e., the preset reference values in the embodiment of the present application), and the obtained content is presented to the target device as the sorting result.
[0064] The online storage unit includes text search Elasticsearch, vector search Milvus, and binary tree structure search Redis, which are used to search in unfiltered documents according to vectorized query information to determine filtered documents.
[0065] The platform-dependent unit indicates that the current preset search system depends on multiple "platforms", including feature platform, reasoning platform, data platform and machine learning platform. The above platforms provide necessary functions and support for the preset search system, such as data support, feature support, model support, etc., and together with the preset search system, they constitute a complete data processing and analysis ecosystem. The preset search system can provide accurate and fast search results through effective data management and search algorithms.
[0066] In some embodiments, before parsing the document information corresponding to the filtered document, the method further includes:
[0067] Input the query information into a preset search system so that the preset search system understands and parses the query information to obtain a structured search tree. The understanding and parsing methods include preprocessing, word segmentation, rewriting, importance analysis, intent recognition, and sensitivity recognition.
[0068] Based on the search tree, perform label recall and vector recall on all unfiltered documents to obtain label recall results and vector recall results;
[0069] The label recall results and vector recall results are integrated to obtain the optimized recall results;
[0070] Determine the optimized recall index value corresponding to each optimized recall result according to the preset recall index, and take the optimized recall result corresponding to the optimized recall index value not less than the preset recall index value as the target recall result;
[0071] The unfiltered documents corresponding to the target recall results are treated as filtered documents.
[0072] The query information is input into the preset search system, which deeply understands and analyzes the input query information to obtain a structured search tree, which can accurately reflect the query intent and search requirements of the query information. The understanding and analysis methods include:
[0073] Preprocessing: Perform preliminary cleaning and formatting of query information, remove irrelevant characters, redundant spaces, etc., to ensure the purity of query information;
[0074] Word segmentation: divide the query information into several meaningful vocabulary units;
[0075] Rewrite: adjust the grammatical or semantic information of the segmented query information to be closer to the user's real intention;
[0076] Importance analysis: evaluate the weight of each lexical unit after segmentation to determine the importance of each lexical unit in the query information;
[0077] Intent recognition: Analyze the user's query intent based on the query information and determine the specific content or direction the user wants to search for;
[0078] Sensitive identification: Detect sensitive words in query information to ensure that the query information complies with relevant regulations and standards.
[0079] Based on the search tree, label recall and vector recall are performed on all unfiltered documents. Label recall is to search based on the label information of each unfiltered document, so as to quickly locate the documents related to the query information as the label recall result. Vector recall is to calculate the vector similarity between each unfiltered document and the query information, and determine the unfiltered documents related to the query information as the vector recall result.
[0080] The label recall results and the vector recall results are fused to obtain the optimized recall results. The fusion process is to conduct a comprehensive evaluation based on certain strategies and weights to ensure that the optimized recall results are comprehensive and accurate.
[0081] According to the preset recall indicators, such as recall rate, accuracy, etc., each optimized recall result is evaluated to determine the optimized recall indicator value corresponding to each optimized recall result, and the recall result whose optimized recall indicator value is not less than the preset recall indicator value is taken as the target recall result, and the unfiltered documents corresponding to the target recall result are taken as the filtered documents.
[0082] According to the technical solution provided by the embodiment of the present disclosure, the accuracy and efficiency of the filtered documents obtained by searching according to the query information can be ensured, thereby ensuring the accuracy of the reply information, providing users with high-quality reply information, and enhancing the user experience.
[0083] In some embodiments, after determining the optimized recall index value corresponding to each optimized recall result according to the preset recall index, and taking the optimized recall result corresponding to the optimized recall index value not less than the preset recall index value as the target recall result, the method further includes:
[0084] Input the target recall results into a preset deep neural network model to obtain the result feature values corresponding to each target recall result;
[0085] Sort the target recall results based on the result characteristic value and the preset reference value to obtain the sorting result;
[0086] Sort each unfiltered document by the sort result.
[0087] The target recall results are further processed, that is, the target recall results are input into a preset deep neural network model, which can extract key features from the target recall results and obtain result feature values corresponding to each target recall result.
[0088] Among them, the preset neural network model can be a bidirectional encoder representation model (Bidirectional Encoder Representations from Transformers, BERT) or a deep relevance matching model (Deep Relevance Matching Model, DRMM). The BERT model is used for natural language processing tasks, can learn rich language knowledge and contextual information, and perform fine-tuning on various natural language processing tasks to achieve better performance. The DRMM model performs deep matching between query information and unfiltered documents to identify the correlation between the query information and the unfiltered documents, which can improve the accuracy and efficiency of information retrieval.
[0089] The result feature value is used to describe the quantitative indicators of the target recall result characteristics, which can accurately and quickly understand the quality of the recall result. At the same time, the preset reference value is introduced to perform more detailed screening and sorting of the recall results.
[0090] Among them, the preset reference value can be a standard value set according to business needs and data characteristics, such as content that improves user experience and ensures content diversity in unfiltered documents. The target recall results are sorted according to the result characteristic value and the preset reference value to obtain a sorting result. The sorting result can reflect the relative advantages and disadvantages of each target recall result.
[0091] Sorting the unfiltered documents according to the sorting result can obtain a more effective and accurate order of the unfiltered documents.
[0092] According to the technical solution provided by the embodiments of the present disclosure, the result feature value of each target recall result is determined according to a preset deep neural network model, and the target recall results are sorted according to preset reference values, so that unscreened documents that are more matched with the query information and take into account the diversity of content can be obtained, thereby improving the recall efficiency. At the same time, the generated reply information can enhance the user experience.
[0093] In some embodiments, determining the information similarity between the vectorized query information and each vectorized parsed information includes:
[0094] Perform secondary vectorization on the current vectorized parsing information to obtain secondary vectorized parsing information corresponding to each vectorized parsing information;
[0095] Based on the vectorized query information and the secondary vectorized parsing information, the information similarity between the vectorized query information and each vectorized parsing information is determined.
[0096] The current vectorized parsing information is re-vectorized. The method of re-vectorization can be to input the current vectorized parsing information into the preset vectorization model again for vectorization, or it can be selected according to the specific application scenario and data type. For example, if the current vectorized parsing information is based on word embedding, then the re-vectorization may involve more complex sentence or paragraph embedding technology.
[0097] The current vectorized parsing information is input into the corresponding model to obtain secondary vectorized parsing information, and the secondary vectorized parsing information can more accurately represent the meaning of the original parsing information.
[0098] According to actual needs and data types, determine the appropriate similarity measurement method, and determine the information similarity between the vectorized query information and each vectorized parsing information based on the vectorized query information and the secondary vectorized parsing information.
[0099] The similarity measurement method may be cosine similarity, Euclidean distance, Jaccard similarity, etc.
[0100] Furthermore, the method can be optimized and adjusted according to the information similarity and actual needs. For example, different vectorization methods can be tried, parameters can be adjusted, or more complex deep learning models can be used to improve the accuracy of similarity calculations to improve the quality and accuracy of parsing results.
[0101] According to the technical solution provided by the embodiment of the present disclosure, the information similarity between the vectorized query information and each vectorized parsing information can be determined based on the vectorized query information and the secondary vectorized parsing information, thereby improving the quality and accuracy of the parsing result.
[0102] In some embodiments, based on the vectorized query information and the secondary vectorized parsing information, determining the information similarity between the vectorized query information and each vectorized parsing information includes:
[0103] Determine the information similarity between the vectorized query information and each vectorized parsed information based on the information similarity formula;
[0104] The mathematical expression corresponding to the information similarity formula is:
[0105]
[0106] Among them, l i represents the information similarity between the vectorized query information and the i-th vectorized parsed information, where i ranges from 1 to N, N is the number of vectorized query information, and h i Represents the i-th vectorized parsing information, represents the secondary vectorized parsing information corresponding to the i-th vectorized parsing information, is a similarity measurement function used to measure the similarity between the current vectorized parsing information and the secondary vectorized parsing information, τ represents the temperature coefficient, Represents the j-th secondary vectorization parsing information, where j ranges from 1 to N.
[0107] In order to clarify the calculation of information similarity, the vectorized query information and vectorized parsing information are explained:
[0108] Vectorized query information is to input query information into a preset vectorization model and convert it into vector form, so as to facilitate efficient matching and searching in large amounts of data;
[0109] Vectorized parsing information is to convert the filtered documents determined according to the query information into vector form so as to efficiently match and search with the vectorized query information.
[0110] Among them, the information similarity formula is derived from the basic idea of SimCSE, that is, contrastive learning. The vectorized parsing information and the secondary vectorized parsing information can be regarded as a group of sample pairs that are positive examples of each other, that is, similar information. The "dissimilar information" uses the same batch of parsing information, that is, other data that are all vectorized parsing information or all secondary vectorized parsing information. In this way, the samples can be enhanced to obtain more accurate results.
[0111] The temperature coefficient is an adjustable parameter used to control the sensitivity of the similarity calculation. A higher temperature coefficient may lead to a smoother similarity distribution, while a lower temperature coefficient may make the similarity difference more significant. The temperature coefficient plays a role in "softening" or "hardening" the similarity distribution in the formula. By adjusting the temperature coefficient, the sensitivity of each unfiltered document to the similarity difference can be controlled, which is conducive to controlling the balance between diversity and accuracy in the filtered documents.
[0112] According to the technical solution provided in the embodiments of the present disclosure, information similarity can be determined based on vectorized query information and secondary vectorized parsing information. The information similarity formula can make positive examples closer in space and negative examples slightly farther in space, thereby achieving data enhancement, making it easier, more accurate and faster to find filtered documents that match the query information.
[0113] In some embodiments, after inputting the target parsed information into a preset large language model to obtain the reply information generated based on the target parsed information, the method further includes:
[0114] Send and display the reply information to the target screen;
[0115] Store the current query information and reply information in the form of documents;
[0116] Receive secondary query information input for the reply information, use the secondary query information as query information, execute the steps of receiving the query information, inputting the query information into the trained vectorized model, and obtaining vectorized query information corresponding to the query information.
[0117] The generated reply information is sent and displayed to a target screen, where the target screen may be a computer monitor, a mobile device screen, or other types of display interfaces, to ensure that the reply information can be presented to the user clearly and accurately.
[0118] At the same time, the current query information and the corresponding reply information are stored in document form, thereby establishing a searchable historical record, which is convenient for subsequent tracking and analysis of the query information and reply information. Storing in document form also facilitates data backup and recovery when necessary, ensuring the security and reliability of information.
[0119] After the display and storage of the reply information are completed, the input query information is continuously received. When the secondary query information for the reply information is received, it is treated as the query information and processed, that is, the above processing steps are executed again.
[0120] According to the technical solution provided in the embodiment of the present disclosure, it is possible to ensure timely and accurate generation of reply information corresponding to query information, and generate more accurate and personalized reply information corresponding to secondary query information based on previous query information, thereby forming a closed loop, so that each generated reply information is more accurate than the last generated reply information, thereby improving the user experience.
[0121] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, which will not be described one by one here.
[0122] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.
[0123] Figure 4 Schematic diagram of a device for generating questions and answers based on retrieval enhancement provided by an embodiment of the present disclosure. Figure 4 As shown, the device for generating questions and answers based on retrieval enhancement includes: a first processing module 401, a parsing module 402, a second processing module 403, a third processing module 404, and a reply module 405, wherein:
[0124] The first processing module 401 is configured to receive query information, input the query information into a trained vectorization model, and obtain vectorized query information corresponding to the query information, wherein the vectorization model includes a multimodal pre-trained neural network CLIP and a preset contrastive learning model SimCSE;
[0125] The parsing module 402 is configured to parse the document information corresponding to the filtered document to obtain parsed information, wherein the document information includes text information and image information, and the parsed information includes a text block corresponding to the text information and an image feature corresponding to the image information;
[0126] The second processing module 403 is configured to input the parsing information into the vectorization model to obtain vectorized parsing information corresponding to the parsing information, wherein the vectorized parsing information includes vectorized text blocks corresponding to the text information and vectorized image features corresponding to the image information;
[0127] The third processing module 404 is configured to determine the information similarity between the vectorized query information and each vectorized parsing information, and use the vectorized parsing information with information similarity greater than a preset threshold as the target parsing information;
[0128] The reply module 405 is configured to input the target parsed information into a preset large language model to obtain reply information generated based on the target parsed information.
[0129] In some embodiments, parsing module 402 is configured to:
[0130] Segmenting the corresponding text information in the document information corresponding to the filtered document according to a preset parsing method to obtain text blocks corresponding to the text information. The preset parsing method includes segmentation according to paragraphs or segmentation according to string length;
[0131] The image information corresponding to the document information is input into a preset image description model to obtain image features corresponding to the image information.
[0132] In some embodiments, before parsing the document information corresponding to the filtered document, the parsing module 402 is further configured to:
[0133] Input the query information into a preset search system so that the preset search system understands and parses the query information to obtain a structured search tree. The understanding and parsing methods include preprocessing, word segmentation, rewriting, importance analysis, intent recognition, and sensitivity recognition.
[0134] Based on the search tree, perform label recall and vector recall on all unfiltered documents to obtain label recall results and vector recall results;
[0135] The label recall results and vector recall results are integrated to obtain the optimized recall results;
[0136] Determine the optimized recall index value corresponding to each optimized recall result according to the preset recall index, and take the optimized recall result corresponding to the optimized recall index value not less than the preset recall index value as the target recall result;
[0137] The unfiltered documents corresponding to the target recall results are treated as filtered documents.
[0138] In some embodiments, the parsing module 402 determines the optimized recall index value corresponding to each optimized recall result according to the preset recall index, and after taking the optimized recall result corresponding to the optimized recall index value not less than the preset recall index value as the target recall result, is further configured to:
[0139] Input the target recall results into a preset deep neural network model to obtain the result feature values corresponding to each target recall result;
[0140] Sort the target recall results based on the result characteristic value and the preset reference value to obtain the sorting result;
[0141] Sort each unfiltered document by the sort result.
[0142] In some embodiments, the third processing module 404 is configured to:
[0143] Perform secondary vectorization on the current vectorized parsing information to obtain secondary vectorized parsing information corresponding to each vectorized parsing information;
[0144] Based on the vectorized query information and the secondary vectorized parsing information, the information similarity between the vectorized query information and each vectorized parsing information is determined.
[0145] In some embodiments, the third processing module 404 is configured to:
[0146] Determine the information similarity between the vectorized query information and each vectorized parsed information based on the information similarity formula;
[0147] The mathematical expression corresponding to the information similarity formula is:
[0148]
[0149] Among them, l i represents the information similarity between the vectorized query information and the i-th vectorized parsed information, where i ranges from 1 to N, N is the number of vectorized query information, and h i Represents the i-th vectorized parsing information, represents the secondary vectorized parsing information corresponding to the i-th vectorized parsing information, is a similarity measurement function used to measure the similarity between the current vectorized parsing information and the secondary vectorized parsing information, τ represents the temperature coefficient, Represents the j-th secondary vectorization parsing information, where j ranges from 1 to N.
[0150] In some embodiments, after the reply module 405 inputs the target parsed information into the preset large language model and obtains the reply information generated based on the target parsed information, it is further configured to:
[0151] Send and display the reply information to the target screen;
[0152] Store the current query information and reply information in the form of documents;
[0153] Receive secondary query information input for the reply information, use the secondary query information as query information, execute the steps of receiving the query information, inputting the query information into the trained vectorized model, and obtaining vectorized query information corresponding to the query information.
[0154] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.
[0155] Figure 5 Schematic diagram of an electronic device 5 provided in an embodiment of the present disclosure. Figure 5 As shown, the electronic device 5 of this embodiment includes: a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the computer program 503, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 501 executes the computer program 503, the functions of the modules / units in the above-mentioned device embodiments are implemented.
[0156] The electronic device 5 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 5 may include, but is not limited to, a processor 501 and a memory 502. Those skilled in the art will appreciate that Figure 5 The electronic device 5 is merely an example and does not limit the electronic device 5 , and may include more or less components than those shown in the figure, or different components.
[0157] The processor 501 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0158] The memory 502 may be an internal storage unit of the electronic device 5, for example, a hard disk or memory of the electronic device 5. The memory 502 may also be an external storage device of the electronic device 5, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 5. The memory 502 may also include both an internal storage unit of the electronic device 5 and an external storage device. The memory 502 is used to store computer programs and other programs and data required by the electronic device.
[0159] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.
[0160] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.
[0161] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be included in the protection scope of the present disclosure.
Claims
1. A method for generating question and answer based on retrieval enhancement, characterized in that: include: Receive query information, input the query information into a trained vectorization model, and obtain vectorized query information corresponding to the query information, wherein the vectorization model includes a multimodal pre-trained neural network CLIP and a preset contrast learning model SimCSE; Parsing document information corresponding to the filtered document to obtain parsed information, wherein the document information includes text information and image information, and the parsed information includes a text block corresponding to the text information and an image feature corresponding to the image information; Inputting the parsing information into the vectorization model to obtain vectorized parsing information corresponding to the parsing information, wherein the vectorized parsing information includes a vectorized text block corresponding to the text information and a vectorized image feature corresponding to the image information; Determine the information similarity between the vectorized query information and each of the vectorized parsing information, and use the vectorized parsing information whose information similarity is greater than a preset threshold as target parsing information; The target parsed information is input into a preset large language model to obtain response information generated based on the target parsed information.
2. The method according to claim 1, characterized in that Parse the document information corresponding to the filtered documents to obtain parsed information, including: Segmenting the corresponding text information in the document information corresponding to the filtered document according to a preset parsing method to obtain a text block corresponding to the text information, wherein the preset parsing method includes segmentation according to paragraphs or segmentation according to character string length; The image information corresponding to the document information is input into a preset image description model to obtain image features corresponding to the image information.
3. The method according to claim 1, characterized in that Before parsing the document information corresponding to the filtered documents, it also includes: Inputting the query information into a preset search system so that the preset search system understands and analyzes the query information to obtain a structured search tree, wherein the understanding and analysis method includes preprocessing, word segmentation, rewriting, importance analysis, intent recognition and sensitivity recognition; Based on the search tree, perform label recall and vector recall on all unfiltered documents to obtain label recall results and vector recall results; Fusion of the label recall result and the vector recall result to obtain an optimized recall result; Determine the optimized recall index value corresponding to each of the optimized recall results according to the preset recall index, and take the optimized recall result corresponding to the optimized recall index value not less than the preset recall index value as the target recall result; The unfiltered documents corresponding to the target recall results are used as filtered documents.
4. The method according to claim 3, characterized in that Determining the optimized recall index value corresponding to each of the optimized recall results according to the preset recall index, and taking the optimized recall result corresponding to the optimized recall index value not less than the preset recall index value as the target recall result, further comprising: Inputting the target recall results into a preset deep neural network model to obtain result feature values corresponding to each of the target recall results; Sorting the target recall results based on the result characteristic value and the preset reference value to obtain a sorting result; The unfiltered documents are sorted according to the sorting result.
5. The method according to claim 1, characterized in that Determining the information similarity between the vectorized query information and each of the vectorized parsed information includes: Performing secondary vectorization on the current vectorized parsing information to obtain secondary vectorized parsing information corresponding to each of the vectorized parsing information; Based on the vectorized query information and the secondary vectorized parsing information, information similarity between the vectorized query information and each of the vectorized parsing information is determined.
6. The method according to claim 5, characterized in that Determining information similarity between the vectorized query information and each of the vectorized parsing information based on the vectorized query information and the secondary vectorized parsing information includes: Determine the information similarity between the vectorized query information and each of the vectorized parsed information based on an information similarity formula; The mathematical expression corresponding to the information similarity formula is: Among them, l i represents the information similarity between the vectorized query information and the i-th vectorized parsing information, where i ranges from 1 to N, N is the number of vectorized query information, and h i represents the i-th vectorized parsing information, h i + represents the secondary vectorized parsing information corresponding to the i-th vectorized parsing information, is a similarity measurement function used to measure the similarity between the current vectorized parsing information and the secondary vectorized parsing information, τ represents the temperature coefficient, Represents the j-th secondary vectorization parsing information, and the value range of j is 1 to N.
7. The method according to any one of claims 1 to 6, characterized in that After inputting the target parsed information into a preset large language model to obtain reply information generated based on the target parsed information, the method further includes: Sending and displaying the reply information to a target screen; Storing the current query information and the response information in a document form; The steps of receiving secondary query information input for the reply information, taking the secondary query information as the query information, executing the step of receiving the query information, inputting the query information into the trained vectorized model, and obtaining vectorized query information corresponding to the query information.
8. A device for generating questions and answers based on retrieval enhancement, characterized in that: include: A first processing module is configured to receive query information, input the query information into a trained vectorization model, and obtain vectorized query information corresponding to the query information, wherein the vectorization model includes a multimodal pre-trained neural network CLIP and a preset contrastive learning model SimCSE; A parsing module is configured to parse the document information corresponding to the filtered document to obtain parsed information, wherein the document information includes text information and image information, and the parsed information includes a text block corresponding to the text information and an image feature corresponding to the image information; A second processing module is configured to input the parsing information into the vectorization model to obtain vectorized parsing information corresponding to the parsing information, wherein the vectorized parsing information includes a vectorized text block corresponding to the text information and a vectorized image feature corresponding to the image information; A third processing module is configured to determine the information similarity between the vectorized query information and each of the vectorized parsing information, and use the vectorized parsing information with the information similarity greater than a preset threshold as the target parsing information; The reply module is configured to input the target parsed information into a preset large language model to obtain reply information generated based on the target parsed information.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.