Question retrieval question answering method based on large language model and multi-dimensional technology fusion

Abstract

The invention relates to a question retrieval question and answer method based on a large language model and multi-dimensional technology fusion, and the method comprises the steps: carrying out the question splitting of a to-be-processed question, and obtaining at least one unit question; key elements of at least one unit question are extracted, semantic vectors of at least one unit question are determined, and retrieval elements are determined according to the key elements and the semantic vectors; performing question retrieval question answering on the retrieval elements of each unit question to obtain a retrieval question answering result; and determining a retrieval question-answering result of the to-be-processed question based on the retrieval question-answering result corresponding to each unit question. By adopting the method, the retrieval efficiency and accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a question retrieval and answering method based on a large language model and multi-dimensional technology fusion. Background Art

[0002] The Large Language Model (LLM) is an artificial intelligence model that is trained using large-scale data and powerful computing power.

[0003] The traditional financial industry's retrieval technology is that users input a series of keywords and send a search request to the browser. The browser performs simple rule matching on the entire knowledge base content based on the requested keywords, sorts the entire text data, and returns the corresponding search question and answer results.

[0004] However, such a retrieval system cannot perform personalized searches for specific financial scenarios, and simple string matching cannot provide deeper understanding at the semantic level. Summary of the invention

[0005] Based on this, it is necessary to provide a question retrieval and answering method based on a large language model and multi-dimensional technology integration that can improve retrieval efficiency and accuracy in response to the above technical problems. Specifically, it can be a retrieval efficiency enhancement system for intelligent financial research reports and regulations based on a large language model and multi-dimensional technology integration.

[0006] The present application provides a question retrieval and answering method based on a large language model and multi-dimensional technology fusion, the method comprising:

[0007] Split the problem to be processed into at least one unit problem;

[0008] Extracting at least one key element of the unit question and determining at least one semantic vector of the unit question, and determining a search element according to the key element and the semantic vector;

[0009] Performing question retrieval and answering on the retrieval elements of each of the unit questions to obtain retrieval and answering results;

[0010] The retrieval question and answer result of the question to be processed is determined based on the retrieval question and answer result corresponding to each of the unit questions.

[0011] In some embodiments, the method further comprises providing a pre-stored history database, wherein the history database comprises historical split question data and historical semantic vectors and historical key elements corresponding to each question data;

[0012] The problem to be processed is split into at least one unit problem, including:

[0013] Determine a current semantic vector of the problem to be processed, and match a historical semantic vector corresponding to the current semantic vector in a historical database;

[0014] In response to matching a historical semantic vector corresponding to the current semantic vector in the historical database, splitting the current problem to be processed based on a historical splitting strategy corresponding to the historical semantic vector to obtain at least one unit problem;

[0015] In response to not finding a historical semantic vector corresponding to the current semantic vector matched in the historical database, splitting the problem to be processed based on a large language model to obtain at least one unit problem.

[0016] In some embodiments, the method further comprises: extracting key elements from the unit problem;

[0017] The extracting of key elements from the unit problem includes:

[0018] In response to matching historical split keywords corresponding to the unit question in the historical database, key elements corresponding to the unit question are determined based on the matched historical split keywords; in response to not matching historical split keywords corresponding to the unit question in the historical database, key elements of the unit question are extracted based on a large language model.

[0019] In some embodiments, matching the historical split keywords corresponding to the unit question in the historical database includes one or more of the following:

[0020] Retrieve and match the key information of each of the unit problems mentioned in the historical database based on the keyword recall channel;

[0021] Retrieve and match the semantic similarity between the unit question to be retrieved and the unit question of the candidate text in the historical database based on the semantic recall channel;

[0022] Normalizing the standardized semantics, keywords, and date scores of the unit question to be retrieved based on the normalized recall channel, and performing retrieval matching based on the normalized data after the normalization processing;

[0023] Based on the weighted recall channel, the normalized data of the unit question to be retrieved is weighted and then retrieved and matched;

[0024] Based on the full recall channel, the semantic vector of the unit question to be retrieved is fully retrieved and matched in the historical database.

[0025] In some embodiments, the method further comprises:

[0026] Sorting one or more search question and answer results for the same unit question;

[0027] Determine the target retrieval question and answer result corresponding to each unit question based on the ranking result;

[0028] The target retrieval question and answer results of each of the unit questions are integrated, and multiple retrieval strategy results are merged, and duplicate items are removed to obtain the retrieval question and answer results of the question to be processed.

[0029] In some embodiments, the ranking of one or more search and answer results for the same unit question includes:

[0030] Determine one or more matching segments in the historical split database that match the current unit problem;

[0031] Calculate the matching scores between the plurality of matching segments and the unit question based on the semantic ranking model;

[0032] One or more search and answer results of the unit question are ranked based on the calculated matching scores.

[0033] In some embodiments, performing question retrieval and answering on the search element of each unit question to obtain a retrieval and answering result includes:

[0034] Obtain unit questions corresponding to the pending questions of each round of tasks, perform question-and-answer processing on the retrieval question-and-answer results corresponding to each unit question retrieved from the historical split database based on the large model, obtain the question-and-answer results of each unit question, and determine the question-and-answer results of the pending questions based on the question-and-answer results of each unit question.

[0035] In some embodiments, performing question retrieval and answering on the search element of each unit question to obtain a retrieval and answering result includes:

[0036] Obtain the unit problems corresponding to the pending problems of each round of tasks, and sort the unit problems of each round of tasks;

[0037] Extract the target unit problem based on the ranking results of the unit problems of each round of tasks, and merge the target unit problems to obtain the merged problem;

[0038] The combined question is processed for question and answer based on the large model to obtain the question and answer result of the question to be processed.

[0039] In some embodiments, the method further comprises:

[0040] Determine the question and answer complexity level of the problem to be processed;

[0041] In response to the problem complexity level of the to-be-processed problem being greater than a preset level, obtaining unit problems corresponding to the to-be-processed problem of each round of tasks, performing question-answering processing on the retrieval question-answering results corresponding to each unit problem retrieved from the historical split database based on the large model, obtaining the question-answering results of each unit problem, and determining the question-answering results of the to-be-processed problem based on the question-answering results of each unit problem;

[0042] In response to the problem complexity level of the problem to be processed being less than a preset level, the unit problems corresponding to the problem to be processed of each round of tasks are obtained, and the unit problems of each round of tasks are sorted; based on the sorting results of the unit problems of each round of tasks, the target unit problems are extracted, and the target unit problems are merged to obtain a merged problem; based on the large model, the merged problem is subjected to question and answer processing to obtain the question and answer results of the problem to be processed.

[0043] In some embodiments, the method also includes: obtaining document information, determining the information type of the document information, matching the corresponding processing method based on the information type to process the corresponding type of pending issues, obtaining key elements and semantic vectors corresponding to the pending issues, and storing the processed key elements and semantic vectors in a historical splitting database.

[0044] A question retrieval and answering device based on a large language model and multi-dimensional technology fusion, the device comprising:

[0045] A splitting module 401 is used to split the problem to be processed into at least one unit problem;

[0046] An extraction module 402, configured to extract a key element of at least one of the unit questions and determine a semantic vector of at least one of the unit questions, and determine a search element according to the key element and the semantic vector;

[0047] Unit question and answer module 403, used for performing question retrieval and answering on the search elements of each unit question to obtain a retrieval and answering result;

[0048] The result question-and-answer module 404 is used to determine the retrieval question-and-answer result of the pending question based on the retrieval question-and-answer result corresponding to each of the unit questions.

[0049] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the question retrieval and answering method based on a large language model and multi-dimensional technology fusion are implemented.

[0050] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a question retrieval and answering method based on a large language model and multi-dimensional technology fusion.

[0051] The above-mentioned question retrieval and answering method and device based on the fusion of large language model and multi-dimensional technology uses large model technology combined with traditional search algorithms to solve the difficulties encountered by traditional financial industry search systems.

[0052] As financial entities are more sensitive, a solution combining NER and large models can be used to accurately extract the required financial entities and related keyword element information, and accurately reach the required information sources.

[0053] As the financial sector is sensitive to time, the method of weighting the time decay coefficient after matching the key time factors is used to ensure the relevance of the time factors in the returned information source and the query.

[0054] In view of the complexity of retrieval questions in the financial industry, that is, questions often have multiple subjects, contrast relationships or parallel logic between multiple objects, by using a few-shot prompt word method to enable the large model to have the ability to split complex questions, split complex questions into simple questions, search for single simple questions, and finally merge multiple search question and answer results through a merging strategy and return them to the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A flowchart of a question retrieval and answering method based on a large language model and multi-dimensional technology fusion provided in one embodiment;

[0056] Figure 2 A schematic diagram of a system for enhancing the retrieval efficiency of intelligent financial research reports and regulations based on a large language model (LLM) and multi-dimensional technology fusion provided in one embodiment;

[0057] Figure 3 A schematic diagram of a flow chart of online and offline execution steps provided in one embodiment;

[0058] Figure 4 A structural block diagram of a question retrieval and answering device based on a large language model and multi-dimensional technology fusion provided in one embodiment;

[0059] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] It should be noted that the Large Language Model (LLM) is an artificial intelligence model that uses large-scale data and powerful computing power for training. The traditional search technology in the financial industry is that the user enters a series of keywords and sends a search request to the browser. The browser performs simple rule matching on the entire knowledge base content based on the requested keywords, sorts the entire text data, and returns the corresponding search question and answer results. However, such a search system cannot perform personalized searches for specific financial scenarios, and simple string matching cannot achieve a deeper understanding of the semantic level.

[0062] With the vigorous development of big model technology, big models have made certain progress in other fields, such as intelligent question answering, machine translation, text generation, image generation, etc. The financial industry urgently needs an opportunity to use big models to improve the shortcomings of traditional search tasks in the financial industry. Financial scenarios have certain particularities. For example, financial scenarios are highly sensitive to financial entities, such as company names, industry names, professional terms, stock codes, etc., and have low error tolerance. In financial scenarios, users are highly sensitive to time. For questions with specific time cutoffs, the results need to be returned as close as possible to the time point involved in the question. For questions without specific time, the latest data or news needs to be returned in real time. In financial scenarios, complex questions are often encountered. The existing retrieval system cannot accurately understand the logical relationship between the sub-questions in the compound question, which often leads to large deviations in search results. The semantic similarity between questions in financial documents is very high, and the search engine needs to accurately understand the user's question intention and return the most relevant results.

[0063] In response to the current technical problems, the present application provides a retrieval efficiency enhancement system for intelligent financial research reports and regulations based on a large language model (LLM) and based on multi-dimensional technology integration. The retrieval enhancement technology for financial document scenarios provided by this application solves the problem of the inability to accurately identify financial entities in traditional retrieval systems. The design is targeted at the sensitivity to time in financial scenarios, the complexity of question structure, and the inability to distinguish the semantic information of similar financial terms, which greatly improves the retrieval accuracy of the retrieval system in financial scenarios.

[0064] In one embodiment, Figure 1 As shown, a question retrieval and answering method based on a large language model and multi-dimensional technology fusion is provided, including the following steps:

[0065] Step 101, splitting the problem to be processed into at least one unit problem;

[0066] The pending question may be a question information of the user, and the user wants to get the corresponding answer to the question after inputting the question information. In some embodiments, the question information input by the user may include multiple questions, and at this time, the question information of the user may be split into unit questions, wherein the unit question may be information including only one question.

[0067] In some embodiments, the question to be processed input by the user can be split into at least one unit question through a large language model.

[0068] In some embodiments, each unit problem may be an independent problem; in some embodiments, different unit problems may have a certain logical relationship, for example, different unit problems have a certain logical relationship and multiple unit problems may together form a large problem.

[0069] In some embodiments, after obtaining the pending questions corresponding to the user's question information, the pending questions input by the user are also preprocessed. The preprocessing may specifically include extracting the elements of the user's question rules and cleaning the rule placeholders. The extraction of the elements of the user's question rules is applicable to the given special conditions, and the cleaning of the rule placeholders is mainly to prevent the user from being unable to recognize the question when asking it. For example, some institution names can be identified by "<Institution Name>".

[0070] In some embodiments, the decomposition of the pending question input by the user may include decomposing the pending question when the pending question includes multiple subjects. For example, in the case of multiple subjects, the question may be decomposed into two short sentences with single subjects, such as "What is the revenue of securities company A and securities company B?" may be decomposed into "What is the revenue of securities company A" and "What is the revenue of securities company B". In other embodiments, the decomposition of the question does not only include decomposing different subjects.

[0071] In some embodiments, the method further comprises providing a pre-stored history database, wherein the history database comprises historical split question data and historical semantic vectors corresponding to each question data.

[0072] The problem to be processed is split into at least one unit problem, including:

[0073] Determine a current semantic vector of the problem to be processed, and match a historical semantic vector corresponding to the current semantic vector in a historical database;

[0074] In response to matching a historical semantic vector corresponding to the current semantic vector in the historical database, splitting the current problem to be processed based on a historical splitting strategy corresponding to the historical semantic vector to obtain at least one unit problem;

[0075] In response to not finding a historical semantic vector corresponding to the current semantic vector matched in the historical database, splitting the problem to be processed based on a large language model to obtain at least one unit problem.

[0076] Specifically, after obtaining the pending problem input by the user, first search the historical disassembly cases in the historical database to check whether there is a similar disassembly method in the historical database. If the corresponding historical disassembly solution is found, the current pending problem can be disassembled with reference to the historical disassembly solution, which can improve the disassembly efficiency of the current pending problem.

[0077] In some embodiments, matching can be performed in the historical database based on the closest vector distance. If the historical disassembly solution can be successfully matched, the disassembly result is directly returned and the process ends. If the corresponding disassembly solution is not matched, the problem can be decomposed into unit problems based on the large language model under given few-shot conditions.

[0078] In the above embodiment, by matching disassembly solutions in a historical disassembly database when disassembling a problem, the efficiency of disassembling the current problem to be processed can be improved.

[0079] In some embodiments, the method further comprises: extracting key elements from the unit problem.

[0080] In some embodiments, the extracting key elements from the unit question includes: extracting key elements from the unit question based on a NER model.

[0081] In some embodiments, the extracting of key elements of the unit question includes: in response to matching historical split keywords corresponding to the unit question in the historical database, determining the key elements corresponding to the unit question based on the matched historical split keywords; in response to not matching historical split keywords corresponding to the unit question in the historical database, extracting key elements of the unit question based on a large language model.

[0082] Specifically, each split unit problem is processed, including using the NER model to extract key information. For example, the key information to be extracted may include one or more of a financial entity, a financial target, a stock code, and a date.

[0083] In some embodiments, a large language model can be used to extract key elements from the split unit problems. For example, "Securities Company A's revenue in 2023" can be split into "Securities Company A" plus "revenue" plus "2003". It should be noted that in the process of extracting key elements from unit problems, historical keyword cases can be retrieved first. If relevant cases are retrieved in the historical keyword cases, the current key elements can be determined directly based on the historical keywords of the retrieved historical related cases, that is, the keyword extraction process is terminated, and the key elements of the current unit problem are determined directly based on the retrieved historical keyword cases. However, when the corresponding historical cases are not successfully retrieved in the historical keyword cases, a large language model can be used to extract key elements of the problem under given few-shot conditions.

[0084] In some embodiments, the method further includes calculating a semantic vector based on the type of question to be retrieved.

[0085] Step 102: extract key elements of at least one of the unit questions and determine a semantic vector of at least one of the unit questions, and determine a search element based on the key elements and the semantic vector.

[0086] Step 103, performing question retrieval and answering on the retrieval elements of each of the unit questions to obtain retrieval and answering results.

[0087] Specifically, question-answer retrieval can be performed for each unit question. The retrieval elements corresponding to each unit question can be determined by calculating the semantic vector or key elements of each unit question, and then the questions can be retrieved based on the retrieval elements of each unit question to obtain the answers to each unit question. Finally, the final answers to the questions to be retrieved can be obtained based on the answers to each unit question.

[0088] In some embodiments, the method further includes: matching key elements corresponding to each unit question in a historical database. In some embodiments, the method further includes extracting a semantic vector of each unit question and matching in the historical database based on the semantic vector.

[0089] Matching the historical split keywords corresponding to the unit question in the historical database, and determining the key elements of each unit question based on the matched historical split keywords. Wherein, matching the historical split keywords corresponding to the unit question in the historical database includes one or more of the following:

[0090] Retrieve and match the key information of each of the unit problems mentioned in the historical database based on the keyword recall channel;

[0091] Retrieve and match the semantic similarity between the unit question to be retrieved and the unit question of the candidate text in the historical database based on the semantic recall channel;

[0092] Normalizing the standardized semantics, keywords, and date scores of the unit question to be retrieved based on the normalized recall channel, and performing retrieval matching based on the normalized data after the normalization processing;

[0093] Based on the weighted recall method, the normalized data of the unit question to be retrieved is weighted and then retrieved and matched; for example, the total score may be calculated based on a preset weighting coefficient and then retrieved and matched.

[0094] Based on the full recall channel, the semantic vector of the unit question to be retrieved is fully retrieved and matched in the historical database.

[0095] Search and match the key information of each unit question in the historical database, for example, search and match the key elements mentioned in the large language model in the historical database;

[0096] It is also possible to perform search matching based on the number of words in the text of the unit question to be searched and the multiple of the set maximum context.

[0097] In some embodiments, the historical database can be searched based on key information, for example, according to the screening conditions of financial entity, subject matter, and date involved, for example, the key elements extracted from the pending question corresponding to "What is the revenue of securities company A in 2023" include "Securities Company A" and "2003". The historical database can be searched and matched based on the key elements including "Securities Company A" and "2003".

[0098] In some embodiments, retrieval and matching can be performed according to the key elements extracted by the large language model. For example, "Securities Company A's revenue in 2023 is a competitor", and the key element (word) extracted by the large language model is "revenue". It should be noted that the sorting model for the extracted keywords (key elements) can be set according to needs, such as the BM25 model, which is not limited here.

[0099] For example, for the search text of the problem to be processed, the number of words in the text can be less than 3 times or other multiples of the set maximum context (for example, it can be a large language model setting).

[0100] In some embodiments, the semantic similarity score between the candidate text and the unit question can also be calculated based on the search text, and the search can be performed based on the similarity score. The similarity score has a value interval range of (-1, 1), and the score is calculated based on bm25. The score of the date can be decayed according to the distance of the question date.

[0101] In some embodiments, semantics, keywords, and date scores may also be standardized, and the standardized value range is (0-1). Standardization may be understood as maximum-minimum normalization, and retrieval matching is performed based on the standardized data.

[0102] In some embodiments, weights can be set in advance for each key element, and a total score can be obtained by weighted calculation according to the set weights. Specifically, weights are pre-set in the retrieval system. For example, the weight of semantics can be 0.6, the weight of keywords can be 0.4, and the weight of dates can be 0.5. The weight settings here can be customized according to different business needs. For example, for research reports, the proportion of dates is high, and real-time requirements are high; for rules and regulations, the proportion of dates is not high, and for fuzzy searches (such as chat records), the keyword score is very low.

[0103] In some embodiments, search and matching can be performed in the historical database according to element rules, such as financial entities and dates as screening conditions. Specifically, full matching can be performed in the historical database according to semantics; in some embodiments, the scores of semantics and dates can be standardized (maximum and minimum normalization, range 0-1). In some embodiments, weighted calculation can also be performed according to pre-set weights to obtain a total score, and search and matching can be performed based on the total score.

[0104] In another embodiment, based on special search strategies (for applications with additional requirements in some financial scenarios, such as "Company C's profit ranking in the past three years", dynamic calling interfaces and customized capabilities are required.

[0105] Step 104: Determine the search question and answer result of the question to be processed based on the search question and answer result corresponding to each of the unit questions.

[0106] In some embodiments, fusing the retrieval question and answer results of the unit question includes: merging the retrieval question and answer results of multiple unit questions to obtain the retrieval question and answer results of the question to be processed. Specifically, the retrieval question and answer results of multiple unit questions can be merged and duplicate items can be removed to obtain the retrieval question and answer results of the question to be processed.

[0107] In some embodiments, the method further includes: sorting one or more retrieval question and answer results for the same unit question.

[0108] In some embodiments, the method also includes: sorting one or more retrieval question and answer results for the same unit question; determining the target retrieval question and answer results corresponding to each unit question based on the sorting results; fusing the target retrieval question and answer results of each unit question, and merging multiple retrieval strategy results, removing duplicates to obtain the retrieval question and answer results of the question to be processed.

[0109] In some embodiments, the sorting of one or more retrieval question and answer results for the same unit question includes: determining one or more matching fragments in the historical split database that match the current unit question; calculating the matching scores of the multiple matching fragments and the unit question based on a semantic sorting model; and sorting the one or more retrieval question and answer results for the unit question based on the calculated matching scores.

[0110] In some embodiments, the step of sorting multiple search question and answer results for the same unit question includes: calculating the matching score between the current segment retrieved from the historical database and the unit question based on the semantic sorting model, and sorting the multiple search question and answer results based on the obtained matching score. For example, the unit question can be placed in the title text and the relevance score can be calculated after being spliced ​​together.

[0111] In some embodiments, the matching score is standardized. The date score is standardized, where the score calculated by the date is a day-level score. If the user does not have date data when asking, the time of the user's question can be used as the question time. The date score is standardized according to the standard that the closer the question time is, the closer it is to 1, and the farther the question time is, the closer it is to 0.

[0112] In some embodiments, performing question retrieval and answering on the search element of each unit question to obtain a retrieval and answering result includes:

[0113] Obtain unit questions corresponding to the pending questions of each round of tasks, perform question-and-answer processing on the retrieval question-and-answer results corresponding to each unit question retrieved from the historical split database based on the large model, obtain the question-and-answer results of each unit question, and determine the question-and-answer results of the pending questions based on the question-and-answer results of each unit question.

[0114] In some embodiments, performing question retrieval and answering on the search element of each unit question to obtain a retrieval and answering result includes:

[0115] Obtain the unit problems corresponding to the pending problems of each round of tasks, and sort the unit problems of each round of tasks;

[0116] Extract the target unit problem based on the ranking results of the unit problems of each round of tasks, and merge the target unit problems to obtain the merged problem;

[0117] The combined question is processed for question and answer based on the large model to obtain the question and answer result of the question to be processed.

[0118] In some embodiments, the method further comprises:

[0119] Determine the question and answer complexity level of the problem to be processed;

[0120] In response to the complexity level of the problem to be processed being greater than a preset level, performing question-answering processing on the search question-answering results corresponding to each unit problem retrieved from the historical split database based on the large model to obtain a question-answering result for each unit problem, and determining a question-answering result for the problem to be processed based on the question-answering result for each unit problem;

[0121] In response to the problem complexity level of the problem to be processed being less than a preset level, the unit problems corresponding to the problem to be processed of each round of tasks are obtained, and the unit problems of each round of tasks are sorted; based on the sorting results of the unit problems of each round of tasks, the target unit problems are extracted, and the target unit problems are merged to obtain a merged problem; based on the large model, the merged problem is subjected to question and answer processing to obtain the question and answer results of the problem to be processed.

[0122] In some embodiments, it also includes weighted sorting, taking the previous number of text fragments, and in some embodiments, the fragments retrieved from the historical database can be used as the large model prompt context for question and answer. In some embodiments, two modes can be included. The first mode can answer the unit questions separately (for example, non-streaming parallel questions), merge the unit questions and the answers they get, and then answer the original questions (streaming), and the large language model merges the above. The second mode can be achieved by merging the text content of the unit questions, for example, taking the most forward-ranked fragment from the fragment of the current unit question in each round, skipping the fragment if it is repeated, and covering each round of unit questions until the number of words exceeds the set context length, merging the context and merging the questions, and answering them by the large language model (streaming). It should be noted that if the question is more complicated, the first mode is preferred. If the question is relatively simple, the second mode is recommended. The second mode can increase the speed and save the step of answering by the large language model.

[0123] In some embodiments, the method further includes: feeding back the obtained retrieval question and answer results for the question to be processed to the user.

[0124] In some embodiments, the method further includes the user providing feedback on the obtained search question and answer results. For example, if the question answer makes the user feel good, the current search question and answer results can be recorded in a case, such as a historical database (for subsequent few-shot). The record is used for clause splitting and keyword splitting. For example, if the user feels that the search question and answer results are poor, feedback can be provided or transferred to manual service.

[0125] In some embodiments, the method further includes: storing the search question and answer results that meet preset conditions in a historical database.

[0126] In some embodiments, the method also includes: obtaining document information, determining the information type of the document information, matching the corresponding processing method based on the information type to process the corresponding type of pending issues, obtaining key elements and semantic vectors corresponding to the pending issues, and storing the processed key elements and semantic vectors in a historical splitting database.

[0127] In some embodiments, it can be understood as parsing documents in an offline state. For example, the acquired document information is first processed and uploaded according to the document type, and the main processed document types include text files, table files, etc.

[0128] In some embodiments, text files (including pdf, word, txt, ppt, etc.) are extracted. Specifically, the text files can be converted into pictures and then analyzed for layout to extract text. Specifically, it can include titles, texts, tables, charts (bar graphs, scatter graphs, and line graphs), pictures, and information includes email addresses, authors, etc., where element titles include table headers and figure headers. Among them, the document structure is analyzed according to the text content, the text and type are matched, and the title ownership is determined, specifically matching according to the text position, and positioning the table header and text.

[0129] In some embodiments, text extraction is performed on table files such as xlsx, csv and other format files. If the table meets the template requirements for information extraction, it is processed according to the template. If it does not meet the requirements, the information is processed according to the header (the format is the header plus the content). For cells without headers, they can be directly merged, for example, they can be directly spelled into text. Specifically, the semantic vector can be calculated by the language model to see whether there is a clear difference between the first line and the following lines to determine whether there is a header.

[0130] In some embodiments, the method further includes storing the pre-processed document information in segments in a database.

[0131] The processed documents are processed into fragments and then put into the database. It should be noted that the processing scope includes text, tables, charts, pictures, information, etc.

[0132] In some embodiments, the information processing method for the text type includes, first, adding an attributed title to the text to form a label. Secondly, extracting financial elements from the text, for example, the financial institutions mentioned in the fragment, financial targets (such as stock codes, bond codes, fund codes, etc.), and the time involved can be extracted through the NER algorithm to form labels (title labels, financial institution labels, target labels, time labels, etc.). Thirdly, the text is cut and segmented according to the number of words, which can be specifically based on the context length limit of the language model. The cut segments here are 512 Chinese characters, with all the above labels. Finally, the text fragments are vectorized and the semantics of the text paragraphs are extracted through the semantic model.

[0133] In some embodiments, the information processing method for table types includes, first, identifying the table and restoring the table structure, which can be specifically achieved through tatr technology; secondly, adding an attributed title to the table, and cutting it by rows and word count, while filling in the table header; and finally, vectorizing the table title and obtaining the title semantics through a semantic model.

[0134] In some embodiments, the chart type is processed by converting the chart into table data, and the remaining steps are consistent with the table.

[0135] In some embodiments, the image type information is processed by first adding an attribution title to the image, and then vectorizing the image title, obtaining the title semantics through a semantic model, understanding the image as text, and obtaining the semantics of the text.

[0136] In some embodiments, the processing method of information key-value pairs includes first processing the information and converting it into a declarative sentence according to a template, such as the author is XXX, the email is XXX, secondly, adding an attribution title, and finally, vectorizing the text fragments and extracting the semantics of the text paragraph through a semantic model.

[0137] Then, data persistence is performed, and the text content and its labels and date information are persisted in the database and the full-text search database (es or zinc), and the semantics, date, and label are stored in the vector database (milvus) (here they are stored according to the corresponding id).

[0138] In some embodiments, when the database is used to search for questions, it can be implemented through an online system, and the online part is used to process the user's questions and return the search question and answer results.

[0139] In some embodiments, see Figure 2, providing a retrieval efficiency enhancement system for intelligent financial research reports and regulations based on the large language model (LLM) and multi-dimensional technology integration, Figure 3 The following is a flow chart of the online and offline execution steps of the specific steps of the present invention. The specific steps include:

[0140] Step 1: Preprocess text information, including adding attribution tags, extracting financial elements, segmenting text fragments, and vectorizing fragments.

[0141] Step 2: Preprocess the table information, including text extraction, processing of header and table content attribution information, and calculation of semantic vectors.

[0142] Step 3: For chart information, restore it to a table and proceed with the same process as step 2.

[0143] Step 4: For image information, first add an attribution title to the image, vectorize the image title, obtain the title semantics through the semantic model, understand the image as text, and obtain semantic information from the text.

[0144] Step 5: Persist the above pre-processed data and store it in the database.

[0145] Step 6: The user inputs the information he wants to query through natural language, which can be a complex question or a simple question.

[0146] Step 7: Use the large language model to split the complex question into unit questions. The unit question here refers to a simple question about a single object and a single query element.

[0147] Step 8. For each unit question, extract keywords through keyword matching or natural language processing models, including dates, financial entities, etc.

[0148] Step 9: By matching the semantic vector and keywords of the unit question with the semantic vector and keywords of the candidate segment, the candidate set is recalled through various strategies. For example, the keyword recall channel is a regular full match method, the semantic recall channel is a method of calculating vector similarity, and the time recall channel is a date weighted method.

[0149] Step 10: De-duplicate the recalled segments, perform rough sorting, and truncate them. The logic of truncation is to normalize the scores of each recalled channel and weightedly merge them into a rough ranking score, sort them from high to low, and retain the highest topN.

[0150] Step 11: Perform fine ranking and scoring on the topN results, and take the topk fragments as the final unit question retrieval and answering results returned.

[0151] Step 12: Merge and organize the returned results of each unit question into the context information of the large language model, and post-process the results returned by the large language model and display them to the user.

[0152] Step 13: Based on user feedback, extract data about keywords involved in positive examples and store them in the database. Feedback negative examples to developers as retrieval negative examples for troubleshooting.

[0153] The intelligent financial research report and regulation retrieval efficiency enhancement system based on the multi-dimensional technology fusion of the large language model (LLM) provided by the present invention uses the large model technology combined with the traditional search algorithm to solve the difficulties encountered by the traditional financial industry search system. In view of the sensitivity of financial entities, the required financial entities and related keyword element information are accurately extracted through the solution combining NER and the large model, and the required information source is accurately reached. In view of the sensitivity of the financial field to time, after matching the key elements of time, the method of weighting the time attenuation coefficient is performed to ensure the correlation between the time elements in the returned information source and the query sentence. In view of the complexity of the retrieval questions in the financial industry, that is, the questions are often multiple subjects, and the contrast relationship or parallel logic between multiple objects, by performing few-shot prompt words on the large model, the large model has the ability to split complex questions, split complex questions into simple questions, and search for single simple questions. Finally, multiple search question and answer results are merged and returned to the user through the merging strategy.

[0154] There are often semantic similarities between research report terms, and traditional natural language models cannot distinguish the semantic differences between financial terms. Since the underlying model of the large model uses a larger corpus for training, it has a stronger semantic understanding ability. By using the large model to distinguish the semantic differences between different financial terms, this defect of the traditional search system can be effectively improved. In addition, the entire method and system are scalable. In the face of more professional needs within the enterprise in the future, the retrieval system can be customized to achieve more and more professional business and work scenarios.

[0155] It is understandable that search enhancement technology in the financial field is usually a highly professional field, which requires professional customization in combination with natural language processing, large model technology, and search ranking system. The present invention proposes an intelligent financial research report and regulatory search efficiency enhancement system based on multi-dimensional technology fusion of large language model (LLM), which aims to enable corporate users to retrieve financial information more conveniently and accurately and improve office efficiency.

[0156] It should be noted that common large language models include the GPT series and Wenxinyiyan. It has large-scale parameters and a huge number of parameters, which enables it to learn rich language knowledge and patterns, a wide range of language understanding capabilities, and can understand the meaning of various natural language expressions, including complex sentence structures and semantic relationships, and generation capabilities, which can generate natural, fluent and logical text content. Knowledge fusion integrates a large amount of text knowledge during the training process. Large language models are widely used in many fields, such as natural language processing tasks such as text generation, machine translation, question-answering systems, etc. It provides people with more intelligent and efficient language interaction methods and services.

[0157] In the above embodiment, by using the large model-based question retrieval and answering method provided in this application, the above method can be executed and the corresponding technical effect can be achieved.

[0158] An embodiment of the present invention also provides a computer terminal device capable of implementing all the steps of the method in the above embodiment, including a processor, a readable storage medium, and a computer program, wherein the computer readable storage medium stores the computer program, and when the computer program is executed by the processor, all the steps of the method in the above embodiment are implemented.

[0159] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0160] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the accompanying drawings may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0161] In one embodiment, Figure 4 As shown, a question retrieval and answering device 400 based on a large language model and multi-dimensional technology fusion is provided. The device can adopt a software module or a hardware module, or a combination of the two to form a part of a computer device. The device specifically includes:

[0162] A splitting module 401 is used to split the problem to be processed into at least one unit problem;

[0163] An extraction module 402, configured to extract a key element of at least one of the unit questions and determine a semantic vector of at least one of the unit questions, and determine a search element according to the key element and the semantic vector;

[0164] Unit question and answer module 403, used for performing question retrieval and answering on the search elements of each unit question to obtain a retrieval and answering result;

[0165] The result question-and-answer module 404 is used to determine the search question-and-answer result of the pending question based on the search question-and-answer result corresponding to each of the unit questions.

[0166] For the specific limitations of the question retrieval and question-answering device based on the fusion of a large language model and multi-dimensional technology, please refer to the limitations of the question retrieval and question-answering method based on the fusion of a large language model and multi-dimensional technology in the above text, which will not be repeated here. Each module in the above-mentioned question retrieval and question-answering device based on the fusion of a large language model and multi-dimensional technology can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0167] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 The computer device includes a processor, a memory and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store pending questions and historical split data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a question retrieval and answering method based on a large language model and multi-dimensional technology fusion is implemented.

[0168] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0169] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0170] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0171] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0172] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0173] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A question retrieval and answering method based on a large language model and multi-dimensional technology fusion, characterized in that: The method comprises: Split the problem to be processed into at least one unit problem; Extracting at least one key element of the unit question and determining at least one semantic vector of the unit question, and determining a search element according to the key element and the semantic vector; Performing question retrieval and answering on the retrieval elements of each of the unit questions to obtain retrieval and answering results; The retrieval question and answer result of the question to be processed is determined based on the retrieval question and answer result corresponding to each of the unit questions.

2. The method according to claim 1, characterized in that The method further includes providing a pre-stored history database, wherein the history database includes historical split question data and historical semantic vectors and historical key elements corresponding to each question data; The problem to be processed is split into at least one unit problem, including: Determine a current semantic vector of the problem to be processed, and match a historical semantic vector corresponding to the current semantic vector in a historical database; In response to matching a historical semantic vector corresponding to the current semantic vector in the historical database, splitting the current problem to be processed based on a historical splitting strategy corresponding to the historical semantic vector to obtain at least one unit problem; In response to not finding a historical semantic vector corresponding to the current semantic vector matched in the historical database, splitting the problem to be processed based on a large language model to obtain at least one unit problem.

3. The method according to claim 1, characterized in that The method further comprises: extracting key elements from the unit problem; The extracting of key elements from the unit problem includes: In response to matching historical split keywords corresponding to the unit question in the historical database, key elements corresponding to the unit question are determined based on the matched historical split keywords; in response to not matching historical split keywords corresponding to the unit question in the historical database, key elements of the unit question are extracted based on a large language model.

4. The method according to claim 3, characterized in that Matching the historical split keywords corresponding to the unit problem in the historical database includes one or more of the following: Retrieve and match the key information of each of the unit problems mentioned in the historical database based on the keyword recall channel; Retrieve and match the semantic similarity between the unit question to be retrieved and the unit question of the candidate text in the historical database based on the semantic recall channel; Normalizing the standardized semantics, keywords, and date scores of the unit question to be retrieved based on the normalized recall channel, and performing retrieval matching based on the normalized data after the normalization processing; Based on the weighted recall channel, the normalized data of the unit question to be retrieved is weighted and then retrieved and matched; Based on the full recall channel, the semantic vector of the unit question to be retrieved is fully retrieved and matched in the historical database.

5. The method according to claim 1, characterized in that The method further comprises: Sorting one or more search question and answer results for the same unit question; Determine the target retrieval question and answer result corresponding to each unit question based on the ranking result; The target retrieval question and answer results of each of the unit questions are integrated, and multiple retrieval strategy results are merged, and duplicate items are removed to obtain the retrieval question and answer results of the question to be processed.

6. The method according to claim 5, characterized in that The sorting of one or more search question and answer results for the same unit question includes: Determine one or more matching segments in the historical split database that match the current unit problem; Calculate the matching scores between the plurality of matching segments and the unit question based on the semantic ranking model; One or more search and answer results of the unit question are ranked based on the calculated matching scores.

7. The method according to any one of claims 1 to 6, characterized in that The step of performing question retrieval and answering on the retrieval element of each unit question to obtain a retrieval and answering result includes: Obtain unit questions corresponding to the pending questions of each round of tasks, perform question-and-answer processing on the retrieval question-and-answer results corresponding to each unit question retrieved from the historical split database based on the large model, obtain the question-and-answer results of each unit question, and determine the question-and-answer results of the pending questions based on the question-and-answer results of each unit question.

8. The method according to any one of claims 1 to 6, characterized in that The step of performing question retrieval and answering on the retrieval element of each unit question to obtain a retrieval and answering result includes: Obtain the unit problems corresponding to the pending problems of each round of tasks, and sort the unit problems of each round of tasks; Extract the target unit problem based on the ranking results of the unit problems of each round of tasks, and merge the target unit problems to obtain the merged problem; The combined question is processed for question and answer based on the large model to obtain the question and answer result of the question to be processed.

9. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Determine the question and answer complexity level of the problem to be processed; In response to the problem complexity level of the to-be-processed problem being greater than a preset level, obtaining unit problems corresponding to the to-be-processed problem of each round of tasks, performing question-answering processing on the retrieval question-answering results corresponding to each unit problem retrieved from the historical split database based on the large model, obtaining the question-answering results of each unit problem, and determining the question-answering results of the to-be-processed problem based on the question-answering results of each unit problem; In response to the problem complexity level of the problem to be processed being less than a preset level, the unit problems corresponding to the problem to be processed of each round of tasks are obtained, and the unit problems of each round of tasks are sorted; based on the sorting results of the unit problems of each round of tasks, the target unit problems are extracted, and the target unit problems are merged to obtain a merged problem; based on the large model, the merged problem is subjected to question and answer processing to obtain the question and answer results of the problem to be processed.

10. The method according to any one of claims 1 to 6, characterized in that The method also includes: obtaining document information, determining the information type of the document information, matching the corresponding processing method based on the information type to process the corresponding type of pending issues, obtaining key elements and semantic vectors corresponding to the pending issues, and storing the processed key elements and semantic vectors in a historical splitting database.

Images