Product retrieval method, device and equipment based on large language model and storage medium

Abstract

The invention relates to the technical field of data processing and financial science and technology, and particularly discloses a product retrieval method and device based on a large language model, computer equipment and a storage medium. Obtaining a user question, and determining a target data source in a preset knowledge base based on the user question; when the target data source is a preset vector database, converting the user question into a question vector; searching in the vector database based on the question vector to obtain an answer text block corresponding to the question vector; and based on a preset large language model and the answer text block, obtaining a question answer corresponding to the user question, and taking the question answer as a product retrieval result. According to the method, the target data source can be determined according to the user question, and the user question is subjected to related processing according to the target data source, so that the data search amount is reduced, the relevancy between the search result and the user question is improved, and the search efficiency and the accuracy of the search result are improved.

Description

Technical Field

[0001] The present application relates to the fields of data processing and financial technology, and in particular to a product retrieval method, device, computer equipment and storage medium based on a large language model. Background Art

[0002] With the development of computer technology, financial RAG (Retrieval-Augmented Generation) systems based on computer technology have gradually emerged. However, in actual application, insurance products cover a wide range, there are many types of insurance, insurance clauses are lengthy, and the same insurance has slight differences in different regions. When business personnel sell insurance products to customers, it is difficult to obtain products that meet user expectations using system search, which reduces user experience. In addition, due to the professionalism and complexity of the financial field, general large language models may not be able to accurately understand and process financial-specific terms, transaction structures, and regulatory policy details. The existing large language models have slow response speeds when processing real-time financial data streams and lack strict verification mechanisms for financial business logic, resulting in low product search efficiency and search accuracy. Therefore, how to improve product search efficiency and search accuracy has become an urgent problem to be solved. Summary of the invention

[0003] The present application provides a product retrieval method, apparatus, computer device and storage medium based on a large language model to improve product search efficiency and search accuracy.

[0004] In a first aspect, the present application provides a product retrieval method based on a large language model, the method comprising:

[0005] Obtaining a user question, and based on the user question, determining a target data source in a preset knowledge base;

[0006] When the target data source is a preset vector database, converting the user question into a question vector;

[0007] Searching the vector database based on the question vector to obtain an answer text block corresponding to the question vector;

[0008] Based on the preset large language model and the answer text block, an answer to the question corresponding to the user question is obtained as a product retrieval result.

[0009] In a second aspect, the present application further provides a product retrieval device based on a large language model, the device comprising:

[0010] A data source determination module, used to obtain a user question and determine a target data source in a preset knowledge base based on the user question;

[0011] A question vector conversion module, used for converting the user question into a question vector when the target data source is a preset vector database;

[0012] A text block obtaining module, used to search the vector database based on the question vector to obtain an answer text block corresponding to the question vector;

[0013] The question answer obtaining module is used to obtain the question answer corresponding to the user question as the product retrieval result based on the preset large language model and the answer text block.

[0014] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the product retrieval method based on a large language model as described above when executing the computer program.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the product retrieval method based on the large language model as described above.

[0016] The present application discloses a product retrieval method, device, computer equipment and storage medium based on a large language model, which obtains user questions and determines a target data source in a preset knowledge base based on the user questions; when the target data source is a preset vector database, the user question is converted into a question vector; based on the question vector, the vector database is searched to obtain an answer text block corresponding to the question vector; based on the preset large language model and the answer text block, the answer to the question corresponding to the user question is obtained as a product retrieval result. The method can determine the target data source according to the user question, and perform relevant processing on the user question according to the target data source, thereby reducing the amount of data search, improving the relevance of the search results to the user question, and thus improving the search efficiency and the accuracy of the search results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1is a schematic flow chart of a first embodiment of a product retrieval method based on a large language model provided by an embodiment of the present application;

[0019] Figure 2 is a schematic flow chart of a second embodiment of a product retrieval method based on a large language model provided in an embodiment of the present application;

[0020] Figure 3 is a schematic flow chart of a third embodiment of a product retrieval method based on a large language model provided in an embodiment of the present application;

[0021] Figure 4 A schematic block diagram of a product search device based on a large language model provided in an embodiment of the present application;

[0022] Figure 5 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0025] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0026] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] The embodiments of the present application provide a product retrieval method, device, computer equipment and storage medium based on a large language model. Among them, the product retrieval method based on a large language model can be applied to a server, determine the target data source according to the user question, and perform relevant processing on the user question according to the target data source, thereby reducing the amount of data search, improving the relevance of search results to user questions, and thus improving search efficiency and the accuracy of search results. Among them, the server can be an independent server or a server cluster.

[0028] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0029] See also Figure 1 , Figure 1 This is a schematic flow chart of a product retrieval method based on a large language model provided by an embodiment of the present application. The product retrieval method based on a large language model can be applied to a server to determine a target data source according to a user question, and to perform relevant processing on the user question according to the target data source, thereby reducing the amount of data search, improving the relevance of search results to user questions, and thereby improving search efficiency and the accuracy of search results.

[0030] like Figure 1 As shown, the product retrieval method based on the large language model specifically includes steps S101 to S104.

[0031] S101, obtaining a user question, and determining a target data source in a preset knowledge base based on the user question;

[0032] In one embodiment, the user question may include questions related to financial products such as insurance and financial management that the user needs to search for.

[0033] In one embodiment, a question input by a user may be received as a user question through an input device (such as a keyboard, a microphone, etc.). For example, if a user inputs text content through a keyboard, the text content input by the user is obtained as the user question. If a user inputs voice content through a microphone, the user's voice is converted into text to obtain text corresponding to the voice content as the user question.

[0034] In one embodiment, the knowledge base stores vectors corresponding to various user questions and search template data, including a vector database and a search database. The vector database stores text blocks and vectors corresponding to each text block. The search database stores relational databases and structural information of each relational database.

[0035] In one embodiment, the obtained user questions are transmitted to the agents distribution system, and the agents distribution system is used to process the user questions to obtain the source of data corresponding to the user questions, that is, the target data source. The agents distribution system generally refers to a software architecture that can analyze the data sources of different user requests or tasks according to the nature of the user requests or tasks.

[0036] Furthermore, after determining the target data source in a preset knowledge base based on the user question, the method further includes: when the target data source is a preset relational database, parsing the user question to obtain key fields; generating a target search statement based on the key fields and a search template of the relational database; searching the relational database based on the target search statement to obtain search results; processing the search results based on a preset instruction framework to obtain answer generation instructions; and guiding the large language model to process the search results based on the answer generation instructions to obtain an answer to the question.

[0037] In one embodiment, when the target data source is a preset retrieval database, natural language processing technology is used to understand the questions raised by the user, and key fields required for retrieval are extracted from the user questions. These key fields are key information for answering the questions.

[0038] In one embodiment, the key fields are filled into the corresponding search template to construct a target search statement, and the constructed target search statement is executed on the relational database to perform a search and obtain a search result.

[0039] In one embodiment, the obtained search structure is filled into a preset instruction framework to generate an answer generation instruction. The answer generation instruction can guide the large language model to sort and select data to generate the final answer. The instruction framework can be a pre-designed database query statement or a similar database operation. Each instruction framework corresponds to a specific query intent, such as finding products based on a price range or finding related products based on conditions.

[0040] In one embodiment, the search results and answer generation instructions are sent to a large language model, which uses its natural language understanding and generation capabilities to analyze the search results and generate answers relevant to the user's questions.

[0041] In one embodiment, the answer generated by the large language model is presented to the user as a product search result, which can be in the form of text, list, table or other visual form.

[0042] In the above embodiment, the answer generation instruction guides the large language model to understand the user's question and organize and select the search results to obtain the answer to the question, thereby improving the understanding accuracy of the large language model and further improving the accuracy of the search results.

[0043] S102, when the target data source is a preset vector database, converting the user question into a question vector;

[0044] In one embodiment, when the target data source is a vector database, the user question is converted into a question vector through an embedding model, where embedding is a method of converting high-dimensional discrete data (such as words) into a low-dimensional continuous vector representation.

[0045] In a specific embodiment, the user questions are cleaned and standardized, including removing punctuation marks, converting to lowercase, segmenting, removing stop words, etc. Then, the segmented text is converted into a word vector, and a word embedding model, such as Word2Vec, GloVe or FastText, can be used to convert the text into a word vector.

[0046] In one embodiment, a pre-trained context-dependent language model (such as BERT, GPT, RoBERTa, etc.) is used to obtain the interaction and semantic relationship between each word in the context of the user's question. According to the interaction and semantic relationship between words, the word vectors of all words in the text block are aggregated into a single vector representation of the user's question to obtain the question vector corresponding to the user's question.

[0047] S103, searching the vector database based on the question vector to obtain an answer text block corresponding to the question vector;

[0048] In one embodiment, the question vector is compared with the text block vector in the vector database, and the text block vector most similar to the question vector is determined in the vector database. Specifically, the similarity between the question vector and each text block vector in the vector database is calculated to obtain the similarity between the question vector and each text block vector, and the text block vector with the highest similarity is determined.

[0049] In one embodiment, the text block corresponding to the text block vector with the highest similarity is searched in the vector database as the answer text block corresponding to the question vector. The vector database stores the text blocks and the vectors corresponding to each text block.

[0050] S104: Based on the preset large language model and the answer text block, an answer corresponding to the user question is obtained as a product search result.

[0051] Furthermore, the method of obtaining an answer to a question corresponding to the user question based on a preset large language model and the answer text block includes: processing the answer text block based on a preset instruction framework to obtain an answer generation instruction; and guiding the large language model to process the answer text block based on the answer generation instruction to obtain the answer to the question.

[0052] In one embodiment, the instruction framework can be a pre-designed database query statement or a similar database operation. Each instruction framework corresponds to a specific query intention, such as searching for products according to a price range or searching for related products according to a condition.

[0053] In one embodiment, the answer text block corresponding to the user's question is matched with a preset instruction frame to determine which instruction frame best meets the user's intention. The instruction frame is then filled in according to the key fields of the user's question and the obtained answer text block to obtain an answer generation instruction.

[0054] In one embodiment, the constructed answer generation instruction is executed, and the obtained answer text block is passed to a preset large language model (such as GPT-3 or other NLP models), and the answer text block is further processed and analyzed using the model's understanding and generation capabilities. Specifically, the answer text blocks from the vector database are integrated into a coherent answer; the answer text blocks are analyzed to extract key information, such as rankings, statistics or trends, etc.; and answers to user questions are generated based on the integrated and analyzed answer text blocks.

[0055] In one embodiment, the answer generated by the large language model is presented to the user as a product search result, which can be in the form of text, list, table or other visual form.

[0056] The above implementation provides a product retrieval method, device, computer equipment and storage medium based on a large language model, obtains user questions, and based on the user questions, determines the target data source in a preset knowledge base; when the target data source is a preset vector database, converts the user question into a question vector; searches the vector database based on the question vector to obtain the answer text block corresponding to the question vector; based on the preset large language model and the answer text block, obtains the answer to the question corresponding to the user question as a product retrieval result. This method can determine the target data source according to the user question, and perform relevant processing on the user question according to the target data source, reducing the amount of data search, improving the relevance of the search results to the user question, and thereby improving the search efficiency and the accuracy of the search results.

[0057] See also Figure 2 , Figure 2This is a schematic flow chart of a product retrieval method based on a large language model provided by an embodiment of the present application. The product retrieval method based on a large language model can be applied to a server to generate a knowledge base based on a text document and a relational database, so that when a user question is received, the target data source can be determined based on the user question and searched in the target data source, thereby reducing the amount of data search, improving the relevance of the search results to the user question, and thus improving the search efficiency and accuracy.

[0058] like Figure 2 As shown, the product retrieval method based on the large language model specifically includes steps S201 to S203.

[0059] S201, obtaining original data, wherein the original data includes text documents and relational databases;

[0060] In one embodiment, the original data may be insurance product information or financial product information uploaded by the user.

[0061] In one embodiment, the original data is divided into two formats, namely text documents and relational databases. The former includes file formats such as txt, pdf, and md, and the latter is structured data stored in a relational database.

[0062] S202, processing the text document to obtain a vector database corresponding to the text document, and processing the relational database to generate a retrieval database;

[0063] Furthermore, the processing of the text document to obtain the vector database corresponding to the text document includes: preprocessing the text document to obtain text data; performing text segmentation on the text data to obtain at least one text block; encoding at least one of the text blocks based on a preset word embedding model to obtain at least one text block vector; and generating the vector database based on at least one of the text block vectors.

[0064] In one embodiment, preprocessing the text document includes extracting text from the text document and storing the text in a fixed format, wherein the fixed format can be set by the user according to actual needs.

[0065] In a specific embodiment, text documents include documents containing only text and documents containing pictures. For documents containing only text, the text data in the document can be extracted through a text document loader. For documents containing pictures, the pictures can be processed by a multimodal document loader for OCR (Optical Character Recognition) to obtain the text data in the pictures and documents. Among them, OCR technology can recognize and extract text information from image files and convert it into an editable and searchable text format, and is widely used in document digitization, data entry and other scenarios where text in images needs to be converted into electronic text.

[0066] In one embodiment, the obtained text data is transmitted to a text cutter to cut the text into small paragraph text blocks to enhance the relevance of the search information. The text cutter is a natural language processing tool used to divide long text into smaller, semantically coherent parts, such as sentences, paragraphs or chapters, to facilitate further text analysis and processing.

[0067] In one embodiment, after the text is cut, the text fragments are encoded into text block vectors through the embedding model, and the text block vector contains all the information of the text block. Specifically, the text fragments are cleaned and standardized, including removing irrelevant characters, converting to lowercase, word segmentation, removing stop words, etc. Each word in the text block is converted into a word vector. A pre-trained context-dependent language model (such as BERT, GPT, RoBERTa, etc.) is used to obtain the interaction and semantic relationship between each word in a specific context. According to the interaction and semantic relationship between words, the word vectors of all words in the text block are aggregated into a single vector representation of the text block to obtain a text block vector.

[0068] In one embodiment, all text block vectors are saved, and a one-to-one mapping relationship between text blocks and text block vectors is maintained to generate a vector database for subsequent retrieval and subsequent text tracing.

[0069] Furthermore, the processing of the relational database to generate a retrieval database includes: obtaining structural information corresponding to the relational database, and generating a retrieval template for the relational database based on the structural information; generating the retrieval database based on the relational database and the retrieval template corresponding to the relational database.

[0070] In one embodiment, for a relational database, it is necessary to extract the scheme information (ie, structural information) in the database and construct it into a specific format to facilitate the creation of a prompt (ie, a search template).

[0071] In one embodiment, in the context of a relational database, the scheme information generally refers to the structure of the relational database, including metadata information such as tables, columns, data types, primary keys, foreign keys, etc. Use a preset query statement (such as DESCRIBE table_name or INFORMATION_SCHEMA query) to extract this metadata information from the database. Then, select one or more formats such as JSON, YAML, XML, etc. according to specific needs. Convert the extracted scheme information into the selected format.

[0072] In one embodiment, a specific prompt structure (ie, a search template) is constructed according to the scheme data in the relational database for querying in a specific database.

[0073] In one embodiment, each relational database and its corresponding search template are stored to generate a search database.

[0074] S203: Generate the knowledge base based on the retrieval database and the vector database.

[0075] In one embodiment, the retrieval database and the vector database are stored together to generate a knowledge base.

[0076] In one embodiment, the knowledge base supports real-time insertion, update, deletion and other operations, and the process of adding new knowledge can be completed by simply uploading documents or adding a relational database, so that the knowledge base can be updated in a timely manner according to the products launched by the enterprise, thereby improving the search accuracy of business personnel.

[0077] In the above embodiment, a knowledge base is generated based on text documents and relational databases, so that when a user question is received, a target data source can be determined based on the user question and a search can be performed in the target data source, thereby reducing the amount of data search, improving the relevance of search results to user questions, and thereby improving search efficiency and accuracy.

[0078] See also Figure 3 , Figure 3 This is a schematic flow chart of a product retrieval method based on a large language model provided by an embodiment of the present application. The product retrieval method based on a large language model can be applied to a server to perform vector conversion on user questions and perform similarity detection in a vector database to obtain an answer text block corresponding to the user question, thereby improving the relevance of the answer text block to the user question and further improving the accuracy of the search results.

[0079] like Figure 3 As shown, the product retrieval method based on the large language model specifically includes steps S301 to S303.

[0080] S301, performing similarity calculation between the question vector and each text block vector in the vector database to obtain the similarity between the question vector and each text block vector in the vector database;

[0081] S302, determining a target vector from each text block vector in the vector database based on the similarity;

[0082] S303: Based on the mapping relationship between the target vector and the text block, obtain the text block corresponding to the target vector as the answer text block.

[0083] In one embodiment, the question vector is respectively similar to each vector in the vector database to obtain similarity. Specifically, in natural language processing, especially in word embedding technology, cosine similarity can be used to calculate the similarity of two word vectors.

[0084] Cosine similarity is a method to measure the directional difference between high-dimensional vectors, rather than their magnitude. It determines their similarity by calculating the cosine of the angle between two non-zero vectors.

[0085] Assume that the vector v a is the problem vector and the vector v b is a text block vector in the vector database, they are all n-dimensional column vectors:

[0086] {v} a =[v {a1} ,v {a2} ,…,v {an} ] T

[0087] {v} b =[v {b1} ,v {b2} ,…,v {bn} ] T

[0088] Then the cosine similarity sin(v a ,v b ) can be calculated by the following formula:

[0089]

[0090] Among them, v a ·v b Represents vector v a and vector v b The dot product of ||v a || represents vector v a The Euclidean norm of ||v b ||Vector vb The Euclidean norm of .

[0091] Vector v a and vector v b The dot product v a ·v b for:

[0092]

[0093] And ||v a || and ||v b ||They are:

[0094]

[0095] Therefore, sin(v a ,v b ) is simplified to:

[0096]

[0097] The value range of cosine similarity is between -1 and 1. The closer the value is to 1, the closer the directions of the two vectors are (that is, the more semantically similar the two words are). Since the embedding vectors are usually normalized, there is no need to divide them by the modulus of each vector. The dot product can be directly calculated to obtain the similarity:

[0098]

[0099] In one embodiment, the number of target vectors can be determined by controlling a hyperparameter. Specifically, the number of similar vectors returned is controlled by setting a hyperparameter (such as the recall number k). According to the recall number set by the hyperparameter, the first k vectors ranked from high to low in similarity are selected as the most similar vectors, i.e., the target vector. The hyperparameter can be set by the user according to actual needs.

[0100] In one embodiment, a text block that has a mapping relationship with the target vector is obtained as the answer text block corresponding to the question vector. Specifically, the target vectors are mapped back to their corresponding text blocks using an index or association table in a vector database.

[0101] In the above embodiment, the user question is converted into a vector and a similarity check is performed in the vector database to obtain the answer text block corresponding to the user question, thereby improving the relevance of the answer text block to the user question and further improving the accuracy of the search results.

[0102] See also Figure 4 , Figure 4The embodiment of the present application provides a schematic block diagram of a product retrieval device based on a large language model, and the product retrieval device based on a large language model is used to execute the aforementioned product retrieval method based on a large language model. The product retrieval device based on a large language model can be configured on a server.

[0103] like Figure 4 As shown, the product search device 400 based on the large language model includes:

[0104] The data source determination module 401 is used to obtain a user question and determine a target data source in a preset knowledge base based on the user question;

[0105] A question vector conversion module 402, configured to convert the user question into a question vector when the target data source is a preset vector database;

[0106] A text block obtaining module 403 is used to search the vector database based on the question vector to obtain an answer text block corresponding to the question vector;

[0107] The question answer obtaining module 404 is used to obtain the question answer corresponding to the user question as the product search result based on the preset large language model and the answer text block.

[0108] Furthermore, the question answer obtaining module 404 includes:

[0109] An answer generation instruction obtaining unit, used to process the answer text block based on a preset instruction framework to obtain an answer generation instruction;

[0110] The question answer obtaining unit is used to generate instructions based on the answer to guide the large language model to process the answer text block to obtain the answer to the question.

[0111] Furthermore, the product search device 400 based on a large language model further includes:

[0112] A key field acquisition module, used to parse the user question and obtain key fields when the target data source is a preset relational database;

[0113] A target template generation module, used to generate a target search statement based on the key field and the search template of the relational database;

[0114] A search result obtaining module, used to perform a search in the relational database based on the target search statement to obtain a search result;

[0115] An instruction acquisition module, used to process the search results based on a preset instruction framework to obtain answer generation instructions;

[0116] A retrieval result obtaining module is used to generate instructions based on the answer to guide the large language model to process the retrieval results to obtain the answer to the question.

[0117] Furthermore, the product search device 400 based on a large language model further includes a knowledge base construction module, and the knowledge base construction module includes:

[0118] An original data acquisition unit, used to acquire original data, wherein the original data includes text documents and relational databases;

[0119] A database generating unit, used to process the text document to obtain a vector database corresponding to the text document, and to process the relational database to generate a retrieval database;

[0120] A knowledge base generating unit is used to generate the knowledge base based on the retrieval database and the vector database.

[0121] Furthermore, the database generating unit comprises:

[0122] A text data acquisition subunit, used for preprocessing the text document to obtain text data;

[0123] A text block obtaining subunit, used for performing text segmentation on the text data to obtain at least one text block;

[0124] A text block vector obtaining subunit, used for encoding at least one of the text blocks based on a preset word embedding model to obtain at least one text block vector;

[0125] The vector database generating subunit is used to generate the vector database based on at least one of the text block vectors.

[0126] Furthermore, the database generating unit comprises:

[0127] A structural information acquisition subunit, used to acquire structural information corresponding to the relational database, and generate a search template for the relational database based on the structural information;

[0128] The template database generating subunit is used to generate the search database based on the relational database and the search template corresponding to the relational database.

[0129] Furthermore, the text block obtaining module 403 includes:

[0130] A similarity obtaining unit, used for performing similarity calculation between the question vector and each text block vector in the vector database to obtain the similarity between the question vector and each text block vector in the vector database;

[0131] a target vector determination unit, configured to determine a target vector from each text block vector in the vector database based on the similarity;

[0132] The answer text block acquisition unit is used to acquire the text block corresponding to the target vector as the answer text block based on the mapping relationship between the target vector and the text block.

[0133] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0134] The above-mentioned device can be implemented in the form of a computer program. Figure 5 Runs on the computer device shown.

[0135] See also Figure 5 , Figure 5 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0136] See also Figure 5 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0137] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any product retrieval method based on a large language model.

[0138] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0139] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any product retrieval method based on the large language model.

[0140] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0141] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0142] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0143] Obtaining a user question, and based on the user question, determining a target data source in a preset knowledge base;

[0144] When the target data source is a preset vector database, converting the user question into a question vector;

[0145] Searching the vector database based on the question vector to obtain an answer text block corresponding to the question vector;

[0146] Based on the preset large language model and the answer text block, an answer to the question corresponding to the user question is obtained as a product retrieval result.

[0147] In one embodiment, when the processor obtains the answer to the user's question based on the preset large language model and the answer text block, it is used to implement:

[0148] Based on a preset instruction framework, the answer text block is processed to obtain an answer generation instruction;

[0149] Based on the answer generation instruction, the large language model is guided to process the answer text block to obtain the answer to the question.

[0150] In one embodiment, after determining the target data source in a preset knowledge base based on the user question, the processor is further configured to implement:

[0151] When the target data source is a preset relational database, parsing the user question to obtain key fields;

[0152] Generate a target search statement based on the key field and the search template of the relational database;

[0153] Based on the target search statement, searching in the relational database to obtain search results;

[0154] Based on a preset instruction framework, the search results are processed to obtain an answer generation instruction;

[0155] Based on the answer generation instruction, the large language model is guided to process the retrieval result to obtain the answer to the question.

[0156] In one embodiment, before determining the target data source in a preset knowledge base based on the user question, the processor is further configured to implement:

[0157] Acquire original data, wherein the original data includes text documents and relational databases;

[0158] Processing the text document to obtain a vector database corresponding to the text document, and processing the relational database to generate a retrieval database;

[0159] The knowledge base is generated based on the retrieval database and the vector database.

[0160] In one embodiment, when the processor processes the text document to obtain the vector database corresponding to the text document, it is used to implement:

[0161] Preprocessing the text document to obtain text data;

[0162] Performing text segmentation on the text data to obtain at least one text block;

[0163] Based on a preset word embedding model, encode at least one of the text blocks to obtain at least one text block vector;

[0164] The vector database is generated based on at least one of the text block vectors.

[0165] In one embodiment, when the processor processes the relational database to generate a search database, it is used to implement:

[0166] Acquire structural information corresponding to the relational database, and generate a search template for the relational database based on the structural information;

[0167] The search database is generated based on the relational database and a search template corresponding to the relational database.

[0168] In one embodiment, when the processor searches the vector database based on the question vector to obtain the answer text block corresponding to the question vector, the processor is used to implement:

[0169] Performing similarity calculation between the question vector and each text block vector in the vector database to obtain the similarity between the question vector and each text block vector in the vector database;

[0170] Based on the similarity, determining a target vector among the text block vectors in the vector database;

[0171] Based on the mapping relationship between the target vector and the text block, the text block corresponding to the target vector is obtained as the answer text block.

[0172] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any one of the product retrieval methods based on a large language model provided in the embodiments of the present application.

[0173] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0174] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A product retrieval method based on a large language model, characterized in that: include: Obtaining a user question, and based on the user question, determining a target data source in a preset knowledge base; When the target data source is a preset vector database, converting the user question into a question vector; Searching the vector database based on the question vector to obtain an answer text block corresponding to the question vector; Based on the preset large language model and the answer text block, an answer to the question corresponding to the user question is obtained as a product retrieval result.

2. The product retrieval method based on a large language model according to claim 1, characterized in that: The obtaining of an answer to the user's question based on the preset large language model and the answer text block includes: Based on a preset instruction framework, the answer text block is processed to obtain an answer generation instruction; Based on the answer generation instruction, the large language model is guided to process the answer text block to obtain the answer to the question.

3. The product retrieval method based on a large language model according to claim 1, characterized in that: After determining the target data source in a preset knowledge base based on the user question, the method further includes: When the target data source is a preset relational database, parsing the user question to obtain key fields; Generate a target search statement based on the key field and the search template of the relational database; Based on the target search statement, searching in the relational database to obtain search results; Based on a preset instruction framework, the search results are processed to obtain an answer generation instruction; Based on the answer generation instruction, the large language model is guided to process the retrieval result to obtain the answer to the question.

4. The product retrieval method based on a large language model according to claim 1, characterized in that: Before determining the target data source in a preset knowledge base based on the user question, the method further includes: Acquire original data, wherein the original data includes text documents and relational databases; Processing the text document to obtain a vector database corresponding to the text document, and processing the relational database to generate a retrieval database; The knowledge base is generated based on the retrieval database and the vector database.

5. The product retrieval method based on a large language model according to claim 4, characterized in that: The processing of the text document to obtain a vector database corresponding to the text document includes: Preprocessing the text document to obtain text data; Performing text segmentation on the text data to obtain at least one text block; Based on a preset word embedding model, encode at least one of the text blocks to obtain at least one text block vector; The vector database is generated based on at least one of the text block vectors.

6. The product retrieval method based on a large language model according to claim 4, characterized in that: The processing of the relational database to generate a retrieval database includes: Acquire structural information corresponding to the relational database, and generate a search template for the relational database based on the structural information; The search database is generated based on the relational database and a search template corresponding to the relational database.

7. The product retrieval method based on a large language model according to any one of claims 1 to 6, characterized in that: The step of searching the vector database based on the question vector to obtain an answer text block corresponding to the question vector includes: Performing similarity calculation between the question vector and each text block vector in the vector database to obtain the similarity between the question vector and each text block vector in the vector database; Based on the similarity, determining a target vector among the text block vectors in the vector database; Based on the mapping relationship between the target vector and the text block, the text block corresponding to the target vector is obtained as the answer text block.

8. A product search device based on a large language model, characterized in that: include: A data source determination module, used to obtain a user question and determine a target data source in a preset knowledge base based on the user question; A question vector conversion module, used for converting the user question into a question vector when the target data source is a preset vector database; A text block obtaining module, used to search the vector database based on the question vector to obtain an answer text block corresponding to the question vector; The question answer obtaining module is used to obtain the question answer corresponding to the user question as the product retrieval result based on the preset large language model and the answer text block.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the product retrieval method based on a large language model as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the product retrieval method based on a large language model according to any one of claims 1 to 7.

Images