Document query method, device and equipment based on LLM
By abbreviating the first document and rewriting the query statement, and optimizing document query using the Large Language Model (LLM), the problem of insufficient accuracy of query results in the existing technology is solved, and more efficient information acquisition is achieved.
Patent Information
- Application Number
- CN202411045086.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-12
AI Technical Summary
In the prior art, in document query, especially in legal case query, the accuracy of the query results is insufficient, making it difficult to effectively obtain relevant information.
The document query method based on the large language model (LLM) is adopted to abbreviate the first document, generate abbreviated text, and rewrite the query statements using the abbreviated text, thereby adjusting and optimizing the query results to improve the query accuracy.
It significantly improves the accuracy of document query, ensures that users can obtain the required information more accurately, and improves the relevance and accuracy of query results.
Smart Images

Figure CN118939760A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the technical fields of large language models, intelligent retrieval, etc., and in particular to a document query method, device and equipment based on LLM. Background Art
[0002] Large Language Model (LLM) has been a hot topic in the field of artificial intelligence in recent years. A large language model is a pre-trained language model that has learned rich language knowledge and world knowledge through pre-training on massive text data, thus achieving amazing results in various natural language processing (NLP), image generation and other tasks. Summary of the invention
[0003] The present disclosure provides a document query method, apparatus and device based on LLM.
[0004] According to one aspect of the present disclosure, a document query method based on LLM is provided, comprising: obtaining a first ranked document based on an original query statement and pre-stored candidate documents; abbreviating the first document in the first ranked document using LLM to obtain an abbreviated text; obtaining a rewritten query statement based on the abbreviated text and the original query statement; obtaining a second ranked document based on the rewritten query statement and the candidate documents; obtaining an original query document corresponding to the original query statement based on the first ranked document and the second ranked document; and obtaining a target query document based on the original query document.
[0005] According to another aspect of the present disclosure, a document query device based on LLM is provided, including: a first acquisition module, used to acquire a first ranked document according to an original query statement and pre-stored candidate documents; a second acquisition module, used to abbreviate the first document in the first ranked document by using LLM to obtain an abbreviated text; a third acquisition module, used to acquire a rewritten query statement according to the abbreviated text and the original query statement; a fourth acquisition module, used to acquire a second ranked document according to the rewritten query statement and the candidate documents; a fifth acquisition module, used to acquire an original query document corresponding to the original query statement according to the first ranked document and the second ranked document; and a sixth acquisition module, used to acquire a target query document according to the original query document.
[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any method as described in any of the above aspects.
[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods according to any one of the above aspects.
[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the methods described in any one of the above aspects.
[0009] The present disclosure can improve the accuracy of document search.
[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0012] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;
[0013] Figure 2 It is a schematic diagram of an application scenario for implementing the LLM-based document query method provided in an embodiment of the present disclosure;
[0014] Figure 3 is a schematic diagram according to a second embodiment of the present disclosure;
[0015] Figure 4 is a schematic diagram according to a third embodiment of the present disclosure;
[0016] Figure 5 is a schematic diagram according to a fourth embodiment of the present disclosure;
[0017] Figure 6 It is a schematic diagram of an electronic device used to implement the LLM-based document query method of an embodiment of the present disclosure. DETAILED DESCRIPTION
[0018] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0019] In document query scenarios, such as searching for legal cases, how to improve the accuracy of query results is a problem that needs to be solved.
[0020] In related technologies, query results are usually obtained based on document tags, but their accuracy is insufficient.
[0021] In order to improve the accuracy of document search, the present disclosure provides the following embodiments.
[0022] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure. This embodiment provides a document query method based on LLM, the method comprising:
[0023] 101. Obtain a first ranked document based on an original query statement and pre-stored candidate documents.
[0024] 102. Use LLM to abbreviate the first document in the first sorted documents to obtain an abbreviated text.
[0025] 103. Obtain a rewritten query statement based on the abbreviated text and the original query statement.
[0026] 104. Obtain a second ranked document based on the rewritten query statement and the candidate document.
[0027] 105. Based on the first ranked document and the second ranked document, obtain an original query document corresponding to the original query statement.
[0028] 106. Obtain a target query document based on the original query document.
[0029] The original query statement refers to the query statement input by the user. For example, in the legal case query scenario, the original query statement is text data that expresses the event situation and appeal situation described by the user.
[0030] The candidate document is a document pre-stored in the database for user query. Based on the above scenario, the candidate document can specifically be an existing legal case.
[0031] The document query process usually includes a recall process and a sorting process. The recall process refers to obtaining candidate documents that match the query, and the sorting process refers to sorting the recalled candidate documents, usually based on the similarity between the query and the candidate documents.
[0032] Based on the original query statement, the candidate documents recalled and ranked are called first ranked documents. The first ranked documents are a preset number, such as 5, and the first ranked documents are represented as top1 to top5.
[0033] The first document in the first ranked document is the above-mentioned top1 document, that is, the first ranked document, and the first document has the highest similarity with the query.
[0034] The document content is usually relatively large. For example, in a legal query scenario, the document is a legal case, and the content of the legal case is relatively large. In order to obtain effective information, the first document is abbreviated to obtain an abbreviated text, which contains key information of the legal case.
[0035] Since LLM has strong understanding and summarizing capabilities, using LLM to abbreviate the first document can improve the accuracy of the abbreviated text.
[0036] A rewrite query statement refers to a statement obtained by modifying an original query statement using an abbreviated text. Specifically, the original query statement and the abbreviated text may be concatenated to obtain a rewrite query statement.
[0037] After obtaining the rewritten query statement, the candidate documents are recalled and ranked similarly to the original query statement, and the ranked documents obtained based on the rewritten query statement are called second ranked documents. The number of second ranked documents is also a preset number, such as the above 5.
[0038] The first document (top1) in the first ranking document is usually more accurate. The non-first documents (such as top2 to top5) in the first ranking document can be adjusted based on the second ranking document, and the top1 document and the adjusted top2 to top5 form the target ranking document.
[0039] The original query document refers to the query result corresponding to the original query statement.
[0040] After obtaining the target ranked document, the target ranked document may be used as the original query document, or the target ranked document may be re-ranked and the re-ranked document may be used as the original query document.
[0041] The target query document refers to the query result finally obtained. After obtaining the target query document, it can be displayed to the user as the final query result.
[0042] Specifically, the original query document can be used as the target query document. Or,
[0043] The original query document may also be processed to obtain a target query document.
[0044] Specifically, it may include: reordering the original query statement to obtain a reordered query statement, processing the reordered query statement similarly to the original query statement to obtain a reordered query document corresponding to the reordered query statement, and then using the intersection of the original query document and the reordered query document as the target query document.
[0045] For example, the original query documents include: a first document, a second document, and a third document; the rearranged query documents include: a first document, a second document, and a fourth document, and the target query statement includes: a first document and a second document.
[0046] In this embodiment, LLM is used to abbreviate the first document in the first ranked document to obtain an abbreviated text, a rewritten query statement is obtained based on the abbreviated text and the original query statement, the original query document is obtained based on the first ranked document corresponding to the original query statement and the second ranked document corresponding to the rewritten query statement, and then the target query document is obtained. The excellent performance of LLM can be used to improve the accuracy of document query.
[0047] In order to better understand the embodiments of the present disclosure, application scenarios to which the embodiments of the present disclosure can be applied are described.
[0048] Figure 2 It is a schematic diagram of an application scenario for implementing the LLM-based document query method provided by the embodiment of the present disclosure. The scenario includes: a user terminal 201 and a server 202, and the user terminal 201 may include: a personal computer (PC), a mobile device (such as a mobile phone), a tablet computer, a laptop computer, a smart wearable device, etc. The server 202 may be a cloud server or a local server, and the user terminal 201 and the server 202 may communicate using a communication network, and the communication network may include, for example, a wired network and / or a wireless network.
[0049] A document query application (Application, APP) can be deployed on the user terminal 201, and a server corresponding to the APP can be deployed on the server 202. The server can be called a data query system. The user inputs an original query statement (query) to the data query system through the query APP. After receiving the original query statement, the data query system searches the pre-stored candidate documents to obtain the target query document, and returns the target query document to the query APP, which displays it to the user.
[0050] In order to improve the accuracy of the query results, this embodiment calls LLM during the query process, uses LLM to abbreviate the first document in the first ranked document to obtain an abbreviated text, uses the abbreviated text to rewrite the original query statement to obtain a rewritten query statement, obtains the original query document based on the first ranked document corresponding to the original query statement and the second ranked document corresponding to the rewritten query statement, and obtains the target query document based on the original query document.
[0051] In combination with the above application scenarios, the present disclosure also provides the following embodiments.
[0052] Figure 3 is a schematic diagram according to a second embodiment of the present disclosure. This embodiment provides a document query method based on LLM, the method comprising:
[0053] 301. Obtain a first ranked document based on an original query statement and pre-stored candidate documents.
[0054] The original query statement is the query statement input by the user.
[0055] After receiving the original query statement, the candidate documents are recalled and sorted, and a preset number of documents ranked first are selected to obtain a first sorted document.
[0056] Recall can be performed based on keywords, and sorting can be performed based on similarity.
[0057] Specifically, the process of obtaining the first ranked document may include: obtaining query keywords in the original query statement; matching the query keywords with document keywords of the candidate documents to obtain recalled documents; and sorting the recalled documents based on the similarity between the original query statement and the recalled documents to obtain the first ranked document.
[0058] In this embodiment, the recalled documents are obtained based on keyword matching, and the first ranked document is obtained from the recalled documents based on similarity, so that the first ranked document can be accurately obtained.
[0059] Among them, the keywords in the original query statement are called query keywords, and the keywords in the candidate document are called document keywords. There is at least one query keyword and document keyword. When the query keyword and the document keyword of a candidate document have an intersection, the candidate document is used as a recalled document.
[0060] The query keywords can be obtained based on an offline generated vocabulary or through online extraction, and both the vocabulary and online extraction can be implemented based on LLM.
[0061] For obtaining query keywords based on the vocabulary: the original query statement is matched with the vocabulary pre-generated by LLM to obtain the query keywords. Specifically, the keywords contained in both the original query statement and the vocabulary are used as query keywords. For example, if the vocabulary contains keyword A, if the original query statement contains keyword A, then keyword A is the query keyword.
[0062] The vocabulary is pre-generated by LLM. Specifically, query samples can be collected, and the query samples and preset first prompt information can be input into LLM. LLM extracts keywords in the query samples based on the first prompt information and adds the extracted keywords to the vocabulary.
[0063] In the legal case query scenario, the query keywords may include: fact keywords and / or appeal keywords. Fact keywords are keywords in the event description data, such as xx dispute, etc. Appeal keywords are keywords in the appeal description data, such as compensation, etc.
[0064] LLM is processed based on prompt information. The first prompt information is used to instruct LLM to extract keywords from the text.
[0065] For example, "given the following text, find the corresponding facts and demands", based on the first prompt information and the query sample, LLM obtains the keywords in the query sample, which include keywords corresponding to facts and keywords corresponding to demands. Through a large number of query samples, a large number of keywords can be obtained, and these keywords are combined into a vocabulary.
[0066] For online extraction of query keywords: Since the number of keywords in the vocabulary is limited, if the original query statement does not contain the keywords in the vocabulary, LLM can be used to perform online keyword extraction on the original query statement to obtain the query keywords.
[0067] Specifically, based on the original query statement and preset first prompt information, first input information is generated, and the first input information is input into the LLM to obtain the query keyword output by the LLM.
[0068] The first prompt information is, for example, the above-mentioned "given the following text, find the corresponding facts and demands". The first prompt information and the original query statement are input into the LLM, and the LLM can obtain the query keywords in the original query statement based on the first prompt information.
[0069] In this embodiment, LLM is used to generate a vocabulary, and query keywords are obtained based on the vocabulary, or query keywords are extracted online based on LLM, and the understanding and summarization capabilities of LLM can be used to improve the accuracy of query keywords. In addition, query keywords are obtained based on the vocabulary, and since the vocabulary is generated in advance, the processing speed can be improved.
[0070] When calculating the similarity, considering that the recalled documents are usually long, the recalled documents can be divided into smaller text units, and the similarity between the original query statement and the target text unit is calculated as the similarity between the original query statement and the recalled document where the target text unit is located.
[0071] The text unit can be divided based on a sliding window of a fixed length, that is, the text within a window of a set length is regarded as a text unit; or it can be divided based on the sentence level, that is, a sentence is regarded as a text unit.
[0072] The target text unit recalls the text unit in the document that is related to the original query sentence. Specifically, according to the correlation calculation method between two texts, the text unit with the highest correlation can be used as the target text unit.
[0073] After obtaining the original query sentence and the target text unit, the embedding model can be used to extract the corresponding embedding vectors respectively, and the similarity between the two is calculated based on the embedding vector corresponding to the original query sentence and the embedding vector corresponding to the target text unit. The embedding network can specifically use the BGE-M3 model. The BGE-M3 model is an embedding model that performs well in multi-linguality, multi-functionality, and multi-granularity.
[0074] 302. Use LLM to abbreviate the first document in the first sorted documents to obtain an abbreviated text.
[0075] The first ranked documents are a preset number of recalled documents ranked first, such as represented by top1 to topk, where k is a preset number.
[0076] The first document is the document that comes first in the ranking, such as the top1 document mentioned above.
[0077] Since the first document has more content, in order to obtain effective information, the first document can be abbreviated to obtain an abbreviated text.
[0078] Specifically, second input information may be generated based on the first document, preset second prompt information and output instruction information, and the second input information may be input into the LLM to obtain the abbreviated text output by the LLM.
[0079] The second prompt information is used to instruct LLM to extract key information, such as "The following text is a legal case. Understand the case and describe the characteristics of this case with five key information. The text content is as follows:".
[0080] The output instruction information is used to indicate the conditions that the LLM output content needs to meet, such as "output requirements: do not miss some key points of facts, and the overall output characters are within two hundred characters."
[0081] In this way, the second prompt information, the first document and the output instruction information are input into the LLM as input information to obtain the abbreviated text. The summarizing ability of the LLM can be used to obtain the abbreviated text accurately and efficiently.
[0082] 303. Concatenate the original query statement and the abbreviated text to obtain the rewritten query statement.
[0083] That is, the rewritten query statement includes the original query statement and the abbreviated text, so that the rewritten query statement can contain more information and improve the query accuracy.
[0084] 304. Obtain a second ranked document based on the rewritten query statement and the candidate document.
[0085] For the rewritten query statement, similar to the original query statement, recall and sort processing are performed to obtain a second sorted document.
[0086] 305. Based on the first ranked document and the second ranked document, obtain an original query document corresponding to the original query statement.
[0087] Generally speaking, the first document in the first ranked document has a higher accuracy, that is, the top 1 document is relatively accurate. Therefore, the top 1 document can be retained.
[0088] The non-first-ranked documents (such as top2 to topk documents) in the first ranked document can be adjusted using the second ranked document to obtain adjusted non-first-ranked documents, such as top2' to topk' documents.
[0089] Specifically, according to the order of similarity from high to low, (k-1) non-repeated documents can be selected from the non-first documents in the second ranked document and the first ranked document as the adjusted non-first documents. Then, the top1 document and top2'~topk' documents are used as target ranked documents.
[0090] In this embodiment, by adjusting the non-first-ranked documents in the first ranked document and acquiring the target ranked document based on the first-ranked document and the adjusted non-first-ranked documents, the accuracy of the target ranked document can be improved, thereby improving the accuracy of document query.
[0091] After obtaining the target ranked document, the target ranked document can be used as the original query document.
[0092] Alternatively, in some cases, the target ranked documents may be re-ranked and the re-ranked documents may be used as the original query documents, for example, re-ranking may be performed based on information such as court level and region.
[0093] In this embodiment, the document search accuracy can be further improved by re-ranking the target ranked documents to obtain the original query document.
[0094] 306. Use the original query document as a target query document.
[0095] That is, the original query document is used as the final query result, and then the original query document can be displayed to the user.
[0096] In this embodiment, by taking the original query document as the target query document, the target query document can be obtained efficiently.
[0097] Figure 4 is a schematic diagram according to a third embodiment of the present disclosure. This embodiment provides a document query method based on LLM, the method comprising:
[0098] 401. Obtain a first ranked document based on the original query statement and pre-stored candidate documents.
[0099] 402. Use LLM to abbreviate the first document in the first sorted documents to obtain an abbreviated text.
[0100] 403. Concatenate the original query statement and the abbreviated text to obtain the rewritten query statement.
[0101] 404. Obtain a second ranked document based on the rewritten query statement and the candidate document.
[0102] 405. Based on the first ranked document and the second ranked document, obtain an original query document corresponding to the original query statement.
[0103] The specific contents of 401 to 405 can be referred to 301 to 305 and will not be repeated here.
[0104] 406. Reorder the original query statements to obtain reordered query statements.
[0105] 407. Obtain a rearrangement query document corresponding to the rearrangement query statement.
[0106] 408. Use the intersection of the original query document and the rearranged query document as a target query document.
[0107] The rearranged query statement is obtained by adjusting the word order of the original query statement. Generally speaking, after the word order of a sentence is adjusted, its semantics usually does not change, and the corresponding query documents should also be consistent. Therefore, rearranging the query statement can play a role in semantic reinforcement, and then taking the above intersection as the target query document can improve the accuracy of the target query document.
[0108] Reordering the original query statement may include: removing punctuation marks in the original query statement to obtain a deleted statement; semantically segmenting the deleted statement to obtain multiple short sentences; and adjusting the order of the multiple short sentences to obtain a rearranged query statement. For example, the original query statement includes short sentence X and short sentence Y, and the order is X first and then Y, which is expressed as "X, Y". After reordering, the rearranged query statement is expressed as "Y, X".
[0109] After obtaining the rearranged query statement, the process is similar to the original query statement to obtain the re-query document.
[0110] After obtaining the original query document and the rearranged query document, the intersection of the two is used as the target query document.
[0111] For example, the original query documents include: a first document, a second document, and a third document; the rearranged query documents include: a first document, a second document, and a fourth document, and the target query statement includes: a first document and a second document.
[0112] In this embodiment, the original query statements are reordered to obtain rearranged query statements, rearranged query documents are obtained based on the rearranged query statements, and the intersection of the original query document and the rearranged query document is used as the target query document, thereby improving the accuracy of the target query document.
[0113] Figure 5 is a schematic diagram according to the fourth embodiment of the present disclosure. This embodiment provides a document query device based on LLM, such as Figure 5 As shown, the device 500 includes: a first acquisition module 501 , a second acquisition module 502 , a third acquisition module 503 , a fourth acquisition module 504 , a fifth acquisition module 505 and a sixth acquisition module 506 .
[0114] The first acquisition module is used to acquire a first ranked document according to an original query statement and pre-stored candidate documents; the second acquisition module is used to abbreviate the first document in the first ranked document by using LLM to obtain an abbreviated text; the third acquisition module is used to acquire a rewritten query statement according to the abbreviated text and the original query statement; the fourth acquisition module is used to acquire a second ranked document according to the rewritten query statement and the candidate documents; the fifth acquisition module is used to acquire an original query document corresponding to the original query statement according to the first ranked document and the second ranked document; and the sixth acquisition module is used to acquire a target query document according to the original query document.
[0115] In this embodiment, LLM is used to abbreviate the first document in the first ranked document to obtain an abbreviated text, a rewritten query statement is obtained based on the abbreviated text and the original query statement, the original query document is obtained based on the first ranked document corresponding to the original query statement and the second ranked document corresponding to the rewritten query statement, and then the target query document is obtained. The excellent performance of LLM can be used to improve the accuracy of document query.
[0116] In some embodiments, the first acquisition module 501 is further used to:
[0117] Obtaining query keywords in the original query statement;
[0118] Matching the query keyword with the document keyword of the candidate document to obtain a recalled document;
[0119] Based on the similarity between the original query statement and the recalled documents, the recalled documents are sorted to obtain the first sorted documents.
[0120] In this embodiment, the recalled documents are obtained based on keyword matching, and the first ranked document is obtained from the recalled documents based on similarity, so that the first ranked document can be accurately obtained.
[0121] In some embodiments, the first acquisition module 501 is further used to:
[0122] Matching the original query statement with the vocabulary pre-generated by LLM to obtain the query keyword; or,
[0123] Based on the original query statement and preset first prompt information, first input information is generated, and the first input information is input into the LLM to obtain the query keyword output by the LLM.
[0124] In this embodiment, LLM is used to generate a vocabulary, and query keywords are obtained based on the vocabulary, or query keywords are extracted online based on LLM, and the understanding and summarization capabilities of LLM can be used to improve the accuracy of query keywords. In addition, query keywords are obtained based on the vocabulary, and since the vocabulary is generated in advance, the processing speed can be improved.
[0125] In some embodiments, the second acquisition module 502 is further configured to:
[0126] Based on the first document, the preset second prompt information and the output instruction information, second input information is generated, and the second input information is input into the LLM to obtain the abbreviated text output by the LLM.
[0127] In this embodiment, the second prompt information, the first document and the output instruction information are input into the LLM as input information to obtain the abbreviated text. The summarizing ability of the LLM can be used to obtain the abbreviated text accurately and efficiently.
[0128] In some embodiments, the third acquisition module 503 is further used to:
[0129] The original query statement and the abbreviated text are concatenated to obtain the rewritten query statement.
[0130] In this embodiment, by concatenating, the rewritten query statement can contain more information, thereby improving the query accuracy.
[0131] In some embodiments, the fifth acquisition module 505 is further used to:
[0132] Based on the second ranked document, adjusting the non-first-place documents in the first ranked document to obtain adjusted non-first-place documents;
[0133] Based on the first-place document and the adjusted non-first-place documents, obtaining a target ranked document;
[0134] Based on the target sorted document, the original query document is obtained.
[0135] In this embodiment, by adjusting the non-first-ranked documents in the first ranked document and acquiring the target ranked document based on the first-ranked document and the adjusted non-first-ranked documents, the accuracy of the target ranked document can be improved, thereby improving the accuracy of document query.
[0136] In some embodiments, the fifth acquisition module 505 is further used to:
[0137] Use the target ranked document as the original query document; or,
[0138] The target ranked documents are re-ranked to obtain the original query document.
[0139] In this embodiment, the document search accuracy can be further improved by re-ranking the target ranked documents to obtain the original query document.
[0140] In some embodiments, the sixth acquisition module 506 is further configured to:
[0141] The original query document is used as the target query document.
[0142] In this embodiment, by taking the original query document as the target query document, the target query document can be obtained efficiently.
[0143] In some embodiments, it also includes:
[0144] A seventh acquisition module is used to reorder the original query statements to obtain rearranged query statements; and obtain rearranged query documents corresponding to the rearranged query statements;
[0145] The sixth acquisition module 506 is further used for:
[0146] The intersection of the original query document and the rearranged query document is used as the target query document.
[0147] In this embodiment, the original query statements are reordered to obtain rearranged query statements, rearranged query documents are obtained based on the rearranged query statements, and the intersection of the original query document and the rearranged query document is used as the target query document, thereby improving the accuracy of the target query document.
[0148] It can be understood that in the embodiments of the present disclosure, the same or similar contents in different embodiments can be referenced to each other.
[0149] It can be understood that the “first”, “second”, etc. in the embodiments of the present disclosure are only used for distinction and do not indicate the degree of importance, time sequence, etc.
[0150] It is understandable that, unless otherwise specified, the order of the steps in the process indicates that the timing relationship between these steps is not limited.
[0151] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0152] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0153] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device 600 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0154] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0155] Multiple components in the electronic device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0156] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the document query method based on LLM. For example, in some embodiments, the document query method based on LLM may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via the ROM 602 and / or the communication unit 606. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the document query method based on LLM described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the LLM-based document query method in any other appropriate manner (eg, by means of firmware).
[0157] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0158] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable document query device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.
[0159] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0160] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0161] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0162] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.
[0163] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0164] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A document query method based on LLM, include: Based on the original query statement and the pre-stored candidate documents, obtain the first ranked document; Using LLM, abbreviating the first document in the first ranked documents to obtain an abbreviated text; Based on the abbreviated text and the original query statement, obtaining a rewritten query statement; Based on the rewritten query statement and the candidate document, obtaining a second ranked document; Based on the first ranked document and the second ranked document, obtaining an original query document corresponding to the original query statement; Based on the original query document, a target query document is obtained.
2. The method according to claim 1, in, The step of obtaining a first ranked document based on the original query statement and the pre-stored candidate documents includes: Obtaining query keywords in the original query statement; Matching the query keyword with the document keyword of the candidate document to obtain a recalled document; Based on the similarity between the original query statement and the recalled documents, the recalled documents are sorted to obtain the first sorted documents.
3. The method according to claim 2, in, The obtaining of the query keywords in the original query statement includes: Matching the original query statement with the vocabulary pre-generated by LLM to obtain the query keyword; or, Based on the original query statement and preset first prompt information, first input information is generated, and the first input information is input into the LLM to obtain the query keyword output by the LLM.
4. The method according to claim 1, in, The adopting LLM to abbreviate the first document in the first sorted documents to obtain an abbreviated text includes: Based on the first document, the preset second prompt information and the output instruction information, second input information is generated, and the second input information is input into the LLM to obtain the abbreviated text output by the LLM.
5. The method according to claim 1, in, The obtaining of a rewritten query statement based on the abbreviated text and the original query statement includes: The original query statement and the abbreviated text are concatenated to obtain the rewritten query statement.
6. The method according to claim 1, in, The acquiring, based on the first ranked document and the second ranked document, the original query document corresponding to the original query statement includes: Based on the second ranked document, adjusting the non-first-place documents in the first ranked document to obtain adjusted non-first-place documents; Based on the first-place document and the adjusted non-first-place documents, obtaining a target ranked document; Based on the target sorted document, the original query document is obtained.
7. The method according to claim 6, in, The step of obtaining the original query document based on the target sorted document includes: Use the target ranked document as the original query document; or, The target ranked documents are re-ranked to obtain the original query document.
8. The method according to claim 1, in, The step of acquiring a target query document based on the original query document includes: The original query document is used as the target query document.
9. The method according to claim 1, further comprising: include: Reordering the original query statements to obtain reordered query statements; Obtaining a rearrangement query document corresponding to the rearrangement query statement; Accordingly, obtaining a target query document based on the original query document includes: The intersection of the original query document and the rearranged query document is used as the target query document.
10. A document query device based on LLM, include: A first acquisition module, used to acquire a first ranked document according to an original query statement and pre-stored candidate documents; A second acquisition module is used to abbreviate the first document in the first sorted documents by using LLM to obtain an abbreviated text; A third acquisition module, configured to acquire a rewritten query statement according to the abbreviated text and the original query statement; A fourth acquisition module, configured to acquire a second ranked document according to the rewritten query statement and the candidate document; A fifth acquisition module, configured to acquire an original query document corresponding to the original query statement according to the first ranked document and the second ranked document; The sixth acquisition module is used to acquire a target query document according to the original query document.
11. The device according to claim 10, in, The first acquisition module is further used to: acquire query keywords in the original query statement; match the query keywords with document keywords of the candidate documents to obtain recalled documents; sort the recalled documents based on the similarity between the original query statement and the recalled documents to obtain the first sorted documents; and / or, The second acquisition module is further used to: generate second input information based on the first document, the preset second prompt information and the output instruction information, and input the second input information into the LLM to obtain the abbreviation text output by the LLM; and / or, The third acquisition module is further used to: concatenate the original query statement and the abbreviated text to obtain the rewritten query statement; and / or, The fifth acquisition module is further used to: adjust the non-first-place documents in the first-place document based on the second-placed document to obtain an adjusted non-first-place document; obtain a target-placed document based on the first-place document and the adjusted non-first-place document; and obtain the original query document based on the target-placed document; and / or, The sixth acquisition module is further used to: use the original query document as the target query document; and / or, The device also includes: a seventh acquisition module, used to reorder the original query statement to obtain a rearranged query statement; and obtain a rearranged query document corresponding to the rearranged query statement; and the sixth acquisition module is further used to: use the intersection of the original query document and the rearranged query document as the target query document.
12. The device according to claim 11, in, The first acquisition module is further used to: match the original query statement with the word list pre-generated by the LLM to obtain the query keyword; or generate first input information based on the original query statement and preset first prompt information, and input the first input information into the LLM to obtain the query keyword output by the LLM; and / or, The fifth acquisition module is further used to: use the target ranked document as the original query document; or rerank the target ranked document to obtain the original query document.
13. An electronic device, include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.
14. A non-transitory computer-readable storage medium storing computer instructions, in, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-9.
15. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Retrieval method, device and equipment for computer field documents in question-answering system
CN116595122A
Session type retrieval method for rewriting query through thinking chain strategy
CN118377883A
Identification of acronym expansions
US20140344263A1
Semantic understanding method, electronic device, and storage medium
US20230089268A1
