Document reordering method, device and electronic device

Abstract

The present application discloses a document re-ranking method, device and electronic device, belonging to the field of artificial intelligence technology. The method includes: in response to received query information, obtaining at least one candidate document matching the query information, wherein the candidate document includes at least one candidate sentence; dividing the query information into multiple query elements, and for each query element, based on the similarity between each candidate sentence in at least one candidate document and the query element, obtaining k first candidate sentences whose similarity with the query element is greater than a preset value among all candidate sentences in at least one candidate document; based on the number of first candidate sentences included in each candidate document, obtaining a correlation index between each candidate document and the query information; and re-ranking each candidate document according to the correlation index of each candidate document. By the above method, not only the computational complexity of document re-ranking can be reduced, but also the efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a document reordering method, device, and electronic device. Background Art

[0002] Re-ranking technology plays a vital role in the entire process of Retrieval Augmented Generation (RAG).

[0003] In related technologies, deep learning-based methods can be used to reorder documents, but this type of reordering method has high computational complexity and low efficiency. Summary of the Invention

[0004] The present application provides a document reordering method, device and electronic device, which can at least solve the problems of high computational complexity and low efficiency of document reordering methods in related technologies.

[0005] In order to solve the above technical problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a document reranking method, the method comprising: in response to received query information, obtaining at least one candidate document matching the query information, wherein the candidate document includes at least one candidate sentence; dividing the query information into multiple query elements, and for each query element, based on the similarity between each candidate sentence in the at least one candidate document and the query element, obtaining k first candidate sentences whose similarity to the query element is greater than a preset value among all candidate sentences in the at least one candidate document, wherein k is an integer greater than 0; based on the number of the first candidate sentences included in each candidate document, obtaining a relevance index between each candidate document and the query information; and reranking each candidate document according to the relevance index of each candidate document.

[0007] In a second aspect, a document reordering device is provided, which includes: a first acquisition module for acquiring, in response to received query information, at least one candidate document matching the query information, wherein the candidate document includes at least one candidate sentence; a second acquisition module for dividing the query information into multiple query elements, and for each query element, based on the similarity between each candidate sentence in the at least one candidate document and the query element, acquiring, from all candidate sentences in the at least one candidate document, k first candidate sentences whose similarity to the query element is greater than a preset value, wherein k is an integer greater than 0; a third acquisition module for acquiring, based on the number of the first candidate sentences included in each candidate document, a relevance index between each candidate document and the query information; and a reordering module for reordering each candidate document according to the relevance index of each candidate document.

[0008] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect above are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect above are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer performs the steps of the method described in the first aspect above.

[0011] The technical solution provided by this application may have the following beneficial effects:

[0012] In an embodiment of the present application, in response to received query information, at least one candidate document matching the query information can be obtained. The query information is then divided into multiple query elements. For each query element, based on the similarity between each candidate sentence in the at least one candidate document and the query element, k first candidate sentences in the at least one candidate document whose similarity to the query element is greater than a preset value are obtained. A relevance index between each candidate document and the query information can then be obtained based on the number of first candidate sentences included in each candidate document. The candidate documents are then reordered based on their relevance indexes. This approach not only reduces the computational complexity of document reordering but also improves efficiency.

[0013] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0015] Figure 1 A schematic diagram of a process for reordering documents provided in an embodiment of the present application is shown;

[0016] Figure 2 A schematic diagram of a document reordering method provided by an embodiment of the present application is shown;

[0017] Figure 3 A schematic structural diagram of a document reordering device provided in an embodiment of the present application is shown;

[0018] Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown;

[0019] Figure 5 A schematic structural diagram of another electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0021] Re-ranking technology plays a crucial role in the entire Retrieval Augmented Generation (RAG) process. In the most primitive RAG methods, a large amount of context may be retrieved, but not all of it is relevant to the question. Re-ranking technology rearranges the order of documents and filters them, eliminating irrelevant or unimportant documents and placing relevant documents at the top, thereby improving the accuracy of the RAG system.

[0022] Reranking algorithms in related technologies can generally be divided into two categories: one is based on deep learning, such as neural networks, decision trees, and the Bidirectional Encoder Representations from Transformers (BERT) model. The other is based on relevance models. These methods analyze the interaction characteristics between the user's question and candidate documents, evaluate their relevance, and rank them based on the relevance score. For example, BM25 (Best Matching 25), an algorithm used in information retrieval and text mining, is an example. Deep learning-based methods generally use a cross encoder to combine semantic information of the document and query for scoring and ranking. The typical BERT algorithm and other transformer-based methods (which use attention mechanisms to speed up model training) suffer from a limitation of 512 tokens (typically the smallest unit of input text), making them unable to cover the entire text of longer documents. Furthermore, these algorithms are complex and computationally inefficient, especially when processing large datasets, which consumes significant computing resources and time. The BM25 algorithm uses term frequency (TF) and inverse document frequency (IDF) and also incorporates document length information to calculate the relevance between documents (D) and queries (Q). However, the BM25 algorithm can produce low scores for long documents and short queries. Furthermore, it fails to consider the relevance between terms, which can lead to inaccurate scores.

[0023] The re-ranking algorithms in related technologies have the following main shortcomings: (1) Although deep learning-based methods, such as neural networks, decision trees, and BERT, can effectively combine the semantic information of documents and queries for scoring and ranking, they are often limited by the length of the text (such as the 512 tokens limit of transformer-based methods such as BERT) and cannot fully cover long documents. Moreover, these algorithms have high computational complexity and low efficiency, and consume a lot of computing resources and time, especially when processing large-scale data.

[0024] (2) Methods based on relevance models, such as BM25, can evaluate relevance by analyzing the interaction features between queries and documents. However, they may have low scores when dealing with long documents and short queries. At the same time, they ignore the correlation between terms, which may lead to inaccurate scores.

[0025] Figure 1A schematic diagram of a document reordering method provided by an exemplary embodiment of the present application is shown. The method can be executed by an electronic device, which can be a terminal such as a mobile phone or a computer. Figure 1 As shown, the method mainly includes the following steps:

[0026] S101: In response to received query information, obtain at least one candidate document matching the query information.

[0027] The candidate document includes at least one candidate sentence.

[0028] In an embodiment of the present application, at least one candidate document that matches the query information can be obtained in response to the received query information. In actual applications, the query information can be a query document or a query statement, which is not specifically limited in the embodiment of the present application. At least one candidate document is a retrieval result that has been retrieved and matches the query information. In actual applications, the number of candidate documents can be n, where n is an integer greater than 1. The specific candidate document retrieval process can adopt a Boolean model (a simple retrieval model based on set theory and Boolean algebra) or a vector space model (VSM), which is not specifically limited in the embodiment of the present application. The document reranking method in the embodiment of the present application is mainly for reranking the candidate documents after the initial ranking, and the main purpose is to exclude irrelevant or unimportant candidate documents and put candidate documents with high relevance in front, thereby improving the accuracy of the retrieval system.

[0029] In the embodiment of the present application, the query information can be used To express, at least one candidate document can be In practical applications, the candidate document can be divided into multiple candidate sentences. The specific division method is not specifically limited in this embodiment of the application, and the number of candidate sentences is not specifically limited. Denote as the set of all candidate sentences in D. Assume D, is the set of all candidate sentences in d, For a sentence in d, for In the embodiment of the present application, the method for generating the sentence vector is not specifically limited in the embodiment of the present application.

[0030] S102: Divide the query information into multiple query elements. For each query element, based on the similarity between each candidate sentence in the at least one candidate document and the query element, obtain k first candidate sentences from all candidate sentences in the at least one candidate document, whose similarity with the query element is greater than a preset value.

[0031] where k is an integer greater than 0.

[0032] In the embodiments of the present application, the query information q can be divided into multiple query elements. The number of query elements and the specific division method are not specifically limited in the embodiments of the present application. In practical applications, if the query information is in the form of a query statement, it can be divided according to specific circumstances. For example, words without actual semantics such as "de" or "a" can be discarded. If the query information is in the form of a query document, it can be divided according to specific semantics, and a complete sentence is preferably divided into one query element. In practical applications, a certain complete syntactic analysis technology can be used to assist in the division of query information, and a sentence or word with complete semantics should be avoided from being split in half. The embodiments of the present application do not make specific limitations.

[0033] In the embodiments of the present application, is used to represent all query elements. If the query information is in the form of a query document, can be the set of all query sentences in q, is a sentence in q, is the corresponding sentence vector. In practical applications, different methods may generate vectors with different dimensions and meanings. In the embodiments of the present application, the method for generating the vector of the query element can be consistent with the method for generating the vector of the candidate sentence, which is convenient for calculating their similarity.

[0034] In the embodiments of the present application, for each query element, based on the similarity between each candidate sentence in at least one candidate document and the query element, k first candidate sentences with a similarity greater than a preset value among all candidate sentences in at least one candidate document are obtained. In this way, for each query element, k first candidate sentences with a similarity greater than the preset value among all candidate sentences are obtained. That is, for each query element, k first candidate sentences with a similarity greater than the preset value can be obtained. k is less than the total number of candidate sentences, which can reduce the calculation amount, improve the calculation efficiency, and can also screen out the first candidate sentences with a similarity greater than the preset value in at least one candidate document, which can be used to further obtain the similarity between the query information and the candidate document, facilitating the re-ranking of the candidate documents. In practical applications, the method for calculating the similarity between each candidate sentence and the query element can adopt the modified cosine similarity. The embodiments of the present application do not make specific limitations.

[0035] In an optional implementation, the obtaining, based on the similarity between each candidate sentence in the at least one candidate document and the query element, k first candidate sentences having a similarity greater than a preset value to the query element from among all candidate sentences in the at least one candidate document, may include the following steps:

[0036] Step 1, calculate the adjusted cosine similarity between the query element and each candidate sentence in the at least one candidate document; in an embodiment of the present application, the adjusted cosine similarity between the query element and each candidate sentence in the at least one candidate document can be calculated. Adjusted cosine similarity (ACS) is an improved cosine similarity calculation method used in text mining, information retrieval and other fields. The deviation caused by the mean offset and the difference in document length (or vector size) can be eliminated by subtracting the mean of the document (or vector) from the value of each dimension. The larger the result, the higher the similarity. The specific calculation formula is as follows:

[0037]

[0038] Where n is the dimension of the vector, is the vector representation of the query element, is the vector representation of the candidate sentence, is a vector The value in the jth dimension, is a vector The value in the jth dimension, is a vector The mean of all dimensions, is a vector The mean of all dimensions.

[0039] Step 2: Sort all candidate sentences in the at least one candidate document in descending order of the modified cosine similarity, and obtain the first k candidate sentences that are ranked first.

[0040] In the embodiment of the present application, the modified cosine similarity can be ordered from large to small for each , sort all candidate sentences in at least one candidate document, and the sorting result can be recorded as , and then get the first k candidate sentences that are ranked first k is a parameter that can be set based on experience and is not specifically limited in this embodiment of the application. Recorded as All The set of corresponding first candidate sentences.

[0041] In practical applications, since the length of the document and the length of the query are usually very different, directly using cosine similarity may lead to inaccurate results. Although traditional cosine similarity can be used to calculate the similarity between the query element and the candidate sentence, when the length difference of the text is large, the longer text may get a higher similarity score because it contains more information, even if the actual semantic similarity between them is not high. The modified cosine similarity can more accurately measure the semantic similarity between texts by taking into account the difference in text length. The modified cosine similarity can not only better evaluate the true semantic importance, but also avoid the problem of low scores in long documents and short queries. Therefore, in an embodiment of the present application, the modified cosine similarity can be used to calculate the similarity between the query element and the candidate sentence. By modifying the cosine similarity, the influence of the text length difference can be eliminated and the accuracy of the retrieval can be improved.

[0042] In the embodiments of this application, modified cosine similarity is used to calculate the similarity between the query element and the candidate sentence. Compared with the deep learning-based methods in related technologies, the calculation method is simpler and the computational complexity is improved, making the calculation faster. This is because modified cosine similarity mainly relies on basic linear algebra operations, without the need for time-consuming neural network training or complex model reasoning.

[0043] S103: Based on the number of the first candidate sentences included in each candidate document, obtain a relevance index between each candidate document and the query information.

[0044] In an embodiment of the present application, a relevance index between each candidate document and the query information can be obtained based on the number of first candidate sentences included in each candidate document. Generally speaking, the more first candidate sentences a candidate document includes, the higher the relevance of the candidate document to the query information. The relevance index can be used to indicate the relevance between each candidate document and the query information, effectively characterizing the degree of match between the candidate document and the query information. By obtaining the relevance index, the accuracy of the reranking can be improved.

[0045] In an optional implementation, the relevance index includes: a query relevance index and a document relevance index; in an embodiment of the present application, the relevance index can be further divided into a query relevance index and a document relevance index to obtain the degree of relevance between the candidate document and the query information in a more detailed and comprehensive manner.

[0046] In the above optional implementation, obtaining the relevance index between each candidate document and the query information based on the number of the first candidate sentences included in each candidate document may include the following steps:

[0047] Step 1: Obtain the weight of each query element. In this embodiment of the present application, the weight of each query element can be obtained. When calculating the relevance index between the candidate documents and the query information, considering the weight of the query element can improve the accuracy of the re-ranking.

[0048] Step 2: Obtain a query relevance index for each candidate document based on the weight of each query element, the number of query sentences, and whether each candidate document includes the first candidate sentence of each query element. In an embodiment of the present application, the query relevance index for each candidate document can be obtained based on the weight of each query element, the number of query sentences, and whether each candidate document includes the first candidate sentence of each query element. By comprehensively considering the weight of the query element, the number of query sentences, and whether each candidate document includes the first candidate sentence, the query relevance index for the candidate document obtained can more accurately evaluate the degree of match between the candidate document and the query information.

[0049] Step 3: Obtain a document relevance index for each candidate document based on the weight of each query element, the number of candidate sentences, and whether the candidate sentence of each candidate document is the first candidate sentence of each query element. In an embodiment of the present application, the document relevance index for each candidate document can be obtained based on the weight of each query element, the number of candidate sentences, and whether the candidate sentence of each candidate document is the first candidate sentence of each query element. By comprehensively considering the weight of the query element, the number of candidate sentences, and whether the candidate sentence of each candidate document is the first candidate sentence of each query element, the document relevance index of the candidate document obtained can more comprehensively evaluate the degree of relevance between the candidate document and the query information.

[0050] In the embodiment of the present application, whether calculating the query relevance index or the document relevance index, all candidate sentences of each candidate document can be traversed. The method of the embodiment of the present application can avoid the problem of the 512 token limit of the typical BERT algorithm and other transformer-based methods in the related art, and can fully cover the full text of longer documents, thereby improving the accuracy of the calculation.

[0051] In an optional implementation, for step 1 above, obtaining the weight of each query element may include:

[0052] The weight of the query element is obtained according to the number of all candidate sentences included in the at least one candidate document and the number of times the query element appears in the at least one candidate document.

[0053] In an embodiment of the present application, the weight of the query element can be obtained based on the number of all candidate sentences included in at least one candidate document and the number of times the query element appears in at least one candidate document. Optionally, the calculation formula is as follows:

[0054] in, Represents a query element The weight of represents the number of all candidate sentences included in at least one candidate document, Represents a query element The number of times a value appears in at least one candidate document. log represents the logarithm of the obtained value.

[0055] In practical applications, if a query element (word / sentence) is more common, the larger the denominator, the smaller the corresponding weight and the closer it is to 0. The reason for adding 0.5 to the denominator is to avoid the denominator being 0 (that is, all candidate documents do not contain the word / sentence). You can also choose to add other values, but try not to be too large or too small. The specific values ​​are not specifically limited in the embodiments of this application.

[0056] In an optional implementation, the query relevance index of each candidate document is obtained based on the weight of each query element, the number of query sentences, and whether each candidate document includes the first candidate sentence of each query element, including: for each candidate document, the query relevance index of the candidate document is calculated using the following formula: :

[0057]

[0058] in, Represents the query element The weight of is the set of all query elements of the query information, is the set of all candidate sentences of the candidate document d, for The set of first candidate sentences, Indicates that for each , if there is at least one candidate sentence in d Corresponding In the example, min() function returns 1, if not, it returns 0, N(q) represents the number of query elements;

[0059] or,

[0060] According to the weight of each query element, the number of candidate sentences, and whether the candidate sentence of each candidate document is the first candidate sentence of each query element, the document relevance index of each candidate document is obtained, including: for each candidate document, the document relevance index of the candidate document is calculated using the following formula :

[0061]

[0062] in, Represents the query element The weight of is the set of all query elements of the query information, is the set of all candidate sentences of the candidate document d, for The corresponding set of first candidate sentences, Represents each candidate sentence for d ,if exist Corresponding In the example, the min() function returns 1, if not, it returns 0, and N(d) represents the number of candidate sentences in the candidate document d.

[0063] In the embodiment of the present application, when calculating the query relevance QP of the candidate document d, it is possible to iteratively calculate All query elements in . For each query element , the higher its weight is, and if there is a corresponding first candidate sentence in the candidate sentences in d, then the query relevance score of d is higher.

[0064] In the embodiment of the present application, when calculating the document relevance DP of the candidate document d, it is possible to iteratively calculate All candidate sentences in , confirm whether each candidate sentence is The first candidate sentence corresponding to d. The candidate sentence in d is The more there are in , the higher the document relevance score of d.

[0065] S104: Reorder the candidate documents according to their relevance index.

[0066] In an embodiment of the present application, each candidate document can be re-ranked based on its relevance index. The higher the relevance index of a candidate document, the more relevant and matching the candidate document is to the query information, and the higher the ranking of the candidate document will be. In practical applications, re-ranking candidate documents based on their relevance index can improve retrieval accuracy.

[0067] In an optional implementation, reordering the candidate documents according to their relevance indicators may include the following steps:

[0068] Step 1: Based on the query relevance index of each candidate document and the document relevance index of the candidate document, a reranking index corresponding to each candidate document is obtained; in this embodiment of the application, based on the query relevance index of each candidate document and the document relevance index of the candidate document, a reranking index corresponding to each candidate document is obtained. This way, more accurate and comprehensive re-ranking indicators corresponding to candidate documents can be obtained.

[0069] Optionally, the calculation formula is as follows:

[0070] In practical applications, other formulas may be used for calculation, and the embodiments of the present application do not specifically limit this.

[0071] Step 2: Reorder each candidate document based on the reordering index corresponding to each candidate document. In an embodiment of the present application, each candidate document can be reordered based on the reordering index corresponding to each candidate document. In actual applications, candidate documents with large reordering indexes can be placed in front, which helps to improve the user's query experience.

[0072] In an embodiment of the present application, in response to received query information, at least one candidate document matching the query information can be obtained. The query information is then divided into multiple query elements. For each query element, based on the similarity between each candidate sentence in the at least one candidate document and the query element, k first candidate sentences in the at least one candidate document whose similarity to the query element is greater than a preset value are obtained. A relevance index between each candidate document and the query information can then be obtained based on the number of first candidate sentences included in each candidate document. The candidate documents are then reordered based on their relevance indexes. This approach not only reduces the computational complexity of document reordering but also improves efficiency.

[0073] The document reordering method proposed in the embodiment of the present application mainly consists of four steps.

[0074] The first step is to calculate the modified cosine similarity between all query elements in the query information and all candidate sentences in the candidate documents, and rank the candidate sentences in the candidate documents for each query element;

[0075] The second step is to calculate the query relevance index of the candidate document for each query element.

[0076] The third step is to calculate the document relevance index with the query element for each candidate document.

[0077] The fourth step is to obtain a reranking index for each candidate document based on the calculated query relevance index and the candidate relevance index, and rerank the documents based on the reranking index.

[0078] The document re-ranking method in the embodiment of the present application comprehensively considers information such as the length, word frequency, and weight of the document, which can effectively improve the accuracy of retrieval ranking and effectively reduce computational complexity.

[0079] Figure 2 The document reordering method proposed in the embodiment of the present application is shown in the schematic diagram of data processing. , and the initial document sort sequence , get each query element Based on the ranking of cosine similarity, the top k rankings are obtained. According to the top k rankings, the QP and DP of each candidate document can be calculated to obtain the final sorting sequence D.

[0080] In the present embodiment, in a test environment (with both long document retrieval and short query conditions), the document reranking method in the present embodiment can be compared with the BM25 model and the BERT model. The results are shown in Table 1. The method in the present embodiment outperforms the BM25 model and the BERT model in terms of precision, recall rate, and F1 score. It also significantly outperforms the BERT model in terms of time.

[0081] Table 1

[0082]

[0083] The method in this embodiment evaluates relevance from the perspectives of both query information and candidate documents, comprehensively considering the relevance and weights between terms. This reduces factors such as sentence length, irrelevant terms and sentences, and repetitiveness, thereby improving re-ranking accuracy. Furthermore, compared to deep learning algorithms, the computational formula is simpler, reducing computing resources and time, resulting in shorter computation times and higher retrieval efficiency.

[0084] The document reordering method in the embodiment of the present application significantly improves the performance of the RAG system and can be widely used in multiple fields, such as search engines, recommendation systems, intelligent question and answer, customer service robots, etc.

[0085] This method can improve user experience by providing more accurate ranking results. It has broad application prospects and can be used in various situations requiring document sorting and similarity matching. Furthermore, the document reranking method in the embodiments of this application is relatively simple and computationally efficient, making it easy to integrate into existing RAG systems, facilitating their application and promotion in the market.

[0086] The embodiments of the present application provide Figure 1 The document reordering method shown in the figure can be executed by Figure 3 The document reordering device shown in the embodiment of the present application is taken as an example to illustrate the document reordering device provided by the embodiment of the present application by performing the document reordering method.

[0087] Figure 3 The schematic diagram of the structure of the document reordering device provided by an exemplary embodiment of the present application is shown. The document reordering device can achieve the following Figure 1 In the embodiment shown, all or part of the content, the document reordering device includes: a first acquisition module 301 , a second acquisition module 302 , a third acquisition module 303 and a reordering module 304 .

[0088] In an embodiment of the present application, a first acquisition module 301 is used to obtain at least one candidate document matching the query information in response to received query information, wherein the candidate document includes at least one candidate sentence; a second acquisition module 302 is used to divide the query information into multiple query elements, and for each query element, based on the similarity between each candidate sentence in the at least one candidate document and the query element, obtain k first candidate sentences whose similarity to the query element is greater than a preset value among all candidate sentences in the at least one candidate document, wherein k is an integer greater than 0; a third acquisition module 303 is used to obtain a relevance index between each candidate document and the query information based on the number of the first candidate sentences included in each candidate document; a reordering module 304 is used to reorder each candidate document according to the relevance index of each candidate document.

[0089] In an optional implementation, the second acquisition module 302, when used to acquire, based on the similarity between each candidate sentence in the at least one candidate document and the query element, k first candidate sentences whose similarity to the query element is greater than a preset value from all candidate sentences in the at least one candidate document, is specifically configured to:

[0090] Calculating a modified cosine similarity between the query element and each candidate sentence in the at least one candidate document;

[0091] All candidate sentences in the at least one candidate document are sorted in descending order of the modified cosine similarity to obtain the first k candidate sentences that are ranked first.

[0092] In an optional implementation, the relevance index includes: a query relevance index and a document relevance index;

[0093] The third acquisition module 303, when used to acquire the relevance index between each candidate document and the query information based on the number of the first candidate sentences included in each candidate document, is specifically used to:

[0094] Obtaining the weight of each query element;

[0095] Obtaining a query relevance index of each candidate document according to the weight of each query element, the number of query sentences, and whether each candidate document includes the first candidate sentence of each query element;

[0096] A document relevance index of each candidate document is obtained according to the weight of each query element, the number of the candidate sentences, and whether the candidate sentence of each candidate document is the first candidate sentence of each query element.

[0097] In an optional implementation, when the third obtaining module 303 is used to obtain the weight of each query element, the third obtaining module 303 is specifically used to:

[0098] The weight of the query element is obtained according to the number of all candidate sentences included in the at least one candidate document and the number of times the query element appears in the at least one candidate document.

[0099] In an optional implementation, the third acquisition module 303, when used to obtain the query relevance index of each candidate document based on the weight of each query element, the number of query sentences, and whether each candidate document includes the first candidate sentence of each query element, is specifically used to:

[0100] For each candidate document, the query relevance index of the candidate document is calculated using the following formula: :

[0101]

[0102] in, Represents the query element The weight of is the set of all query elements of the query information, is the set of all candidate sentences of the candidate document d, for The set of first candidate sentences, Indicates that for each , if there is at least one candidate sentence in d Corresponding In the example, min() function returns 1, if not, it returns 0, N(q) represents the number of query elements;

[0103] or,

[0104] The third acquisition module 303 is configured to obtain the document relevance index of each candidate document according to the weight of each query element, the number of candidate sentences, and whether the candidate sentence of each candidate document is the first candidate sentence of each query element, specifically to:

[0105] For each candidate document, the document relevance index of the candidate document is calculated using the following formula: :

[0106]

[0107] in, Represents the query element The weight of is the set of all query elements of the query information, is the set of all candidate sentences of the candidate document d, for The corresponding set of first candidate sentences, Represents each candidate sentence for d ,if exist Corresponding In the example, the min() function returns 1, if not, it returns 0, and N(d) represents the number of candidate sentences in the candidate document d.

[0108] In an optional implementation, when reordering the candidate documents using the relevance index of each candidate document, the reordering module 304 is specifically configured to:

[0109] Obtaining a reranking index corresponding to each candidate document based on the query relevance index of each candidate document and the document relevance index of the candidate document;

[0110] Reorder each of the candidate documents based on the reordering index corresponding to each of the candidate documents.

[0111] The document reordering device in the embodiments of the present application can be an electronic device or a component of an electronic device, such as an integrated circuit or chip. The electronic device can be a terminal or other device other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc., and the embodiments of the present application are not specifically limited.

[0112] The document reordering device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0113] The document reordering device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented in the method embodiment are not described here.

[0114] Alternatively, as Figure 4 As shown, the embodiment of the present application further provides an electronic device 400, including a processor 401 and a memory 402, wherein the memory 402 stores a program or instruction that can be run on the processor 401, and when the program or instruction is executed by the processor 401, the above Figure 1 The various steps of the document reordering method shown in the figure can achieve the same technical effect. To avoid repetition, they will not be described here.

[0115] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0116] Figure 5The following is a block diagram of another electronic device 500 according to an exemplary embodiment of the present application. The electronic device 500 can be implemented as a smartphone, tablet computer, laptop computer, desktop computer, smartwatch, television, etc. The electronic device 500 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, etc.

[0117] Typically, the electronic device 500 includes a processor 501 and a memory 502 .

[0118] Processor 501 may include one or more processing cores, such as a quad-core processor or a penta-core processor. Processor 501 may be implemented in hardware using at least one of the following: a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), or a PLA (Programmable Logic Array). Processor 501 may also include a main processor and a coprocessor. The main processor is used to process data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 501 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing content displayed on the display screen. In some embodiments, processor 501 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0119] The memory 502 may include one or more computer-readable storage media, which may be non-transitory. The memory 502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 502 is used to store at least one instruction, which is executed by the processor 501 to implement all or part of the steps in the document reordering method shown in the method embodiment of the present application.

[0120] In some embodiments, electronic device 500 may optionally include a peripheral device interface 503 and at least one peripheral device. Processor 501, memory 502, and peripheral device interface 503 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 503 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 504, a display screen 509, a camera assembly 506, an audio circuit 507, and a power supply 508.

[0121] In some embodiments, the electronic device 500 further includes one or more sensors 509 , including but not limited to: an acceleration sensor 510 , a gyroscope sensor 511 , a pressure sensor 512 , an optical sensor 513 , and a proximity sensor 514 .

[0122] Those skilled in the art will understand that Figure 5 The structure shown in the figure does not constitute a limitation on the electronic device 500, and the electronic device 500 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0123] The embodiment of the present application also provides a computer-readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the above Figure 1 The various processes of the document reordering method shown in the figure can achieve the same technical effect. To avoid repetition, they will not be described here.

[0124] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0125] The embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the above Figure 1 The various processes of the document reordering method shown in the figure can achieve the same technical effect. To avoid repetition, they will not be described here.

[0126] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0127] The present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program includes program instructions, which, when executed by a computer, implement the above-mentioned Figure 1The steps of the document reordering method shown in the figure can achieve the same technical effect. To avoid repetition, they will not be repeated here.

[0128] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered merely as exemplary, and the true scope and spirit of the present application are indicated by the claims.

[0129] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A document reordering method, characterized in that: include: In response to the received query information, obtaining at least one candidate document matching the query information, wherein the candidate document includes at least one candidate sentence; Dividing the query information into a plurality of query elements, and for each query element, obtaining, based on similarities between each candidate sentence in the at least one candidate document and the query element, k first candidate sentences having a similarity with the query element greater than a preset value from among all candidate sentences in the at least one candidate document, where k is an integer greater than 0; Based on the number of the first candidate sentences included in each candidate document, obtaining a relevance index between each candidate document and the query information, wherein the relevance index includes: a query relevance index and a document relevance index; Reordering the candidate documents according to their relevance index; The obtaining of a relevance index between each candidate document and the query information based on the number of the first candidate sentences included in each candidate document includes: Obtaining the weight of each query element; Obtaining a query relevance index of each candidate document according to the weight of each query element, the number of candidate sentences, and whether each candidate document includes the first candidate sentence of each query element; Obtaining a document relevance index for each candidate document according to the weight of each query element, the number of candidate sentences, and whether the candidate sentence of each candidate document is the first candidate sentence of each query element; The reordering of the candidate documents according to the relevance index of each candidate document includes: Obtaining a reranking index corresponding to each candidate document based on the query relevance index of each candidate document and the document relevance index of the candidate document; Reorder each of the candidate documents based on the reordering index corresponding to each of the candidate documents.

2. The method according to claim 1, characterized in that The obtaining, based on the similarity between each candidate sentence in the at least one candidate document and the query element, k first candidate sentences having a similarity greater than a preset value to the query element from among all candidate sentences in the at least one candidate document, includes: Calculating a modified cosine similarity between the query element and each candidate sentence in the at least one candidate document; All candidate sentences in the at least one candidate document are sorted in descending order of the modified cosine similarity to obtain the first k candidate sentences that are ranked first.

3. The method according to claim 1, characterized in that Obtaining a query relevance index of each candidate document according to the weight of each query element, the number of the candidate sentences, and whether each candidate document includes the first candidate sentence of each query element includes: For each candidate document, the query relevance index of the candidate document is calculated using the following formula: : in, Represents the query element The weight of is the set of all query elements of the query information, is the set of all candidate sentences of the candidate document d, for The set of first candidate sentences, Indicates that for each , if there is at least one candidate sentence in d Corresponding In the example, min() function returns 1, if not, it returns 0, N(q) represents the number of query elements; or, Obtaining a document relevance index for each candidate document according to the weight of each query element, the number of the candidate sentences, and whether the candidate sentence of each candidate document is the first candidate sentence of each query element includes: For each candidate document, the document relevance index of the candidate document is calculated using the following formula: : in, Represents the query element The weight of is the set of all query elements of the query information, is the set of all candidate sentences of the candidate document d, for The corresponding set of first candidate sentences, Represents each candidate sentence for d ,if exist Corresponding In the example, the min() function returns 1, if not, it returns 0, and N(d) represents the number of candidate sentences in the candidate document d.

4. The method according to claim 3, characterized in that For each query element, obtaining a weight of the query element includes: The weight of the query element is obtained according to the number of all candidate sentences included in the at least one candidate document and the number of times the query element appears in the at least one candidate document.

5. A document reordering device, characterized in that: include: a first acquisition module, configured to acquire, in response to received query information, at least one candidate document matching the query information, wherein the candidate document includes at least one candidate sentence; a second acquisition module configured to divide the query information into a plurality of query elements, and for each query element, based on the similarity between each candidate sentence in the at least one candidate document and the query element, acquire k first candidate sentences from among all candidate sentences in the at least one candidate document, whose similarity with the query element is greater than a preset value, where k is an integer greater than 0; a third acquisition module, configured to acquire, based on the number of the first candidate sentences included in each candidate document, a relevance index between each candidate document and the query information, wherein the relevance index includes: a query relevance index and a document relevance index; A reordering module, configured to reorder each candidate document according to a relevance index of each candidate document; The reordering of the candidate documents according to the relevance index of each candidate document includes: Obtaining a reranking index corresponding to each candidate document based on the query relevance index of each candidate document and the document relevance index of the candidate document; Reorder each of the candidate documents based on the reordering index corresponding to each of the candidate documents.

6. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

8. A computer program product, characterized in that The computer program product comprises a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions. When the program instructions are executed by a computer, the computer is caused to perform the steps of the method according to any one of claims 1 to 4.

Images