Information retrieval method and device, equipment, storage medium and product

Abstract

The invention discloses an information retrieval method and device, equipment, a storage medium and a product. The method comprises the steps that a retrieval instruction for retrieving a to-be-retrieved text is received, the retrieval instruction is used for searching a corresponding retrieval result in a document set through the to-be-retrieved text, the document set comprises multiple layers of documents, and the document granularity of each layer of documents is sequentially reduced from top to bottom; sequentially carrying out recursive retrieval in the document set from bottom to top by utilizing the to-be-retrieved text to obtain a first result corresponding to each layer of document; and processing the first result to obtain the retrieval result. According to the embodiment, the comprehensiveness of retrieval results of information retrieval can be improved.

Description

Technical Field

[0001] The present application belongs to the technical field of computer text processing, and in particular, relates to an information retrieval method, device, equipment, storage medium and product. Background Art

[0002] Information retrieval technology is an important branch of the current information technology field, which involves extracting valuable information and knowledge from massive text data.

[0003] Traditional information retrieval generally calculates the similarity between the query statement / question vector and the vector of each text block, and takes the text block with the highest similarity as the retrieval result. The retrieval process considers a single dimension, which will lead to incomplete retrieval results. Summary of the invention

[0004] The embodiments of the present application provide an information retrieval method, apparatus, device, storage medium and product to solve the problem that the information retrieval process considers a single dimension, resulting in incomplete retrieval results.

[0005] In a first aspect, an embodiment of the present application provides an information retrieval method, the method comprising:

[0006] Receiving a search instruction for searching a text to be searched, the search instruction is used to search for a corresponding search result in a document set using the text to be searched, the document set includes multiple layers of documents, and the document granularity of each layer of documents decreases from top to bottom;

[0007] Using the text to be searched, recursively search the document collection from bottom to top to obtain a first result corresponding to each layer of documents;

[0008] The first result is processed to obtain the search result.

[0009] In a second aspect, an embodiment of the present application provides an information retrieval device, the device comprising:

[0010] A receiving module, used for receiving a search instruction for searching a text to be searched, wherein the search instruction is used for searching a corresponding search result in a document set using the text to be searched, wherein the document set includes multiple layers of documents, and the document granularity of each layer of documents decreases from top to bottom;

[0011] A retrieval module, used to perform recursive retrieval in the document collection from bottom to top using the text to be retrieved, and obtain a first result corresponding to each layer of documents;

[0012] A processing module is used to process the first result to obtain the search result.

[0013] In a third aspect, an embodiment of the present application provides a terminal device, the device comprising: a processor and a memory storing computer program instructions;

[0014] When the processor executes the computer program instructions, the information retrieval method of the first aspect is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the information retrieval method of the first aspect is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the information retrieval method of the first aspect.

[0017] The information retrieval method provided by the embodiment of the present application, first, realizes the information organization method from broad to fine by constructing a multi-layer document system with gradually decreasing granularity, so that in the subsequent recursive retrieval process, the retrieval can be performed first in the documents at the bottom layer. Since the granularity of the documents at the bottom layer is small and the information is more specific, the matching accuracy is better, and the user's query needs can be met more accurately. Then, the accurate retrieval results are used to perform a recursive retrieval from bottom to top, which can make full use of the information of each layer of documents and avoid the problems of "missed detection" or "false detection" that may occur in traditional retrieval. It ensures that in a more refined document hierarchy, even if the direct match between the text to be retrieved and the upper-level document is not high, it can be indirectly verified by its existence in the next-level document, which can cover the entire document structure and provide a more comprehensive retrieval perspective. Thereby improving the comprehensiveness and accuracy of the retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 It is a flowchart of the information retrieval method provided in the embodiment of the present application;

[0020] Figure 2 is a schematic diagram of a document collection provided in an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of the structure of an information retrieval device provided in an embodiment of the present application;

[0022] Figure 4 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0024] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0025] In order to solve the problems of the prior art, the embodiments of the present application provide an information retrieval method, device, equipment, storage medium and product.

[0026] The following first introduces the information retrieval method provided in the embodiment of the present application.

[0027] Figure 1 FIG. 1 is a flow chart of an information retrieval method provided by an embodiment of the present application. Figure 1 As shown, the method includes:

[0028] Step 101, receiving a search instruction for searching a text to be searched, wherein the search instruction is used to search for a corresponding search result in a document set using the text to be searched, wherein the document set includes multiple layers of documents, and the document granularity of each layer of documents decreases from top to bottom;

[0029] In this embodiment, the search instruction may be sent by the user through a terminal device. The search instruction carries the text to be searched, which may be a question raised by the user or a text that the user wants to query. The document set is the search target of the text to be searched, including documents of multiple levels, and the granularity of the content of the documents in each level decreases from top to bottom. That is, the content in the top-level document contains more extensive and abstract information, while the bottom-level document contains more specific and detailed information.

[0030] Step 102, using the text to be searched to perform recursive search in the document collection from bottom to top, to obtain a first result corresponding to each layer of documents;

[0031] In this embodiment, the text to be searched is used to perform a recursive search from the bottom layer (ie, the layer with the smallest granularity) of the document collection upward layer by layer.

[0032] Specifically, at the bottom layer, each document is first matched to find the document with the highest similarity to the text to be retrieved as the candidate result. Then, the documents in the previous layer where these candidate results are located are used as the new search scope to continue similarity matching and screening. This process continues until the top layer document, and each layer will generate the first result related to the text to be retrieved.

[0033] Step 103: Process the first result to obtain the search result.

[0034] In this embodiment, the first result outputted by each layer may be summarized to obtain the final search result.

[0035] In this embodiment, first, by constructing a multi-layer document system with gradually decreasing granularity, an information organization method from broad to fine is realized, so that in the subsequent recursive retrieval process, the retrieval can be performed first in the documents at the bottom layer. Since the granularity of the documents at the bottom layer is smaller and the information is more specific, the matching accuracy is better, and the user's query needs can be met more accurately. Then, by using the accurate retrieval results to perform a recursive retrieval from bottom to top, the information of each layer of documents can be fully utilized, avoiding the "missed detection" or "false detection" problems that may occur in traditional retrieval. It ensures that in a more refined document hierarchy, even if the direct match between the text to be retrieved and the upper-layer document is not high, it can be indirectly verified through its existence in the next layer of documents, which can cover the entire document structure and provide a more comprehensive retrieval perspective. Thereby improving the comprehensiveness and accuracy of the retrieval.

[0036] In an embodiment of the present application, before receiving a search instruction for searching a text to be searched, the method further includes:

[0037] For any first grandparent document among the plurality of first grandparent documents, split the any first grandparent document according to paragraphs to obtain a plurality of first parent documents;

[0038] For any of the first parent documents, split the document according to the sentence punctuation marks to obtain multiple sentences;

[0039] Among the multiple sentences, a first preset number of sentences are sequentially determined as the first sub-documents according to the arrangement order of the sentences, so as to obtain a plurality of the first sub-documents.

[0040] In this embodiment, if Figure 2 As shown, each layer of documents is, from top to bottom, the first grandfather document (i.e., grandfather document 1, grandfather document 2, and grandfather document 3), the first parent document (i.e., parent document 1, parent document 2, and parent document 3), and the first child document (i.e., child document 1, child document 2, child document 3... child document N). The number of the first grandfather document, the first parent document, and the first child document is multiple. Among them, the first grandfather document is the original document content itself, that is, the highest-level document. The first grandfather document is split according to paragraphs to obtain the first parent document, and the first parent document is constructed as a group of a preset number of sentences to obtain a child document. The first child document is a subdivision of the first parent document content, and the first parent document is a subdivision of the first grandfather document content.

[0041] Specifically, there are multiple original documents, each of which is a first grandfather document. For each first grandfather document, first split according to the paragraphs of the first grandfather document to obtain multiple first parent documents. For example, if the first grandfather document consists of 5 paragraphs, it is split into 5 first parent documents.

[0042] For each first parent document, the document is split according to the sentence punctuation marks in the first parent document to obtain multiple sentences. The sentence punctuation marks may include a period (.), a question mark (?), an exclamation mark (!), etc. Specifically, a regular expression may be used to match the sentence punctuation marks and the spaces or line breaks following them to divide the first parent document into multiple sentences. After obtaining multiple sentences, a preset number of sentences may be sequentially treated as a sub-document according to the order of arrangement of the sentences, and multiple sub-documents may be obtained, wherein the preset number may be 2, 3, or 4, etc., which is not limited in this embodiment.

[0043] In this embodiment, by performing hierarchical document splitting and construction on the grandfather document, the fine management and efficient retrieval of the document content are achieved, which not only helps the maintenance and update of the document, but also makes the information retrieval more efficient and accurate, ensuring the accuracy of the retrieval results.

[0044] In one embodiment of the present application, the recursive search is performed from bottom to top in the document collection using the text to be searched to obtain the first result corresponding to each layer of documents, including:

[0045] Sort the plurality of first sub-documents according to their similarities to the text to be retrieved, and select the first N first sub-documents with the highest similarities as similar sub-documents, where N is a positive integer;

[0046] Filter out a plurality of second parent documents containing the similar sub-documents from the plurality of first parent documents;

[0047] Sort multiple second parent documents according to their similarity to the text to be retrieved, and select the first M second parent documents with the highest similarity as similar parent documents, where M is a positive integer;

[0048] Filter out a plurality of second grandparent documents containing the similar parent document from the plurality of first grandparent documents;

[0049] The plurality of second grandfather documents are sorted according to the similarity with the text to be retrieved, and the first K second grandfather documents with the highest similarity are selected as similar grandfather documents, where K is a positive integer.

[0050] In this embodiment, since the document set is recursively searched from the bottom up, a search is first performed in multiple first sub-documents. Specifically, the first text embedding vector of each first sub-document and the second text embedding vector of the text to be searched can be obtained; then the cosine similarity between each first text embedding vector and the second text embedding vector is determined; the cosine similarity is the similarity between the first sub-document and the text to be searched. Then the similarity between each first sub-document and the text to be searched is sorted, and the first N first sub-documents with the highest similarity are selected as similar sub-documents, where N is a positive integer, such as 1, 2, 3 or 4.

[0051] After obtaining similar child documents, further search is performed at the parent document level. Specifically, the system will screen out those parent documents that contain at least one similar child document as candidate parent documents, i.e., second parent documents. Next, similarity is calculated for these candidate parent documents and they are sorted according to the similarity scores. Finally, the top M parent documents with the highest similarity are selected as similar parent documents, where M is also a positive integer.

[0052] After obtaining similar parent documents, further search is performed at the grandparent document level to select those grandparent documents that contain at least one similar parent document as candidate grandparent documents, i.e., second grandparent documents. Then, similarity calculations are performed on these candidate grandparent documents, and they are sorted according to the similarity scores. Finally, the top K grandparent documents with the highest similarity are selected as similar grandparent documents, where K is also a positive integer.

[0053] In this embodiment, through the above recursive retrieval process, the system can filter out the documents most relevant to the text to be retrieved layer by layer, and generate the first result including similar child documents, similar parent documents and similar grandparent documents. This process not only makes full use of the hierarchical structure of the document collection, but also improves the efficiency and accuracy of the retrieval by gradually narrowing the retrieval scope and optimizing the matching algorithm.

[0054] In one embodiment, the step of sorting the plurality of second parent documents according to their similarities to the text to be retrieved and selecting the first M second parent documents with the highest similarities as similar parent documents includes:

[0055] For any second parent document, determine the similarity between each third child document contained in the second parent document and the text to be retrieved, and obtain multiple similarities;

[0056] Determine an average value of the multiple similarities as the similarity between the second parent document and the text to be retrieved;

[0057] The plurality of second parent documents are sorted according to their similarities with the text to be retrieved, and the first M second parent documents with the highest similarities are selected as similar parent documents.

[0058] In this embodiment, since the sub-files are split from the parent document, the overall similarity of the second parent document can be approximately represented by the similarity of the third sub-documents it contains. The third sub-documents mentioned above are all the sub-documents contained in the second parent document. Specifically, the similarity between the parent text and the text to be retrieved is obtained in the following way:

[0059] First, the similarity between each third child document and the text to be retrieved is calculated. The similarity can be calculated by cosine similarity. The specific calculation method is the same as that in the above embodiment, and this embodiment will not be repeated here. Secondly, the similarity corresponding to each third child document is averaged to obtain the similarity between the second parent document and the text to be retrieved. By taking the average value, the influence of the high or low similarity of individual child documents on the overall evaluation can be smoothed out, thereby obtaining a more stable and reliable similarity index. Finally, multiple second parent documents are sorted according to similarity, and the top M second parent documents are taken as similar parent documents. Here, M is a preset positive integer, which can be adjusted according to the actual application scenario and requirements. By limiting the number of selections, the amount of data processed can be reduced while ensuring the relevance of the retrieval results, thereby improving the retrieval efficiency.

[0060] In this embodiment, in the process of sorting multiple second parent documents according to their similarity to the text to be retrieved and selecting the top M with the highest similarity as similar parent documents, the system uses cosine similarity to calculate the similarity between the texts and evaluates the overall similarity of the second parent documents by taking the average value. This method makes full use of the advantages of the hierarchical structure of the document collection and text similarity calculation, effectively improving the accuracy and efficiency of information retrieval.

[0061] In an embodiment of the present application, the step of sorting the plurality of second grandparent documents according to the similarity with the text to be retrieved and selecting the first K second grandparent documents with the highest similarity as similar grandparent documents includes:

[0062] For any second grandparent document, determine the similarity between each third parent document contained in the second grandparent document and the text to be retrieved, and obtain multiple cosine similarities;

[0063] Determine an average value of the multiple similarities as the similarity between the second grandfather document and the desired text;

[0064] The plurality of second grandfather documents are sorted according to the similarity with the text to be retrieved, and the first K second grandfather documents with the highest similarity are selected as similar grandfather documents.

[0065] In this embodiment, since the sub-files are split from the grandparent document, the overall similarity of the second grandparent document can be approximately represented by the similarity of the third parent document it contains. The third parent document mentioned above is all the parent documents contained in the second grandparent document. Specifically, the similarity between the parent text and the text to be retrieved is obtained in the following way:

[0066] First, calculate the similarity between each third parent document and the text to be retrieved. The similarity can be calculated by cosine similarity. The specific calculation method is the same as the above embodiment, and this embodiment will not be repeated here. Secondly, take the average of the similarities corresponding to each third parent document to obtain the similarity between the second grandfather document and the text to be retrieved. By taking the average value, the impact of the excessively high or low similarity of individual parent documents on the overall evaluation can be smoothed out, thereby obtaining a more stable and reliable similarity index. Finally, sort multiple second grandfather documents according to similarity, and take the top M second grandfather documents as similar grandfather documents. Here M is a preset positive integer, which can be adjusted according to the actual application scenario and needs. By limiting the number of selections, the amount of data processed can be reduced while ensuring the relevance of the retrieval results, thereby improving the retrieval efficiency.

[0067] In this embodiment, in the process of sorting multiple second grandfather documents according to their similarity to the text to be retrieved and selecting the top M with the highest similarity as similar grandfather documents, the system uses cosine similarity to calculate the similarity between texts and evaluates the overall similarity of the second grandfather documents by taking the average value. This method makes full use of the advantages of the hierarchical structure of the document collection and text similarity calculation, effectively improving the accuracy and efficiency of information retrieval.

[0068] In an embodiment of the present application, the processing of the first result to obtain the search result includes:

[0069] The similar child documents, the similar parent documents, the similarities and secondary subject words corresponding to the similar parent documents, the similar grandparent documents, the similarities and primary subject words corresponding to the similar grandparent documents are determined as the retrieval results, the secondary subject words are used to represent the subject of the similar parent documents, and the primary subject words are used to represent the subject of the similar grandparent documents.

[0070] In this embodiment, similar sub-documents are sub-documents that are closest to the user's query in terms of content, semantics or keywords. They are located at the bottom layer of the document collection and provide the most specific and detailed information.

[0071] A similar parent document is a parent document of a similar child document, which contains multiple similar child documents and is related to the user query in a wider range. A similar parent document not only provides the direct context of the child document, but also may contain other important information related to the child document.

[0072] A similar grandfather document is located at a higher level in the document collection and is the parent document of a similar parent document. It contains multiple similar parent documents and is related to the user query at a more macro level. Similar grandfather documents provide users with broader and more comprehensive background information.

[0073] Both the first-level subject words and the second-level subject words can be determined by topic modeling. Please refer to the following embodiments for details.

[0074] All the above information is combined to form the final search results. Such results not only provide users with directly related document content, but also help users understand the relevance between the document and the user's query and the subject area to which the document belongs through similarity and subject terms, thereby improving the efficiency and accuracy of information retrieval.

[0075] In an embodiment of the present application, before sorting the plurality of first sub-documents according to their similarities to the text to be retrieved and selecting the first N first sub-documents with the highest similarities as similar sub-documents, the method further includes:

[0076] For any first sub-document, input the first sub-document into a pre-acquired text model to obtain a plurality of first text embedding vectors;

[0077] Among the plurality of first text embedding vectors, determining a first text embedding vector with a classification mark as a target vector, wherein the first text embedding vector with a classification mark is used to summarize semantic information of the first sub-document;

[0078] The target vector is used to determine the similarity between the first sub-document and the text to be retrieved.

[0079] In this embodiment, the text model can be a pre-trained Transformer model, and the above-mentioned classification marker is [CLS]token. In the Transformer model, [CLS]token is a special marker, which is usually used in classification tasks. Its main function is to serve as a classification feature to help the model understand the input sequence as a whole and extract global information. In the classification task, the output of the Transformer model is usually obtained through the representation of [CLS]token. Since [CLS]token has aggregated global information, its representation can be used as an input feature for the classification task. Therefore, the first sub-document can be input into the Transformer model as a whole, and the first text embedding vector corresponding to [CLS]token is extracted as the target vector of the first sub-document.

[0080] In addition, if the first sub-document is relatively long, and the maximum text length that can be input into the Transformer pre-trained model is 512 characters, the first sub-document can be further divided into smaller texts, and their first text embedding vectors are generated respectively. Then, these first text embedding vectors are averaged, and the average value is the target vector of the sub-document.

[0081] After obtaining the target vector, the target vector can be directly used to determine the similarity between the first sub-document and the text to be retrieved. Specifically, the similarity between the target vector and the text to be retrieved can be directly calculated. The calculation method thereof is described in the above embodiment, which will not be described in detail in this embodiment.

[0082] In this embodiment, by using the text model to extract text features from any first sub-document to obtain a target vector, the target vector is then used to calculate the similarity between the first sub-document and the text to be retrieved, thereby improving the accuracy of the similarity calculation.

[0083] In an embodiment of the present application, after splitting the arbitrary first grandparent document into paragraphs to obtain a plurality of the first parent documents, the method further includes:

[0084] Performing topic modeling on the plurality of first grandfather documents to obtain at least one first-level topic word contained in each of the plurality of first grandfather documents;

[0085] Before recursively searching the document collection from bottom to top using the text to be searched, the method further includes:

[0086] The plurality of grandparent documents are screened using the text subject words corresponding to the text to be retrieved and the at least one first-level subject word to obtain a plurality of first parent documents matching the text subject words.

[0087] In this embodiment, topic modeling is a technique for extracting abstract topics from a large amount of text data. Specifically, a topic modeling algorithm (such as LDA, PLSA, etc.) is executed on multiple first grandfather documents to identify at least one first subject word contained in each first grandfather document. These subject words represent the main content or core topic of the document, which is helpful for subsequent screening and classification of documents.

[0088] In order to narrow the search scope and improve the search efficiency, before recursive search is performed using the text to be searched, the second subject word corresponding to the text to be searched and the first subject word in the grandfather document are first screened.

[0089] Specifically, firstly, subject word matching is performed: the text subject words of the text to be retrieved are matched with the first subject words of each grandparent document, and a grandparent document containing at least one first subject word matching the text subject word is found.

[0090] Secondly, filter the target parent document: Since these filtered out grandparent documents may contain multiple parent documents, the system needs to further split these grandparent documents into parent documents and use the parent documents belonging to the filtered out grandparent documents as the target parent documents.

[0091] Through the above steps, the document content most relevant to the user's query can be located more accurately, and retrieved and presented in a more efficient manner.

[0092] like Figure 3 As shown, the embodiment of the present application further provides an information retrieval device 300, the device comprising:

[0093] A receiving module 301 is used to receive a search instruction for searching a text to be searched, wherein the search instruction is used to search for a corresponding search result in a document set using the text to be searched, wherein the document set includes multiple layers of documents, and the document granularity of each layer of documents decreases from top to bottom;

[0094] A search module 302 is used to perform a recursive search from bottom to top in the document collection using the text to be searched, and obtain a first result corresponding to each layer of documents;

[0095] The processing module 303 is used to process the first result to obtain the search result.

[0096] Optionally, the information retrieval device 300 further includes:

[0097] A first splitting module is used for splitting any first grandparent document among the plurality of first grandparent documents according to paragraphs to obtain a plurality of first parent documents;

[0098] A second splitting module is used to split any of the first parent documents according to sentence punctuation marks to obtain multiple sentences;

[0099] The first determining module is used to determine, from the multiple sentences, a first preset number of sentences as the first sub-documents in sequence according to the arrangement order of the sentences, to obtain a plurality of the first sub-documents.

[0100] Optionally, the retrieval module 302 includes:

[0101] A first sorting submodule, used to sort the plurality of first sub-documents according to their similarities with the text to be retrieved, and select the first N first sub-documents with the highest similarities as similar sub-documents, where N is a positive integer;

[0102] A first screening submodule, used for screening out a plurality of second parent documents containing the similar sub-documents from the plurality of the first parent documents;

[0103] A second sorting submodule is used to sort the multiple second parent documents according to their similarity to the text to be retrieved, and select the first M second parent documents with the highest similarity as similar parent documents, where M is a positive integer;

[0104] A second screening submodule, configured to screen out a plurality of second grandparent documents containing the similar parent document from the plurality of the first grandparent documents;

[0105] The third sorting submodule is used to sort the multiple second grandparent documents according to the similarity with the text to be retrieved, and select the first K second grandparent documents with the highest similarity as similar grandparent documents, where K is a positive integer.

[0106] Optionally, the second sorting submodule includes:

[0107] A first determination unit is used to determine, for any second parent document, a similarity between each third child document contained in the second parent document and the text to be retrieved, to obtain multiple similarities;

[0108] A second determining unit, configured to determine an average value of the plurality of similarities as the similarity between the second parent document and the text to be retrieved;

[0109] The sorting unit is used to sort the multiple second parent documents according to the similarity with the text to be retrieved, and select the first M second parent documents with the highest similarity as similar parent documents.

[0110] Optionally, the sorting unit is specifically configured to:

[0111] For any second grandfather document, determine the similarity between each third father document contained in the second grandfather document and the text to be retrieved, and obtain multiple cosine similarities;

[0112] Determine an average value of the multiple similarities as the similarity between the second grandfather document and the desired text;

[0113] The plurality of second grandfather documents are sorted according to the similarity with the text to be retrieved, and the first K second grandfather documents with the highest similarity are selected as similar grandfather documents.

[0114] Optionally, the processing module 303 is specifically configured to:

[0115] The similar child documents, the similar parent documents, the similarities and secondary subject words corresponding to the similar parent documents, the similar grandparent documents, the similarities and primary subject words corresponding to the similar grandparent documents are determined as the retrieval results, the secondary subject words are used to represent the subject of the similar parent documents, and the primary subject words are used to represent the subject of the similar grandparent documents.

[0116] Optionally, the information retrieval device 300 further includes:

[0117] An input module, configured to input any first sub-document into a pre-acquired text model to obtain a plurality of first text embedding vectors;

[0118] a second determination module, configured to determine, among the plurality of first text embedding vectors, a first text embedding vector with a classification mark as a target vector, wherein the first text embedding vector with a classification mark is used to summarize the semantic information of the first sub-document;

[0119] The third determination module is used to determine the similarity between the first sub-document and the text to be retrieved by using the target vector.

[0120] Optionally, the information retrieval device 300 further includes:

[0121] A modeling module, used for performing topic modeling on the plurality of first grandfather documents to obtain at least one first-level topic word contained in each of the plurality of first grandfather documents;

[0122] The screening module is used to screen the plurality of grandparent documents using the text subject words corresponding to the text to be retrieved and the at least one first-level subject word to obtain a plurality of first parent documents matching the text subject words.

[0123] It should be noted that the information retrieval device 300 is a device corresponding to the above-mentioned information retrieval method, and all implementation methods in the above-mentioned method embodiment are applicable to the embodiment of the device and can achieve the same technical effect.

[0124] Figure 4 A schematic diagram of the hardware structure of a terminal device provided in an embodiment of the present application is shown.

[0125] The terminal device may include a processor 401 and a memory 402 storing computer program instructions.

[0126] Specifically, the processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0127] Memory 402 may include a large capacity memory for data or instructions. By way of example and not limitation, memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, memory 402 may include a removable or non-removable (or fixed) medium. In appropriate cases, memory 402 may be inside or outside of an integrated gateway disaster recovery device. In a specific embodiment, memory 402 is a non-volatile solid-state memory.

[0128] In certain embodiments, the memory 402 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0129] The processor 401 implements any one of the information retrieval methods in the above embodiments by reading and executing computer program instructions stored in the memory 402 .

[0130] In one example, the terminal device may further include a communication interface 404 and a bus 410. Figure 4 As shown, the processor 401, the memory 402, and the communication interface 404 are connected via a bus 410 and communicate with each other.

[0131] The communication interface 404 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0132] Bus 410 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 410 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0133] In addition, in combination with the information retrieval method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the information retrieval methods in the above embodiment is implemented.

[0134] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0135] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0136] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0137] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable information retrieval device to produce a machine so that these instructions executed by the processor of the computer or other programmable information retrieval device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be implemented by the dedicated hardware that performs the specified function or action, or can be implemented by the combination of dedicated hardware and computer instructions.

[0138] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. An information retrieval method, characterized in that: The method comprises: Receiving a search instruction for searching a text to be searched, the search instruction is used to search for a corresponding search result in a document set using the text to be searched, the document set includes multiple layers of documents, and the document granularity of each layer of documents decreases from top to bottom; Using the text to be searched, recursively search the document collection from bottom to top to obtain a first result corresponding to each layer of documents; The first result is processed to obtain the search result.

2. The information retrieval method according to claim 1, characterized in that: The documents in each layer are, from top to bottom, the first grandfather document, the first parent document and the first child document; Before receiving a search instruction for searching the text to be searched, the method further includes: For any first grandparent document among the plurality of first grandparent documents, split the any first grandparent document according to paragraphs to obtain a plurality of first parent documents; For any of the first parent documents, split the document according to the sentence punctuation marks to obtain multiple sentences; Among the multiple sentences, a first preset number of sentences are sequentially determined as the first sub-documents according to the arrangement order of the sentences, so as to obtain a plurality of the first sub-documents.

3. The information retrieval method according to claim 2, characterized in that: The first result includes similar child documents, similar parent documents and similar grandparent documents; The method of recursively searching the document set from bottom to top using the text to be searched to obtain first results corresponding to each layer of documents includes: Sort the plurality of first sub-documents according to their similarities to the text to be retrieved, and select the first N first sub-documents with the highest similarities as similar sub-documents, where N is a positive integer; Filter out a plurality of second parent documents containing the similar sub-documents from the plurality of first parent documents; Sort multiple second parent documents according to their similarity to the text to be retrieved, and select the first M second parent documents with the highest similarity as similar parent documents, where M is a positive integer; Filter out a plurality of second grandparent documents containing the similar parent document from the plurality of first grandparent documents; The plurality of second grandfather documents are sorted according to the similarity with the text to be retrieved, and the first K second grandfather documents with the highest similarity are selected as similar grandfather documents, where K is a positive integer.

4. The information retrieval method according to claim 3, characterized in that: The step of sorting the plurality of second parent documents according to their similarities to the text to be retrieved and selecting the first M second parent documents with the highest similarities as similar parent documents comprises: For any second parent document, determine the similarity between each third child document contained in the second parent document and the text to be retrieved, and obtain multiple similarities; Determine an average value of the multiple similarities as the similarity between the second parent document and the text to be retrieved; The plurality of second parent documents are sorted according to their similarities with the text to be retrieved, and the first M second parent documents with the highest similarities are selected as similar parent documents.

5. The information retrieval method according to claim 3, characterized in that: The step of sorting the plurality of second grandfather documents according to the similarity with the text to be retrieved and selecting the first K second grandfather documents with the highest similarity as similar grandfather documents comprises: For any second grandparent document, determine the similarity between each third parent document contained in the second grandparent document and the text to be retrieved, and obtain multiple similarities; Determine an average value of the multiple similarities as the similarity between the second grandfather document and the desired text; The plurality of second grandfather documents are sorted according to the similarity with the text to be retrieved, and the first K second grandfather documents with the highest similarity are selected as similar grandfather documents.

6. The information retrieval method according to claim 5, characterized in that: The processing of the first result to obtain the search result includes: The similar child documents, the similar parent documents, the similarities and secondary subject words corresponding to the similar parent documents, the similar grandparent documents, the similarities and primary subject words corresponding to the similar grandparent documents are determined as the retrieval results, the secondary subject words are used to represent the subject of the similar parent documents, and the primary subject words are used to represent the subject of the similar grandparent documents.

7. The information retrieval method according to claim 3, characterized in that: Before sorting the plurality of first sub-documents according to their similarities with the text to be retrieved and selecting the first N first sub-documents with the highest similarities as similar sub-documents, the method further includes: For any first sub-document, input the first sub-document into a pre-acquired text model to obtain a plurality of first text embedding vectors; Among the plurality of first text embedding vectors, determining a first text embedding vector with a classification mark as a target vector, wherein the first text embedding vector with a classification mark is used to summarize semantic information of the first sub-document; The target vector is used to determine the similarity between the first sub-document and the text to be retrieved.

8. The information retrieval method according to claim 3, characterized in that: After splitting the arbitrary first grandparent document into paragraphs to obtain a plurality of the first parent documents, the method further includes: Performing topic modeling on the plurality of first grandfather documents to obtain at least one first-level topic word contained in each of the plurality of first grandfather documents; Before recursively searching the document collection from bottom to top using the text to be searched, the method further includes: The plurality of grandparent documents are screened using the text subject words corresponding to the text to be retrieved and the at least one first-level subject word to obtain a plurality of first parent documents matching the text subject words.

9. An information retrieval device, characterized in that: The device comprises: A receiving module, used for receiving a search instruction for searching a text to be searched, wherein the search instruction is used for searching a corresponding search result in a document set using the text to be searched, wherein the document set includes multiple layers of documents, and the document granularity of each layer of documents decreases from top to bottom; A retrieval module, used to perform recursive retrieval in the document collection from bottom to top using the text to be retrieved, and obtain a first result corresponding to each layer of documents; A processing module is used to process the first result to obtain the search result.

10. A terminal device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the information retrieval method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the information retrieval method according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that The computer product comprises a computer program, and when the computer program is executed by a processor, the information retrieval method according to any one of claims 1 to 8 is implemented.

Images