Data retrieval methods and devices, electronic devices and storage media
By generating an index hash table in the knowledge platform and using hash value comparison, the problem of low document retrieval efficiency in the knowledge platform is solved, and fast and efficient retrieval results are obtained.
Patent Information
- Application Number
- CN202310075336.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-01-17
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-01-17
AI Technical Summary
In a knowledge platform, traditional document retrieval methods require comprehensive searching of a large number of documents, resulting in excessively long retrieval times and low efficiency.
By generating an index hash table, the hash value of the search keyword is compared with the index hash value to determine the target index hash value, thereby obtaining the search results, reducing the retrieval of irrelevant hash values, and improving search efficiency.
By limiting the determination of search results to the target hash value in the index hash table, search efficiency is significantly improved and search time is reduced.
Smart Images

Figure CN116126899B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of information processing technology, specifically to a data retrieval method and apparatus, electronic device and storage medium. Background Technology
[0002] With the advent and rapid development of the information age, a vast amount of knowledge and information has been generated. In their daily work, life, and studies, people often face the challenge of finding the information they need from this massive amount of data. Taking a knowledge platform as an example, a knowledge platform can use artificial intelligence to collect, organize, retrieve, and apply various types of knowledge. A vast amount of knowledge and information can be stored in the knowledge platform and presented in document form.
[0003] In traditional technologies, documents are typically parsed to create index data, which is then used to perform a comprehensive search of the documents based on search keywords to obtain results. However, knowledge platforms usually store a large number of documents, and performing a comprehensive search on such a large number of documents typically requires a significant amount of time. Summary of the Invention
[0004] This application provides a data retrieval method and apparatus, an electronic device, and a storage medium.
[0005] In a first aspect, one embodiment of this application provides a data retrieval method, the method comprising: in response to a retrieval request for a preset document library, obtaining retrieval keywords corresponding to the retrieval request; wherein the preset document library includes at least one document; obtaining index data of the at least one document; the index data including an index hash table; determining a hash value corresponding to the retrieval keywords; comparing the hash value corresponding to the retrieval keywords with the index hash values in the index hash table to obtain a target index hash value most similar to the hash value corresponding to the retrieval keywords; and obtaining retrieval results corresponding to the retrieval keywords based on the target index hash value.
[0006] Secondly, one embodiment of this application provides a data retrieval device, which includes: a first acquisition module, configured to acquire search keywords corresponding to a search request in response to a search request for a preset document library; wherein the preset document library includes at least one document; a second acquisition module, configured to acquire index data of the at least one document; the index data of the document includes an index hash table; a determination module, configured to determine the hash value corresponding to the search keyword; a comparison module, configured to compare the hash value corresponding to the search keyword with the index hash value in the index hash table to obtain a target index hash value that is most similar to the hash value corresponding to the search keyword; and a retrieval module, configured to obtain the retrieval result corresponding to the search keyword based on the target index hash value.
[0007] Thirdly, one embodiment of this application provides an electronic device, the electronic device comprising: a processor; a memory for storing processor-executable instructions; the processor being configured to perform the method described in the first aspect.
[0008] Fourthly, one embodiment of this application provides a computer-readable storage medium storing a computer program for performing the method described in the first aspect.
[0009] This application provides a data retrieval method that, in response to a retrieval request for a preset document library, obtains the retrieval keywords corresponding to the retrieval request. The preset document library includes at least one document. The method then obtains the index data of the at least one document, which includes an index hash table. Next, it determines the hash value corresponding to the retrieval keyword, compares this hash value with the index hash values in the index hash table, and obtains the target index hash value most similar to the hash value corresponding to the retrieval keyword. Finally, based on the target index hash value, it obtains the retrieval results corresponding to the retrieval keyword. By comparing the hash value corresponding to the retrieval keyword with the index hash values in the index hash table to determine the target index hash value, and then obtaining the retrieval results based on the target index hash value, the determination of the retrieval results for the retrieval keyword can be limited to one index hash value in the index hash table, namely the target index hash value. This avoids using index hash values unrelated to the retrieval keyword to determine the retrieval results, thus improving retrieval efficiency. Attached Figure Description
[0010] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.
[0011] Figure 1 The diagram shown is a scenario applicable to an embodiment of this application.
[0012] Figure 2a The diagram shown is a flowchart illustrating a data retrieval method provided in an exemplary embodiment of this application.
[0013] Figure 2b A schematic diagram of an exemplary document summary tree is shown.
[0014] Figure 2c An exemplary index hash representation intent is shown.
[0015] Figure 3The diagram shown is a structural schematic of a data retrieval device provided in an exemplary embodiment of this application.
[0016] Figure 4 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation
[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0018] The technical solutions of this application embodiment can be used as plug-ins in any platform and software with retrieval function on the network, such as knowledge platforms, documents, browsers, and applications, and are not limited here.
[0019] Taking a knowledge platform as an example, document retrieval within the platform relies on index data. This index data is generated by segmenting the documents into words and determining the positions of the resulting feature words within the documents, thus representing the location of these feature words. During retrieval, a comprehensive search of the feature words in each document is typically performed based on the search keywords. However, knowledge platforms usually store a large number of documents, and performing a comprehensive search on such a large number of documents typically requires a significant amount of time.
[0020] To improve retrieval speed and reduce retrieval time, the inventors have proposed the technical solution of this application after a series of studies. In the embodiments of this application, in response to a retrieval request for a preset document library, the retrieval keywords corresponding to the retrieval request are obtained; wherein, the preset document library includes at least one document; the index data of the at least one document is obtained; the index data includes an index hash table; the hash value corresponding to the retrieval keyword is determined; the hash value corresponding to the retrieval keyword is compared with the index hash values in the index hash table to obtain the target index hash value most similar to the hash value corresponding to the retrieval keyword; based on the target index hash value, the retrieval result corresponding to the retrieval keyword is obtained. By comparing the hash value corresponding to the retrieval keyword with the index hash values in the index hash table to determine the target index hash value, and then obtaining the retrieval result corresponding to the retrieval keyword based on the target index hash value, the determination of the retrieval result for the retrieval keyword can be limited to one index hash value in the index hash table, namely the target index hash value, avoiding the use of index hash values unrelated to the index hash value of the retrieval keyword to determine the retrieval result, thus improving retrieval efficiency.
[0021] Figure 1The diagram shown illustrates a scenario applicable to an embodiment of this application. The scenario includes a preset document library 101, a retrieval system 102, and a user 103. The retrieval system 102 may include an index data generation device 104 and a retrieval device 105.
[0022] For example, the preset document library 101 includes at least one document for user 103 to retrieve through retrieval system 102.
[0023] For example, the index data generation device 104 can parse and process all documents in the preset document library 101 to generate index data. The specific method for determining the index data will be described in detail in the following embodiments, and will not be repeated here.
[0024] For example, after receiving a search request from user 103, the search device 105 will search the index data according to the search keywords in the search request to obtain the position of the search keywords in each document.
[0025] For example, after user 103 searches the preset document library 101 through the retrieval system 102, he / she can edit the retrieved document. The index data generation device 104 can update the index data of the document in real time based on the user 103's editing of the document, so that other users 103 can obtain more comprehensive and timely search results when they search.
[0026] In practical applications, the preset document library 101 can be a knowledge platform, and the index data generation device 104 will pre-generate index data corresponding to all documents in the knowledge platform. When user 103 wants to search the knowledge platform, they can enter search keywords in the knowledge platform's search engine. The retrieval device 105 will then search for the search keywords in the index data generated by the index data generation device 104, and display the search results to user 103 through the knowledge platform's page. The search results will include titles and paragraphs strongly related to the search keywords, as well as the position of the search keywords within those paragraphs. If there are many strongly related titles and paragraphs, the page will adaptively collapse the search results. User 103 can view all search results using controls on the webpage, such as "show all" or "uncollapse".
[0027] In another practical application, the preset document library 101 can be a document. The index data generation device 104 pre-generates index data for the document. When a user searches for the document through the retrieval system 102, they can use the document's built-in search box to search for keywords. The retrieval device 105 will then search for the keywords in the index data generated by the index data generation device 104 and display the search results to the user 103 through the search box page. This search result includes titles, paragraphs, and the position of the keywords within those paragraphs that are strongly related to the keywords. It also displays all positions of the keywords within the document. This document can be a shared editing document. When multiple users 103 edit the shared editing document, the index data generation device 104 can update the index data of the shared editing document in real time for each user's edit, so that other users 103 can obtain more comprehensive and timely search results when searching.
[0028] Figure 2a The diagram shown is a flowchart illustrating a data retrieval method provided in an exemplary embodiment of this application. Figure 2a As shown in the embodiments of this application, the data retrieval method may include the following steps.
[0029] Step S201: In response to a search request for a preset document library, obtain the search keywords corresponding to the search request.
[0030] For example, the preset document library includes at least one document. The search request may include search keywords, which are words that the user needs to search for.
[0031] Step S202: Obtain the index data of at least one document.
[0032] For example, index data may include an index hash table.
[0033] Specifically, the method for determining the index data of each document in the preset document library may include the following steps.
[0034] Step A: Parse the document to obtain its document structure and M document feature words.
[0035] For example, M is a positive integer.
[0036] For example, parsing a document involves structural partitioning and word segmentation. For ease of understanding, Figure 2b A schematic diagram of an exemplary document summary tree is shown. Figure 2bAs shown, after structural partitioning, the document can be divided into a structure of first-level headings - second-level headings - ... - n-level headings - paragraphs - content. After structural partitioning, the resulting headings and corresponding paragraphs are constructed into a document summary tree. Word segmentation is the process of splitting the document content into document feature words. After word segmentation, M document feature words can be obtained. Word segmentation can be performed using open-source tools such as Lucene; specific open-source tools are not limited here.
[0037] Step B: Based on the document structure and M document feature words, determine the document location information for each document feature word.
[0038] For example, the document location information includes the location information of the document feature words in the document structure, and the location information of the document feature words in the document.
[0039] Specifically, determining the document location information for each document feature word based on the document structure and M document feature words can include the following steps.
[0040] Step B1: Determine the position information of each document feature word in the document structure, and the position information of each document feature word in the document.
[0041] Step B2: Concatenate the position information of the document feature words in the document structure and the position information of the document feature words in the document to obtain the document position information of the document feature words.
[0042] For example, concatenating the positional information of document feature words in the document structure and the positional information of document feature words in the document is equivalent to integrating the positional information of document feature words in the document structure and the positional information of document feature words in the document. In a document containing document feature words, the document feature words usually appear at least once. When a document feature word appears multiple times in the document, the document feature word must appear in multiple positions. During the concatenation process, firstly, all corresponding positions of the document feature word in the document are integrated together, and then the corresponding positions of the document feature word in the document structure and all corresponding positions in the document are integrated together, i.e., in the form of document feature word - position in document structure - position 1 in document - position 2 in document - ... position n in document.
[0043] Specifically, the concatenated index data can be represented as {keyword: "", target: [{title: "", position: [startIndex: "", endIndex: ""]}]}. Here, keyword represents the document feature word after word segmentation. target represents all positions of the document feature word within the document, presented as an array. target-title represents the node number of the document feature word in the overview tree. target-position represents the start and end positions of the document feature word within the document. startIndex represents the start position, and endIndex represents the end position.
[0044] By concatenating the positional information of document feature words in the document structure and the positional information of document feature words in the target document, the search results can be displayed in the form of first-level heading - second-level heading - ... - n-level heading - paragraph - content, making it convenient for users to quickly locate the position of the search keywords in paragraphs that are strongly related to the search keywords.
[0045] Step C: Generate document index data based on the document location information of each document feature word.
[0046] Specifically, generating document index data based on the document location information of each document feature word may include the following steps.
[0047] Step C1: Determine the index hash value of each document feature word.
[0048] For example, a hash algorithm can be used to calculate the index hash value of each document feature word.
[0049] Step C2: Merge the document feature words with the same index hash value among the index hash values of each document feature word to obtain at least one group of document feature words.
[0050] For example, after calculating each document feature word using a hash algorithm, there may be cases where different document feature words have the same index hash value. In this case, the document feature words with the same index hash value and the document location information of the document feature words are integrated together to form at least one document feature word group.
[0051] For example, each document feature word group corresponds to an index hash value, and each document feature word group contains at least one document feature word and the document location information of each document feature word.
[0052] Step C3: Construct an index hash table based on the index hash value corresponding to each document feature word group.
[0053] For ease of understanding, Figure 2c An exemplary index hash representation is shown. Figure 2c As shown, index hash values are mapped to document feature word groups to form an index hash table. The index hash table displays at least one document feature word corresponding to each index hash value, along with the document location information for each document feature word. For example, if the index hash values for document feature words 1 and 2 are both index hash value 1, then document feature words 1, 2, and their respective document location information 1 and 2 are all located in document feature group 1, which corresponds to index hash value 1. Similarly, if the index hash values for document feature words 3 and 4 are both index hash value 2, then document feature words 3, 4, and their respective document location information 3 and 4 are all located in document feature group 2, which corresponds to index hash value 2. It should be understood that... Figure 2c The example uses two index hash values. The specific number of index hash values is determined based on the index hash values of the document feature words obtained after word segmentation of different documents.
[0054] Step C4: Use the index hash table as the index data for the documents.
[0055] For example, the index data includes an index hash table.
[0056] Since new documents edited from the original document can be uploaded to the preset document library, the search results obtained using the index data may contain multiple documents with high similarity. Therefore, after generating the document index data based on the document location information of each document's feature words, the following steps can also be included.
[0057] Step C 4-1 : Determine if a historical version of a document exists in the preset document library.
[0058] For example, the preset document library is updated as users upload documents. After uploading a document, a user may modify the content and upload it again. This can lead to some documents in the preset document library having high similarity. This problem can cause other users to retrieve multiple highly similar documents when searching, making it impossible to determine the timeliness of the content. To solve this problem, the preset document library is checked for the existence of historical versions of documents. The determination of historical versions is based on document similarity. A similarity threshold can be preset. If the similarity of multiple documents meets the threshold, the document uploaded earlier is considered a historical version.
[0059] Step C4-2 If a document has a historical version in the preset document library, the index data of the historical version is updated using the document's index data, and the updated index data is stored.
[0060] For example, if a historical version is found, the index data of the historical version is updated using the document's index data, and the updated index data is stored. After the index data of the historical version is updated, the original index data of the historical version can be unloaded to reduce memory usage.
[0061] Step C 4-3 If the document does not have a historical version in the preset document library, the document's index data will be stored directly.
[0062] By updating the index data of historical versions and unloading the original index data of historical versions, not only is the timeliness of search results guaranteed, but the memory usage of the index data of historical versions is also reduced.
[0063] Step S203: Determine the hash value corresponding to the search keyword.
[0064] For example, a hash algorithm can be used to calculate the hash value of the search keywords.
[0065] Step S204: Compare the hash value corresponding to the search keyword with the index hash value in the index hash table to obtain the target index hash value that is most similar to the hash value corresponding to the search keyword.
[0066] For example, if a hash value corresponding to the search keyword exists in the index hash table, the index hash value that matches the hash value corresponding to the search keyword is selected as the target index hash value. If no hash value corresponding to the search keyword exists in the index hash table, the index hash value that is closest to the hash value corresponding to the search keyword is selected as the target index hash value. Of course, there may be multiple closest index hash values, so the selection of the target index hash value can be pre-defined when multiple closest index hash values exist. For example, all of the multiple closest index hash values can be used as the target index hash value, or the largest or smallest index hash value among the multiple closest index hash values can be used as the target index hash value; this is not limited here.
[0067] Step S205: Based on the target index hash value, obtain the search results corresponding to the search keywords.
[0068] For example, the index hash table includes at least one document feature word corresponding to the index hash value and document location information corresponding to each of the at least one document feature word, wherein the document feature word is obtained after the document is segmented.
[0069] Specifically, obtaining the search results corresponding to the search keywords based on the target index hash value can include the following steps.
[0070] Step A: Determine the document feature words corresponding to the target index hash value.
[0071] For example, by using an index hash table, the document feature words corresponding to the target index hash value can be determined.
[0072] Step B: Determine the search keywords from the document feature words corresponding to the target index hash value.
[0073] For example, the document feature words corresponding to the target index hash value are compared with the search keywords one by one to select the search keywords. Of course, there are also cases where the search keywords are not found in the document feature words corresponding to the target index hash value. In this case, the document corresponding to the index hash table is not used as the search result for the search keywords.
[0074] Step C: Determine the document location information corresponding to the search keywords.
[0075] For example, an index hash table can be used to determine the document location information corresponding to the search keywords.
[0076] Step D: Based on the document location information corresponding to the search keywords, obtain the search results corresponding to the search keywords.
[0077] The search results show the location information of the search keywords in at least one document.
[0078] Specifically, obtaining search results based on the document location information corresponding to the search keywords may include the following steps.
[0079] Step D1: Based on the document location information corresponding to the search keywords, identify the paragraphs in at least one document where the search keywords appear most frequently.
[0080] For example, the paragraph in which the search keyword appears most frequently can be considered to be a paragraph that is strongly related to the search keyword.
[0081] Step D2: Present the search keywords in the paragraphs where the search keywords appear most frequently in each of the at least one document, and obtain the search results corresponding to the search keywords.
[0082] For example, in the search results corresponding to the search keywords, the search keywords in the paragraphs of at least one document that contain the search keywords with the highest frequency can be highlighted for the user's viewing.
[0083] In practical applications, taking a browser as an example, users can search for keywords using the browser's search engine. After the browser detects that the search function has been triggered, it will send a search request containing the search keywords to the search system. After receiving the search request, the search system will search all documents in the preset document library through index data to obtain the search results. Then, it will highlight the search keywords in all positions in the document and trace back to their parent paragraphs or titles, presenting the search results in the form of title-paragraph-content.
[0084] In another practical application, the search request can contain only one search keyword. The search for a single keyword has already been described in the previous embodiments and will not be repeated here. Of course, the search request can also contain multiple search keywords. When there are multiple search keywords, the index data can be filtered to present search results containing all search keywords. Different search keywords can be highlighted using different colors to help users distinguish between them during the browsing process.
[0085] As one implementation method, obtaining the search results corresponding to the search keywords based on the target index hash value can also include the following steps.
[0086] Step D1: Divide the preset document library into multiple sub-document libraries.
[0087] For example, a preset document library can be split into multiple sub-document libraries according to the document type, or the preset document library can be randomly split into multiple sub-document libraries.
[0088] Step D2: For each sub-document library, determine the index hash table for each document in the sub-document library.
[0089] Step D3: Based on the index hash table of each document in the sub-document library, perform a search for each document in the sub-document library using search keywords to obtain the search results for the corresponding sub-document library.
[0090] For example, the method of determining the index hash table of each document in the sub-document library is the same as the method of determining the index data of each document in the preset document library, and the method of retrieving each document in the sub-document library is the same as the method of retrieving each document in the preset document library, which will not be elaborated here.
[0091] Step D4: Merge the search results corresponding to the multiple sub-document libraries to obtain the search results corresponding to the search keywords.
[0092] By splitting the preset document library into multiple sub-document libraries, and then searching each sub-document library separately, and finally merging the search results, the server load during the search is reduced, thereby improving search efficiency.
[0093] After retrieving the search results corresponding to the search keywords, there may be errors in the retrieved documents or they may need further updates. Therefore, after obtaining the search results corresponding to the search keywords based on the target index hash value, the following steps can also be included.
[0094] Step A: Iterate through the documents corresponding to the search results for the keywords.
[0095] Step B: If the document corresponding to the search results for the search keywords contains information that needs to be modified, then edit the document corresponding to the search results for the search keywords.
[0096] Specifically, editing the documents corresponding to the search results for the search keywords can include the following steps.
[0097] Step B1: In response to an editing request for at least one document, obtain the editing data corresponding to the editing request.
[0098] When a user is browsing in a preset document library, if they wish to edit the content of a document, they can make real-time modifications, provided they have the necessary editing permissions. The editing request can include editable data, such as newly added or deleted content.
[0099] Step B2: Based on the edit data, update the document structure and document feature words of at least one document.
[0100] User editing of documents changes the content of the documents, so it is necessary to update the document structure and document feature words of at least one document based on the changes in the original document structure caused by the editing data and the changes in document feature words obtained after document segmentation.
[0101] Step B3: Update the document's index data based on the updated document structure and document feature words.
[0102] For example, the updated document structure and document feature words are used to update the index data, and the resulting updated index data is the index data of the document after user editing.
[0103] When users directly edit documents while browsing, the document structure and key terms are updated based on the edited data. This updated document structure and key terms are then used to update the index data, ensuring high timeliness of retrieval and preventing situations where the index data is out of sync with the document, thus affecting the user experience.
[0104] In this embodiment of the application, the target index hash value is determined by comparing the hash value corresponding to the search keyword with the index hash value in the index hash table. Then, the search results corresponding to the search keyword are obtained based on the target index hash value. This limits the determination of the search results of the search keyword to the target index hash value in the index hash table, avoids using index hash values that are unrelated to the index hash value of the search keyword to determine the search results, and improves the efficiency of the search.
[0105] Figure 3 The diagram shown is a structural schematic of a data retrieval device provided in an exemplary embodiment of this application. Figure 3 As shown, the data retrieval device provided in this application embodiment may include the following modules.
[0106] The first acquisition module 301 is used to acquire the search keywords corresponding to the search request in response to a search request for a preset document library.
[0107] For example, the preset document library includes at least one document.
[0108] The second acquisition module 302 is used to acquire index data of at least one document.
[0109] For example, the index data includes an index hash table.
[0110] The determination module 303 is used to determine the hash value corresponding to the search keyword.
[0111] The comparison module 304 is used to compare the hash value corresponding to the search keyword with the index hash value in the index hash table to obtain the target index hash value that is most similar to the hash value corresponding to the search keyword.
[0112] The retrieval module 305 is used to obtain the retrieval results corresponding to the retrieval keywords based on the target index hash value.
[0113] In one embodiment of this application, the data retrieval device may further include the following modules.
[0114] The generation module is used to parse the document, obtain the document structure and M document feature words, where M is a positive integer; based on the document structure and the M document feature words, determine the document location information of each document feature word, which includes the position information of the document feature word in the document structure and the position information of the document feature word in the document; based on the document location information of each document feature word, generate the document's index data.
[0115] In one embodiment of this application, the generation module is further configured to: determine the index hash value of each document feature word; merge the document feature words corresponding to the same index hash value among the index hash values of each document feature word to obtain at least one document feature word group; wherein, each document feature word group corresponds to an index hash value, and each document feature word group contains at least one document feature word and the document location information of each document feature word; construct an index hash table based on the index hash value corresponding to each document feature word group; and use the index hash table as the index data of the document.
[0116] In one embodiment of this application, the retrieval module 305 is further configured to: determine document feature words corresponding to the target index hash value; determine retrieval keywords from the document feature words corresponding to the target index hash value; determine document location information corresponding to the retrieval keywords; and obtain retrieval results corresponding to the retrieval keywords based on the document location information corresponding to the retrieval keywords, wherein the retrieval results present the location information of the retrieval keywords in at least one document. The index hash table includes at least one document feature word corresponding to the index hash value and document location information corresponding to each of the at least one document feature word, wherein the document feature word is obtained after word segmentation of the document.
[0117] In one embodiment of this application, the retrieval module 305 is further configured to: determine the paragraph in at least one document that contains the highest frequency of the retrieval keyword based on the document location information corresponding to the retrieval keyword; present the retrieval keyword in the paragraph in each of the at least one document that contains the highest frequency of the retrieval keyword, and obtain the retrieval result corresponding to the retrieval keyword.
[0118] In one embodiment of this application, the retrieval module 305 is further configured to: split the preset document library into multiple sub-document libraries; determine the index hash table for each document in each sub-document library; perform a retrieval for each document in the sub-document library using retrieval keywords based on the index hash table of each document in the sub-document library, and obtain the retrieval results corresponding to the sub-document library; and merge the retrieval results corresponding to the multiple sub-document libraries to obtain the retrieval results corresponding to the retrieval keywords.
[0119] In one embodiment of this application, the data retrieval device may further include the following modules.
[0120] The traversal module is used to traverse the documents corresponding to the search results for the search keywords.
[0121] The editing module is used to edit the documents corresponding to the search results for the search keywords if there is information that needs to be modified.
[0122] It should be understood that Figure 3 The operation and functions of the first acquisition module 301, the second acquisition module 302, the determination module 303, and the comparison module 304 in the provided data retrieval device can be referred to the above. Figure 2a The data retrieval methods provided will not be elaborated here to avoid repetition.
[0123] Below, for reference Figure 4 This describes an electronic device according to embodiments of the present application. Figure 4 The diagram shown is a structural schematic of an electronic device provided in an exemplary embodiment of this application.
[0124] like Figure 4 As shown, the electronic device 40 includes one or more processors 401 and memory 402.
[0125] The processor 401 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 40 to perform desired functions.
[0126] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may execute the program instructions to implement the methods of the various embodiments of this application described above and / or other desired functions. Various content, such as target documents, document structures, and document feature words, may also be stored in the computer-readable storage medium.
[0127] In one example, the electronic device 40 may also include an input device 403 and an output device 404, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).
[0128] The input device 403 may include, for example, a keyboard, a mouse, etc.
[0129] The output device 404 can output various information to the outside, including the target document, document structure, and document keywords. The output device 404 may include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected thereto.
[0130] Of course, for the sake of simplicity, Figure 4 Only some of the components of the electronic device 40 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 40 may include any other suitable components depending on the specific application.
[0131] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods described above according to various embodiments of this application.
[0132] The computer program product can be written in any combination of one or more programming languages to perform the operations of the embodiments of this application. The programming languages include object-oriented programming languages such as Java and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0133] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods described above according to various embodiments of this application.
[0134] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0135] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.
[0136] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.
[0137] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.
[0138] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.
[0139] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.
Claims
1. A data retrieval method, characterized in that, include: In response to a search request for a preset document library, the search keywords corresponding to the search request are obtained; wherein the preset document library includes at least one document; Based on the document location information of each document feature word of the document, the index data of the document is generated; Determine the index hash value of each document feature word in the document; The document feature words with the same index hash value among the index hash values of each document feature word are merged to obtain at least one document feature word group; wherein, each document feature word group corresponds to an index hash value, and each document feature word group contains at least one document feature word and the document location information of each of the at least one document feature word; An index hash table is constructed based on the index hash value corresponding to each of the document feature word groups; Use the index hash table as the index data for the document; Determine the hash value corresponding to the search keyword; The hash value corresponding to the search keyword is compared with the index hash value in the index hash table to obtain the target index hash value that is most similar to the hash value corresponding to the search keyword; if the hash value corresponding to the search keyword exists in the index hash table, the index hash value that is the same as the hash value corresponding to the search keyword is selected as the target index hash value; if the hash value corresponding to the search keyword does not exist in the index hash table, the index hash value that is closest to the hash value corresponding to the search keyword is selected as the target index hash value. Based on the target index hash value, the search results corresponding to the search keywords are obtained.
2. The method according to claim 1, characterized in that, The methods for determining the index data of each document in the preset document library are as follows: Parse the document to obtain its document structure and M document feature words, where M is a positive integer; Based on the document structure and M document feature words, determine the document location information of each document feature word, wherein the document location information includes the location information of the document feature word in the document structure and the location information of the document feature word in the document; Based on the document location information of each document feature word, the index data of the document is generated.
3. The method according to claim 1, characterized in that, The index hash table includes at least one document feature word corresponding to the index hash value and document location information corresponding to each of the at least one document feature word. The document feature word is obtained after the document is segmented. The step of obtaining the search results corresponding to the search keywords based on the target index hash value includes: Determine the document feature words corresponding to the target index hash value; The search keywords are determined from the document feature words corresponding to the target index hash value; Determine the document location information corresponding to the search keywords; Based on the document location information corresponding to the search keyword, the search results corresponding to the search keyword are obtained, wherein the search results present the location information of the search keyword in the at least one document.
4. The method according to claim 3, characterized in that, The step of obtaining the search results corresponding to the search keywords based on the document location information corresponding to the search keywords includes: Based on the document location information corresponding to the search keywords, determine the paragraphs in the at least one document in which the search keywords appear most frequently; The search keywords are presented in the paragraphs of each of the at least one document where the search keyword appears most frequently, and the search results corresponding to the search keywords are obtained.
5. The method according to claim 1, characterized in that, The step of obtaining the search results corresponding to the search keywords based on the target index hash value includes: The preset document library is divided into multiple sub-document libraries; For each sub-document library, determine the index hash table for each document in the sub-document library; Based on the index hash table of each document in the sub-document library, the search keywords are used to search for each document in the sub-document library to obtain the search results corresponding to the sub-document library; The search results corresponding to the multiple sub-document libraries are merged to obtain the search results corresponding to the search keywords.
6. The method according to claim 1, characterized in that, After obtaining the search results corresponding to the search keywords based on the target index hash value, the method further includes: Iterate through the documents corresponding to the search results for the search keywords; If the document corresponding to the search result for the search keyword contains information that needs to be modified, then the document corresponding to the search result for the search keyword will be edited.
7. A data retrieval device, characterized in that, include: The first acquisition module is used to acquire the search keywords corresponding to the search request in response to a search request for a preset document library; wherein the preset document library includes at least one document; The second acquisition module is configured to generate index data for the document based on the document location information of each document feature word; determine the index hash value of each document feature word; merge document feature words with the same index hash value among the index hash values of each document feature word to obtain at least one document feature word group; wherein each document feature word group corresponds to an index hash value, and each document feature word group contains at least one document feature word and the document location information of each of the at least one document feature word; construct an index hash table based on the index hash value corresponding to each document feature word group; and use the index hash table as the index data of the document. The determination module is used to determine the hash value corresponding to the search keyword; The comparison module is used to compare the hash value corresponding to the search keyword with the index hash value in the index hash table to obtain the target index hash value that is most similar to the hash value corresponding to the search keyword; if the hash value corresponding to the search keyword exists in the index hash table, the index hash value that is the same as the hash value corresponding to the search keyword is selected as the target index hash value; if the hash value corresponding to the search keyword does not exist in the index hash table, the index hash value that is closest to the hash value corresponding to the search keyword is selected as the target index hash value. The retrieval module is used to obtain the retrieval results corresponding to the retrieval keywords based on the target index hash value.
8. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to perform the method described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for performing the method described in any one of claims 1 to 6.
Citation Information
Patent Citations
Hash function-based efficient hierarchical index construction method and retrieval method
CN107273529A
System and method for data indexing and retrieval
US20070136243A1
