Methods for creating and searching document indexes, electronic devices and program products

Abstract

This disclosure provides a method for creating and searching document indexes, an electronic device, and a program product. The index creation method includes: segmenting the content of an original document into characters to obtain initial word segments; for each initial word segment, querying a dictionary word segmentation dictionary that matches the initial word segmentation, wherein the word segmentation dictionary includes a pre-set pinyin dictionary and the latest English dictionary, and the dictionary word segments that match the initial word segmentation constitute the initial word segmentation; if no dictionary word segmentation matching the initial word segmentation is found, adding the initial word segmentation as its matching dictionary word segmentation to the English dictionary to generate the latest English dictionary; obtaining the dictionary word segments that match the initial word segmentation, and establishing a correspondence between the original document and the dictionary word segments based on the dictionary word segments that match the initial word segmentation in the original document, wherein the dictionary word segments in the word segmentation dictionary serve as the index of the original document. This technical solution can improve search performance.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, specifically to a method for creating and searching document indexes, electronic devices, and program products. Background Technology

[0002] With the development of Internet technology, search engines are now widely used for document search services. The primary condition for using search engines is to create an index based on the original document, including every field in the original document so that it can be searched. Since the fewer the number of indexes, the higher the document search performance, how to reduce the number of indexes has become a pressing technical problem to be solved. Summary of the Invention

[0003] To address the problems in the related technologies, embodiments of this disclosure provide a method for creating and searching document indexes, an electronic device, and a program product.

[0004] In a first aspect, this disclosure provides a method for creating a document index.

[0005] Specifically, the method for creating the document index includes:

[0006] The original document's content is segmented into characters to obtain the initial word segments in the original document;

[0007] For each initial word segment, the dictionary segment that matches the initial word segment is queried from the dictionary segmentation dictionary. The dictionary segmentation dictionary includes a pre-set pinyin dictionary and the latest English dictionary. The dictionary segment in the pinyin dictionary is a pre-set single-character pinyin. The dictionary segment that matches the initial word segment constitutes the initial word segment.

[0008] If no dictionary segment matching the initial segment is found, the initial segment is added to the English dictionary as its matching dictionary segment to generate the latest English dictionary;

[0009] Obtain the dictionary segmentation that matches the initial segmentation, and establish a correspondence between the original document and the dictionary segmentation based on the dictionary segmentation that matches the initial segmentation in the original document, wherein the dictionary segmentation in the segmentation dictionary is the index of the original document.

[0010] Secondly, this disclosure provides a document search method, wherein the method uses an index created by any of the above methods to perform document search, and the method includes:

[0011] Obtain a search request, wherein the search request carries a key string for the requested search;

[0012] The key string is segmented into characters to obtain the initial key words;

[0013] For each initial key word segment, query the dictionary word segment that matches the initial key word segment from the dictionary word segment of the pre-stored word segmentation dictionary, wherein the matching dictionary word segment constitutes the initial key word, and the pre-stored word segmentation dictionary includes a pinyin dictionary formed by single-character pinyin and an English dictionary;

[0014] If a dictionary segment that matches the initial key segment is found, the original document corresponding to the key string is determined based on the dictionary segment that matches the initial key segment and the correspondence between the pre-stored original document and the dictionary segment.

[0015] Thirdly, this disclosure provides a document indexing and creation apparatus, comprising:

[0016] The first segmentation module is configured to perform character segmentation on the document content of the original document to obtain the initial word segments in the original document;

[0017] The first query module is configured to query dictionary segmentations that match the initial segmentation from the dictionary segmentation dictionary for each initial segmentation. The dictionary segmentation dictionary includes a pre-set pinyin dictionary and the latest English dictionary. The dictionary segmentations in the pinyin dictionary are pre-set single-character pinyin. The dictionary segmentations that match the initial segmentation constitute the initial segmentation.

[0018] The module is configured to add the initial word segment as its matching dictionary word to the English dictionary if no dictionary word segment matching the initial word segment is found, thereby generating the latest English dictionary.

[0019] The module is configured to obtain the dictionary segmentation matched by the initial segmentation, and establish a correspondence between the original document and the dictionary segmentation based on the dictionary segmentation matched by the initial segmentation in the original document, wherein the dictionary segmentation in the segmentation dictionary is the index of the original document.

[0020] Fourthly, this disclosure provides a document search device, comprising:

[0021] The acquisition module is configured to acquire a search request, wherein the search request carries a key string for the requested search.

[0022] The second segmentation module is configured to perform character segmentation on the key string to obtain initial key words;

[0023] The second query module is configured to, for each initial key word segment, query the dictionary word segmentation that matches the initial key word segment from the dictionary word segmentation of the pre-stored word segmentation dictionary, wherein the matching dictionary word segmentation constitutes the initial key word segmentation, and the pre-stored word segmentation dictionary includes a pinyin dictionary formed by single-character pinyin and an English dictionary;

[0024] The determination module is configured to, if a dictionary segment that matches the initial key segment is found, determine the original document corresponding to the key string based on the dictionary segment that matches the initial key segment and the pre-stored correspondence between the original document and the dictionary segment.

[0025] Fifthly, embodiments of this disclosure provide an electronic device including a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method as described in any one of the first or second aspects.

[0026] In a sixth aspect, embodiments of this disclosure provide a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the method as described in any one of the first or second aspects.

[0027] In a seventh aspect, this disclosure provides a computer program product including computer instructions that, when executed by a processor, implement the method steps as described in any one of the first or second aspects.

[0028] According to the technical solution provided in this disclosure, single-character pinyin is set as the dictionary segmentation word in the pinyin dictionary. When the original document is obtained, the document content can be initially segmented to obtain initial words. For each initial word, a dictionary segmentation word matching the initial word is queried from the dictionary segmentation word in the word segmentation dictionary. The dictionary segmentation words matching the initial word form the initial word. If no dictionary segmentation word matching the initial word is found, the initial word is added to the English dictionary as its matching dictionary segmentation word to generate the latest English dictionary. In this way, the dictionary segmentation word matching the initial word can be directly queried or stored. Thus, a correspondence between the original document and the dictionary segmentation word can be established based on the dictionary segmentation word corresponding to the initial word in the original document, and the dictionary segmentation word can be used as the index of the original document. Since single-character pinyin is used as the index of the original document, compared with the existing method of using pinyin combinations as the index, the number of indexes is effectively reduced, the search efficiency is improved, and the document search performance is enhanced.

[0029] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0030] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:

[0031] Figure 1 A flowchart illustrating a document index creation method according to an embodiment of the present disclosure;

[0032] Figure 2 A flowchart illustrating a document search method according to an embodiment of the present disclosure;

[0033] Figure 3 A structural block diagram of a document indexing and creation apparatus according to an embodiment of the present disclosure is shown;

[0034] Figure 4 A structural block diagram of a document search apparatus according to an embodiment of the present disclosure is shown;

[0035] Figure 5 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0036] Figure 6 A schematic diagram of the structure of a computer system suitable for implementing the method according to embodiments of the present disclosure is shown. Detailed Implementation

[0037] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement them. Furthermore, for clarity, portions unrelated to the description of exemplary embodiments have been omitted from the drawings.

[0038] In this disclosure, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, figures, steps, behaviors, components, parts or combinations thereof disclosed in this specification, and are not intended to exclude the possibility of the presence or addition of one or more other features, figures, steps, behaviors, components, parts or combinations thereof.

[0039] It should also be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0040] In this disclosure, the acquisition of user information or user data is done with the user's authorization or confirmation, or by the user's active choice.

[0041] As mentioned above, with the development of Internet technology, users now generally use search engines to search for documents. The primary condition for using search engines is to create an index based on the original document, including every field in the original document, so that it can be searched.

[0042] This disclosure provides a method for creating document indexes. This method can set up a word segmentation dictionary composed of single-character pinyin, split words that are suspected to be pinyin combinations into single-character pinyin during the word segmentation process, and create the single-character pinyin as the index of the original document. Compared with the existing method of using pinyin combinations as indexes, this method effectively reduces the number of indexes, improves search efficiency, and enhances the search performance of documents.

[0043] Figure 1 A flowchart illustrating a document index creation method according to an embodiment of this disclosure is shown. Figure 1 As shown, the document index creation method includes the following steps S101-S104:

[0044] In step S101, the document content of the original document is processed by character segmentation to obtain the initial word segmentation in the original document;

[0045] In step S102, for each initial word segment, the dictionary word segment that matches the initial word segment is queried from the dictionary word segmentation in the word segmentation dictionary. The word segmentation dictionary includes a pre-set pinyin dictionary and the latest English dictionary. The dictionary word segmentation in the pinyin dictionary is a pre-set single-character pinyin. The dictionary word segmentation that matches the initial word segmentation constitutes the initial word segmentation.

[0046] In step S103, if no dictionary segment matching the initial segment is found, the initial segment is added to the English dictionary as its matching dictionary segment to generate the latest English dictionary.

[0047] In step S104, the dictionary segmentation matching the initial segmentation is obtained, and a correspondence between the original document and the dictionary segmentation is established based on the dictionary segmentation matching the initial segmentation in the original document, wherein the dictionary segmentation in the segmentation dictionary is the index of the original document.

[0048] In one possible implementation, this document indexing method is applicable to devices such as computers, computing devices, servers, and server clusters that can perform document indexing. This document indexing method is primarily used for indexing English documents containing Chinese pinyin, especially Chinese-to-English translations of English documents.

[0049] In one possible implementation, the single-character pinyin refers to the pinyin of a single Chinese character. Although there are many combinations of Chinese pinyin, the number of single-character pinyin is limited to just over 400. For example, the single-character pinyin could include "a", "ai", "an", "ang", etc. This pinyin dictionary could be a dictionary containing 400 single-character pinyin entries as dictionary segments.

[0050] In one possible implementation, when creating an index for a large number of original documents, it is necessary to index each field in the original documents. First, the document content of these original documents, i.e., the fields in the documents, needs to be initially segmented to obtain a set of initial word segments corresponding to each original document.

[0051] In one possible implementation, the word segmentation dictionary includes a pre-set pinyin dictionary and the latest English dictionary. The original documents are mainly English documents containing Chinese pinyin. The initial word segments in these original documents will contain some Chinese pinyin combinations, such as "beijing", "wangjing", "xiehe", etc. Therefore, for each initial word segment, the dictionary word segments that match the initial word segment can be queried. Here, matching means that these dictionary word segments can form the initial key word. For example, the dictionary word segments that match "Beijing" are "Bei" and "jing".

[0052] In one possible implementation, if no dictionary segment matching the initial segment is found, the initial segment can be added to the English dictionary as its matching dictionary segment to generate the latest English dictionary. At this time, the English dictionary stores the dictionary segment matching the initial segment, and the dictionary segment matching the initial segment can be obtained. Furthermore, subsequent initial segments can use the latest English dictionary and the pre-set pinyin dictionary to query the dictionary segment matching the initial segment.

[0053] In one possible implementation, for an original document, the initial word segmentation in the original document can be obtained. If a dictionary word segmentation matching the initial word segmentation is found, the dictionary word segmentation matching the initial word segmentation can be directly obtained. If no dictionary word segmentation matching the initial word segmentation is found, the initial word segmentation can be added to the English dictionary as its matching dictionary word segmentation, and then the dictionary word segmentation matching the initial word segmentation can be obtained. After obtaining the dictionary word segmentation matching the initial word segmentation in the original document, the correspondence between the original document and the dictionary word segmentation can be established. The dictionary word segmentation is used as the index of the original document. When the user enters an English word or single-character pinyin from the dictionary word segmentation to search, the original document containing the English word or single-character pinyin can be found.

[0054] For example, for the original document 1 "Beijing Capital International Airport", initial character segmentation can be performed to obtain the initial word segments "Beijing", "Capital", "International", and "Airport". The dictionary word segmentation dictionary is then used to query for matching words that match these initial word segments. For the initial word segment "Beijing", the dictionary word segments matching "Beijing" are "Bei" and "jing". For the initial word segment "Capital", there is no matching word segment in the dictionary. Therefore, the initial word segment "Capital" can be added to the dictionary as its matching word, i.e., the dictionary word segment "Capital" is added to the English dictionary. Following this logic, we can obtain a word segmentation dictionary including a pinyin dictionary and the latest English dictionary, specifically one containing "Capital," "International," and "Airport." Simultaneously, we obtain the dictionary segmentations corresponding to the initial segmentations "Beijing," "Capital," "International," and "Airport" in the original document 1 as "Bei," "jing," "Capital," "International," and "Airport." Thus, we can determine that the dictionary segmentations corresponding to the original document 1 are "Bei," "jing," "Capital," "International," and "Airport." When a user enters "Airport" as a search term, they can find original document 1 containing "Airport."

[0055] This implementation can set single-character pinyin as the dictionary segmentation term in the pinyin dictionary. When the original document is obtained, the document content can be initially segmented to obtain initial words. For each initial word, a dictionary segmentation term matching the initial word is queried from the dictionary segmentation term in the word segmentation dictionary. The dictionary segmentation terms matching the initial word form the initial word. If no dictionary segmentation term matching the initial word is found, the initial word is added to the English dictionary as its matching dictionary segmentation term to generate the latest English dictionary. In this way, the dictionary segmentation term matching the initial word can be directly queried or stored. Thus, a correspondence between the original document and the dictionary segmentation term can be established based on the dictionary segmentation term corresponding to the initial word in the original document, and the dictionary segmentation term can be used as the index of the original document. Since single-character pinyin is used as the index of the original document, compared with the existing method of using pinyin combinations as the index, the number of indexes is effectively reduced, search efficiency is improved, and document search performance is enhanced.

[0056] In one possible implementation, the above index creation method, wherein performing character segmentation on the document content of the original document to obtain the initial word segments in the original document, includes:

[0057] The original document's content is segmented according to spaces to obtain the initial word segments of the original document.

[0058] In this implementation, since spaces are used as natural delimiters between words in English documents, the content of the original document can be segmented according to spaces to obtain the initial word segments. The following uses original document 1: “Beijing Capital International Airport” and original document 2: “Wangjing SOHO” as examples. The initial word segments corresponding to original document 1 and original document 2 are shown in the table below:

[0059]

[0060] Table 1

[0061] In one possible implementation, the step of querying the dictionary segmentation in the word segmentation dictionary to find the dictionary segmentation that matches the initial segmentation includes:

[0062] The initial word segment is segmented using the dictionary segmentation in the Pinyin dictionary. If the segmentation is successful, the dictionary segmentation that matches the initial word segmentation is retrieved.

[0063] If word segmentation fails, then query the current English dictionary for dictionary segmentation that matches the initial segmentation.

[0064] In this implementation, the forward maximum matching segmentation is commonly referred to as the MM method. Its basic idea is as follows: using all fields from the initial segmentation as matching fields, a search is performed in the pinyin dictionary. If a dictionary segmentation exists in the pinyin dictionary and matches the initial segmentation, the match is successful, and segmentation is directly successful, allowing the acquisition of the dictionary segmentation matched by the initial segmentation. If no dictionary segmentation consisting of all the letters of the initial segmentation is found in the segmentation dictionary, the match fails. The last letter of the initial segmentation is removed, and the remaining letter string is re-matched. This process continues until a match is successful or the length of the remaining string is zero. If no match is found by the time the length of the remaining string is zero, segmentation fails and ends. If a match is successful, a dictionary segmentation matching the initial segmentation is obtained. The unmatched letter string is then used as the matching field for forward maximum matching segmentation. If all strings in the initial segmentation match successfully, segmentation is successful; if some strings in the initial segmentation match successfully while others do not, segmentation fails.

[0065] For example, let's take "Beijing" as the initial word segmentation as an example:

[0066] First round of scanning:

[0067] If we select the seven letters "Beijing" from front to back in the forward sequence, there is no corresponding word segment in the word segmentation dictionary, so the match fails.

[0068] Removing one letter from "Beijin" results in a match failing because there is no corresponding word in the word segmentation dictionary.

[0069] Removing one letter from "Beiji" results in a match failing because there is no corresponding word in the word segmentation dictionary.

[0070] Removing one letter "Beij" results in a match failing because there is no corresponding word in the word segmentation dictionary.

[0071] Removing one letter "Bei" results in a matching word in the word segmentation dictionary;

[0072] The first round of scanning terminates, and a dictionary segment "Bei" matching the initial segment is obtained. The dictionary segment "Bei" is removed, and the second round of scanning begins.

[0073] Second round of scanning:

[0074] After removing "Bei", the remaining four letters "jing" are still selected as the matching field for matching. The word segmentation dictionary has the corresponding word segmentation, and the match is successful.

[0075] At this point, the initial document has been scanned. The word segmentation of "Beijing" was successfully performed using positive maximum matching. The dictionary segmentation results for the initial word segmentation are "Bei" and "jing".

[0076] In this implementation, if no dictionary segment in the Pinyin dictionary matches the matching field in the initial segmentation during each round of scanning, it indicates that the segmentation has failed and no dictionary segment matching the initial segmentation has been found. In this case, the latest English dictionary can be used to check if there is a dictionary segment matching the initial segmentation. If there is, the dictionary segment matching the initial segmentation has been found; if not, no dictionary segment matching the initial segmentation has been found. In this case, the initial segmentation can be added to the English dictionary as its matching dictionary segmentation to generate the latest English dictionary.

[0077] Using original document 1, “Beijing Capital International Airport”, and original document 2, “Wangjing SOHO”, as examples, the dictionary segmentation results for the initial word segmentation matching in original document 1 and original document 2 are shown in the table below:

[0078]

[0079] Table 2

[0080] In one possible implementation, the word segmentation dictionary further includes word segmentation identifiers corresponding to the dictionary word segments, and the step of establishing a correspondence between the original document and the dictionary word segments based on the dictionary word segments corresponding to the initial word segments in the original document includes:

[0081] Based on the dictionary segmentation corresponding to the initial segmentation in the original document and the segmentation identifier corresponding to the dictionary segmentation, a correspondence is established between the document identifier of the original document and the segmentation identifier.

[0082] In this embodiment, to facilitate querying and recording, the word segmentation dictionary can also use word segmentation identifiers to identify the corresponding dictionary words. For example, the word segmentation identifier can be a dictionary serial number, and the original document can be identified by a document identifier. Establishing a correspondence between the original document and the dictionary words is equivalent to establishing a correspondence between the document identifier of the original document and the word segmentation identifier of the dictionary words.

[0083] In one possible implementation, establishing the correspondence between the original document and the dictionary word segmentation includes:

[0084] An inverted index data storage structure is used to record the correspondence between the original document and the dictionary word segmentation.

[0085] In this implementation, the inverted index, also known as a reverse index, inverted archive, or reverse archive, is an indexing method used to store a mapping of the storage location of a word within a document or a group of documents in a full-text search. It is a data structure in a document retrieval system that records the correspondence between the original document and the dictionary word segmentation through the inverted index's data storage structure. It allows for the rapid retrieval of a list of documents containing a search term. For example, the data storage structure of this inverted index can be shown in Table 3 below:

[0086] bei Document 1 jing Document 1, Document 2 wang Document 2 capital Document 1 international Document 1 airport Document 1 soho Document 2

[0087] Table 3

[0088] Figure 2 A flowchart illustrating a document search method according to an embodiment of this disclosure is shown. Figure 2 As shown, the document search method includes the following steps S201-S204:

[0089] In step S201, a search request is obtained, wherein the search request carries a key string for the requested search.

[0090] In step S202, the key string is processed by character segmentation to obtain initial key words;

[0091] In step S203, for each initial key word segment, the dictionary word segment that matches the initial key word segment is queried from the dictionary word segment of the pre-stored word segmentation dictionary, wherein the matching dictionary word segment constitutes the initial key word, and the pre-stored word segmentation dictionary includes a pinyin dictionary formed by single-character pinyin and an English dictionary;

[0092] In step S204, if a dictionary segment that matches the initial key segment is found, the original document corresponding to the key string is determined based on the dictionary segment that matches the initial key segment and the pre-stored correspondence between the original document and the dictionary segment.

[0093] In one possible implementation, this document search method is applicable to devices such as computers, computing devices, servers, and server clusters capable of performing document searches. This document search method primarily relies on the index created by the aforementioned document indexing method to perform document searches.

[0094] In one possible implementation, when a user searches for the original document they need using a keyword string, they can enter a search request on the client side. This search request carries the keyword string that the user is searching for. The client can then send the search request to the server, which will receive the search request sent by the client.

[0095] In one possible implementation, when the original document searched by the user is an English document, the input key string is an English keyword. This English keyword is usually separated by spaces. Therefore, after receiving the search request, the server can use spaces to segment the key string to obtain initial key words. The key string can correspond to one or more initial key words. For example, if the key string in the search request is "Beijing airport", segmenting it by spaces can yield the initial key words "Beijing" and "airport"; if the key string in the search request is "airport", the initial key word "airport" can be obtained.

[0096] In one possible implementation, the pre-stored word segmentation dictionary refers to the word segmentation dictionary generated during the index creation process, which includes a pinyin dictionary formed by single-character pinyin and an English dictionary gradually generated based on English words in the original document during index creation.

[0097] In one possible implementation, for each initial key segment, the dictionary segments that match the initial key segment are queried. Here, matching means that these dictionary segments can form the initial key segment. For example, the dictionary segments that match "Beijing" are "Bei" and "jing".

[0098] In one possible implementation, if no dictionary segment matching the initial key segment is found, it means that the initial key segment does not appear in the original document, and no corresponding original document can be found. In this case, a search result indicating that no relevant document was found can be sent to the client. If a dictionary segment matching the initial key segment is found, it means that the initial key segment appears in the original document. In this case, the original document corresponding to the dictionary segment matching the initial key segment can be obtained based on the dictionary segment matching the initial key segment and the pre-stored correspondence between the original document and the dictionary segment, thus obtaining the original document corresponding to the key string.

[0099] For example, taking the inverted index shown in Table 3 above as an example, if the user inputs the key string "airport", the initial key word can be obtained as "airport", and the dictionary word "airport" matching "airport" can be found from the pre-stored word segmentation dictionary. At this time, according to the correspondence shown in Table 3, the original document corresponding to the key string "airport" can be determined to be document 1.

[0100] This implementation uses a word segmentation dictionary, including a pinyin dictionary formed from single-character pinyin and an English dictionary, as an index. After segmenting the key string to be searched to obtain initial key segments, it queries the word segmentation dictionary to see if there are dictionary segments that can form the initial key segment. If found, it means that the initial key segment appears in the original document. At this time, the original document corresponding to the key string can be determined based on the dictionary segments matched by the initial key segment and the pre-stored correspondence between the original document and the dictionary segments. Since single-character pinyin and English words are used as the index for the original document during the search, compared with the existing method of using pinyin combinations as the index, the number of indexes is effectively reduced, the search efficiency is improved, and the document search performance is enhanced.

[0101] In one possible implementation, the step of querying the dictionary segmentation that matches the initial key segmentation from the pre-stored dictionary segmentation dictionary may include the following steps:

[0102] The initial key word is segmented using the single-character pinyin from the pinyin dictionary with maximum positive matching. If the segmentation is successful, the dictionary segmentation that matches the initial key word is obtained.

[0103] If word segmentation fails, the English dictionary is queried to obtain dictionary-segmented words that match the initial key word segmentation.

[0104] In this implementation, the initial key word can first be segmented using a pinyin dictionary composed of single-character pinyin. If the forward maximum matching is successful, it means the initial key word can be composed of single-character pinyin from the pinyin dictionary. In this case, the dictionary segment matching the initial key word, i.e., the single-character pinyin, can be directly obtained. If the forward maximum matching fails, it means the initial key word cannot be composed of single-character pinyin from the pinyin dictionary, and the initial key word may be an English word. In this case, the English dictionary can be queried in whole words to obtain the dictionary segment matching the initial key word. If no dictionary segment matching the initial key word is found, it means the initial key word does not appear in the original document, and the corresponding original document cannot be found. In this case, the search results indicating no relevant document was found can be sent to the client. If a dictionary segment matching the initial key word is found, the dictionary segment matching the initial key word is obtained.

[0105] In one possible implementation, the step of determining the original document corresponding to the key string based on the dictionary segmentation matched by the initial key segmentation and the pre-stored correspondence between the original document and the dictionary segmentation includes:

[0106] Based on the dictionary segmentation matched by the initial key segmentation and the correspondence between the pre-stored original documents and the dictionary segmentation, determine the candidate original documents corresponding to each dictionary segmentation in the key string;

[0107] Based on the dictionary segmentation order of the key string, the original document corresponding to the key string is determined from the candidate original documents.

[0108] In this implementation, the key string is segmented into one or more initial key segments. Each initial key segment can match one or more dictionary segments. Therefore, the key string will contain one or more dictionary segments. Each dictionary segment will correspond to one or more original documents as shown in Table 3. It is necessary to obtain the candidate original documents corresponding to each dictionary segment in the key string from these original documents. Only in this way will the candidate original documents include all the dictionary segments in the key string.

[0109] For example, assuming the key string carried in the search request is "Beijing airport", the initial key segments "Beijing" and "airport" can be obtained. Then, the dictionary segments matching the initial key segment "Beijing" are "Bei" and "jing", and the dictionary segment matching the initial key segment "airport" is "airport". Thus, each dictionary segment in the key string is obtained as "Bei", "jing", and "airport". According to the correspondence between dictionary segments and original documents shown in Table 3, "Bei"-document1, "jing"-document1 and document2, and "airport"-document1 can be obtained. Thus, the candidate original document-document1 can be obtained.

[0110] In this implementation, the acquired candidate original documents include all dictionary segments of the key string. To ensure more accurate search results, the target original document corresponding to the key string is determined from the candidate original documents according to the order of the dictionary segments in the key string. For example, assuming the original documents contain document 3: "airport……Beijing", and taking the key string "Beijing airport" in the search request as an example, the candidate original documents will include document 1 and document 3. However, the order of the matched dictionary segments in document 1 is "Bei", "jing", and "airport", which is the same as the order of the dictionary segments in the key string. The order of the matched dictionary segments in document 3 is "airport" separated by some characters "Bei" and "jing". Thus, the target original document, i.e., document 1, with the same order of dictionary segments in the key string can be selected from the candidate original documents.

[0111] Of course, it should be noted here that the candidate original documents corresponding to each dictionary segment of the key string can be used as the target original documents, or the original documents corresponding to each dictionary segment of the key string can be used as the target original documents corresponding to the key string individually, to ensure the range of original documents found and avoid omissions. However, these target original documents can be displayed and sorted. Candidate original documents with the same dictionary segmentation order as those in the key string are placed before other candidate original documents, and other target original documents are placed after the candidate original documents. This way, the client can send this sorting information to the client when displaying search results, so that the client can display the searched target original documents according to this sorting, ensuring that the target original documents ranked higher are more closely matched with the key string.

[0112] Figure 3 A structural block diagram of a document indexing and creation apparatus according to embodiments of the present disclosure is shown. This apparatus can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 3 As shown, the document indexing and creation device includes:

[0113] The first segmentation module 301 is configured to perform character segmentation on the document content of the original document to obtain the initial word segments in the original document;

[0114] The first query module 302 is configured to query dictionary segmentations that match the initial segmentation from the dictionary segmentation dictionary for each initial segmentation. The dictionary segmentation dictionary includes a pre-set pinyin dictionary and the latest English dictionary. The dictionary segmentations in the pinyin dictionary are pre-set single-character pinyin. The dictionary segmentations that match the initial segmentation constitute the initial segmentation.

[0115] The module 303 is configured to add the initial word segment as its matching dictionary word to the English dictionary if no dictionary word segment matching the initial word segment is found, thereby generating the latest English dictionary.

[0116] The module 304 is configured to obtain the dictionary segmentation matched by the initial segmentation, and establish a correspondence between the original document and the dictionary segmentation based on the dictionary segmentation matched by the initial segmentation in the original document, wherein the dictionary segmentation in the segmentation dictionary is the index of the original document.

[0117] In one possible implementation, the first query module 302 is configured as follows:

[0118] The initial word segment is segmented using the dictionary segmentation in the Pinyin dictionary. If the segmentation is successful, the dictionary segmentation that matches the initial word segmentation is retrieved.

[0119] If word segmentation fails, then query the latest English dictionary for dictionary segmentation that matches the initial segmentation.

[0120] In one possible implementation, the word segmentation dictionary further includes word segmentation identifiers corresponding to the word segmentation in the dictionary, and the establishment module is configured to:

[0121] Based on the dictionary segmentation matched by the initial segmentation in the original document and the segmentation identifier corresponding to the dictionary segmentation, a correspondence is established between the document identifier of the original document and the segmentation identifier.

[0122] In one possible implementation, the segmentation module is configured as follows:

[0123] The original document's content is segmented according to spaces to obtain the initial word segments of the original document.

[0124] In one possible implementation, the part of the establishment module that establishes the correspondence between the original document and the dictionary word segmentation is configured as follows:

[0125] An inverted index data storage structure is used to record the correspondence between the original document and the dictionary word segmentation.

[0126] Figure 4 A structural block diagram of a document search device according to an embodiment of the present disclosure is shown. This device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 4 As shown, the document search device includes:

[0127] The acquisition module 401 is configured to acquire a search request, wherein the search request carries a key string for the requested search.

[0128] The second segmentation module 402 is configured to perform character segmentation on the key string to obtain initial key words;

[0129] The second query module 403 is configured to, for each initial key word segment, query the dictionary word segmentation that matches the initial key word segment from the dictionary word segmentation of the pre-stored word segmentation dictionary, wherein the matching dictionary word segmentation constitutes the initial key word segmentation, and the pre-stored word segmentation dictionary includes a pinyin dictionary formed by single-character pinyin and an English dictionary;

[0130] The determination module 404 is configured to, if a dictionary segment that matches the initial key segment is found, determine the original document corresponding to the key string based on the dictionary segment that matches the initial key segment and the pre-stored correspondence between the original document and the dictionary segment.

[0131] In one possible implementation, the second query module 403 is configured as follows:

[0132] The initial key word is segmented using the single-character pinyin from the pinyin dictionary. If the segmentation is successful, the dictionary segmentation that matches the initial key word is obtained.

[0133] If word segmentation fails, the English dictionary is queried to obtain dictionary-segmented words that match the initial key word segmentation.

[0134] In one possible implementation, the determining module 404 is configured to:

[0135] Based on the dictionary segmentation matched by the initial key segmentation and the correspondence between the pre-stored original documents and the dictionary segmentation, determine the candidate original documents corresponding to each dictionary segmentation in the key string;

[0136] Based on the order of dictionary segmentation in the key string, the target original document corresponding to the key string is determined from the candidate original documents.

[0137] The technical terms and features mentioned in this device implementation are the same or similar. For the explanation and description of the technical terms and features involved in this device, please refer to the explanation of the above method implementation, which will not be repeated here.

[0138] This disclosure also discloses an electronic device. Figure 5 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0139] like Figure 5 As shown, the electronic device 500 includes a memory 501 and a processor 502, wherein the memory 501 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 502 to implement the method according to embodiments of the present disclosure.

[0140] Figure 6 A schematic diagram of the structure of a computer system suitable for implementing the method according to embodiments of the present disclosure is shown.

[0141] like Figure 6As shown, the computer system 600 includes a processing unit 601, which can execute various processes described in the above embodiments according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the computer system 600. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0142] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed. The processing unit 601 can be implemented as a CPU, GPU, TPU, FPGA, NPU, etc.

[0143] In particular, according to embodiments of this disclosure, the methods described above can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising computer instructions that, when executed by a processor, implement the steps of the methods described above. In such embodiments, the computer program product can be downloaded and installed from a network via communication section 609, and / or installed from removable media 611.

[0144] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0145] The units or modules described in the embodiments of this disclosure can be implemented in software or programmable hardware. The described units or modules can also be located in a processor, and the names of these units or modules do not necessarily constitute a limitation on the unit or module itself.

[0146] In another aspect, this disclosure also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system described above; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to perform the methods described in this disclosure.

[0147] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A method for creating a document index, comprising: The original document's content is segmented into characters to obtain the initial word segments in the original document; For each initial word segment, the dictionary word segment that matches the initial word segment is queried from the dictionary word segmentation in the word segmentation dictionary. The word segmentation dictionary includes a pre-set pinyin dictionary and the latest English dictionary. The dictionary word segmentation in the pinyin dictionary is a pre-set single-character pinyin, where single-character pinyin refers to the pinyin of a single Chinese character. The dictionary word segmentation that matches the initial word segment constitutes the initial word segment. If no dictionary segment matching the initial segment is found, the initial segment is added to the English dictionary as its matching dictionary segment to generate the latest English dictionary; Obtain the dictionary segmentation that matches the initial segmentation, and establish a correspondence between the original document and the dictionary segmentation based on the dictionary segmentation that matches the initial segmentation in the original document, wherein the dictionary segmentation in the segmentation dictionary is the index of the original document.

2. The method according to claim 1, wherein, The step of querying the dictionary segmentation in the word segmentation dictionary to find the dictionary segmentation that matches the initial word segmentation includes: The initial word segment is segmented using the dictionary segmentation in the Pinyin dictionary. If the segmentation is successful, the dictionary segmentation that matches the initial word segmentation is retrieved. If word segmentation fails, then query the latest English dictionary for dictionary segmentation that matches the initial segmentation.

3. The method according to claim 1, wherein, The word segmentation dictionary also includes word segmentation identifiers corresponding to the word segmentations in the dictionary. The step of establishing a correspondence between the original document and the word segmentations in the dictionary based on the initial word segmentations in the original document includes: Based on the dictionary segmentation matched by the initial segmentation in the original document and the segmentation identifier corresponding to the dictionary segmentation, a correspondence is established between the document identifier of the original document and the segmentation identifier.

4. The method according to claim 1, wherein, The step of performing character segmentation on the content of the original document to obtain the initial word segments in the original document includes: The original document's content is segmented according to spaces to obtain the initial word segments of the original document.

5. The method according to claim 1, wherein, Establishing the correspondence between the original document and the dictionary word segmentation includes: An inverted index data storage structure is used to record the correspondence between the original document and the dictionary word segmentation.

6. A document search method, wherein, The method uses an index created by any one of claims 1 to 5 to perform document search, the method comprising: Obtain a search request, wherein the search request carries a key string for the requested search; The key string is segmented into characters to obtain the initial key words; For each initial key word segment, query the dictionary word segment that matches the initial key word segment from the dictionary word segment of the pre-stored word segmentation dictionary, wherein the matching dictionary word segment constitutes the initial key word, and the pre-stored word segmentation dictionary includes a pinyin dictionary formed by single-character pinyin and an English dictionary; If a dictionary segment that matches the initial key segment is found, the original document corresponding to the key string is determined based on the dictionary segment that matches the initial key segment and the correspondence between the pre-stored original document and the dictionary segment.

7. The method according to claim 6, wherein, The step of querying the dictionary segmentation that matches the initial key segmentation from the pre-stored dictionary segmentation dictionary includes: The initial key word is segmented using the single-character pinyin from the pinyin dictionary. If the segmentation is successful, the dictionary segmentation that matches the initial key word is obtained. If word segmentation fails, the English dictionary is queried to obtain dictionary-segmented words that match the initial key word segmentation.

8. The method according to claim 6 or 7, wherein, The step of determining the original document corresponding to the key string based on the dictionary segmentation matched by the initial key segmentation and the pre-stored correspondence between the original document and the dictionary segmentation includes: Based on the dictionary segmentation matched by the initial key segmentation and the correspondence between the pre-stored original documents and the dictionary segmentation, determine the candidate original documents corresponding to each dictionary segmentation in the key string; Based on the order of dictionary segmentation in the key string, the target original document corresponding to the key string is determined from the candidate original documents.

9. An electronic device comprising a memory and a processor; wherein, The memory is used to store one or more computer instructions, which are executed by the processor to implement the steps of the method according to any one of claims 1 to 8.

10. A computer program product comprising computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images