A hybrid information retrieval method and device

Abstract

This invention relates to a hybrid information retrieval method and apparatus. The method includes: acquiring user input information; decomposing the input information into multiple word units and identifying the word unit category of each word unit, wherein the word unit is the smallest semantic unit; analyzing the word units according to the word unit categories to obtain multiple word unit analysis results; generating a query statement based on the multiple word unit analysis results; and querying a data table according to the query statement to obtain query results, wherein the data table includes multiple data entries, each data entry including keywords of multiple word unit types. This invention improves the retrieval hit rate by decomposing and analyzing the input information into word units, and then performing hybrid retrieval based on different word units. It supports mixed input of Chinese characters, full pinyin, and initial letters, and is case-insensitive.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a hybrid information retrieval method and apparatus. Background Technology

[0002] Traditional database retrieval relies on the field content stored in the database. If Chinese characters are stored, only Chinese characters can be used for retrieval; if Pinyin is stored, only Pinyin can be used for retrieval. If the user's search information contains both Chinese characters and Pinyin, a mixed search cannot be completed. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to overcome the shortcomings of the prior art and provide a hybrid information retrieval method and apparatus.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A hybrid information retrieval method includes:

[0006] Obtain user input information;

[0007] The input information is decomposed into multiple word units, and the word unit category of each word unit is identified, wherein the word unit is the smallest semantic unit;

[0008] The lexical elements are analyzed according to the lexical category to obtain multiple lexical analysis results;

[0009] Generate a query statement based on the results of the multiple word element analysis;

[0010] The query is performed on the data table according to the query statement to obtain the query results. The data table includes multiple data entries, and each data entry includes keywords of various word types.

[0011] Optionally, the word type includes Chinese, full pinyin, and abbreviated pinyin.

[0012] Optionally, the step of decomposing the input information into multiple tokens specifically includes:

[0013] The input information is decomposed into multiple tokens using regular expressions;

[0014] Determine whether the word type of the word is full pinyin. If not, continue to decompose the word type of the word that is full pinyin.

[0015] Optionally, the step of analyzing the lexical based on the lexical category specifically includes:

[0016] Define query conditions, which include: full Pinyin search, Chinese search, abbreviated Pinyin search, Chinese filtering, and full Pinyin filtering;

[0017] Based on the lexical type and the query conditions, multiple query fields are generated.

[0018] Optionally, the step of generating multiple query fields based on the lexical type and the query conditions specifically includes:

[0019] When the word element category is abbreviation, the abbreviation search is used to search for the word element;

[0020] When the word element category is full pinyin, the word element is searched using the full pinyin search and the simplified pinyin search respectively, and the word element is filtered using the full pinyin filter;

[0021] When the word element category is Chinese, the word element is searched using the full pinyin search, the Chinese search, and the simplified pinyin search respectively, while the word element is filtered using the Chinese filtering.

[0022] Optionally, generating a query statement based on the multiple lexical analysis results specifically includes:

[0023] The type of the input information is determined, and the type of the input information is the lowest level word type in the input information, wherein the level of the Chinese is higher than that of the full pinyin and the abbreviated pinyin, and the level of the full pinyin is higher than that of the abbreviated pinyin;

[0024] A query statement is generated based on the type of input information and the results of the multiple lexical analysis.

[0025] Optionally, generating a query statement based on the type of the input information and the results of the multiple lexical analysis specifically includes:

[0026] When the type of input information is Chinese, a query is performed based on the Chinese search.

[0027] Optionally, generating a query statement based on the type of the input information and the results of the multiple lexical analysis specifically includes:

[0028] When the type of input information is Pinyin, a query is performed based on the Pinyin search and the Chinese filtering.

[0029] Optionally, generating a query statement based on the type of the input information and the results of the multiple lexical analysis specifically includes:

[0030] When the type of input information is abbreviated pinyin, the query is performed based on the abbreviated pinyin search, the full pinyin filter, and the Chinese character filter.

[0031] A hybrid information retrieval device, the device comprising:

[0032] The information acquisition module is used to acquire user input information;

[0033] The lexical decomposition module is used to decompose the input information into multiple lexical units and identify the lexical unit category of each lexical unit, wherein the lexical unit is the smallest semantic unit;

[0034] The lexical analysis module is used to analyze the lexical based on the lexical category and obtain multiple lexical analysis results;

[0035] The query statement generation module is used to generate query statements based on the results of the multiple word element analysis.

[0036] The information query module is used to query the data table according to the query statement and obtain the query results. The data table includes multiple data entries, and each data entry includes keywords of various word types.

[0037] The technical solution provided by this invention decomposes and analyzes the input information into lexical units, and then performs mixed retrieval based on different lexical units. It supports mixed input of Chinese characters, full pinyin, and initial letters, and does not distinguish between uppercase and lowercase letters, thereby improving the retrieval hit rate. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart of a first embodiment of the hybrid information retrieval method of the present invention;

[0040] Figure 2 This is a schematic diagram of the structure of a first embodiment of the hybrid information retrieval device of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0042] This invention provides a hybrid information retrieval method. Hybrid information can be understood as including at least two different types of information, which can be Chinese characters, full pinyin, and abbreviated pinyin. Abbreviated pinyin can be understood as the first letters of the pinyin.

[0043] In order to achieve hybrid information retrieval, first, a data table corresponding to the pinyin of the basic data needs to be prepared, which stores Chinese keywords, full pinyin keywords, and abbreviated pinyin keywords respectively, as shown in Table 1.

[0044] Table 1

[0045] id Chinese keyword (chinese) Full Pinyin Keyword (whole) Abbreviated keyword (acronym) 1 China Aerospace China Hangtian zght 2 Beijing Aerospace AVIC beijinghangtianaiwei bjhtaw 3 database shujuku sjk

[0046] As can be seen from Table 1, each data includes three forms: "chinese", "whole", and "acronym", which represent Chinese, full pinyin, and abbreviated pinyin respectively. When retrieving information, hybrid information can be retrieved.

[0047] Figure 1 This is the flowchart of the first embodiment of the hybrid information retrieval method in this specification. From a program perspective, the execution entity of the process can be a program or an application client running on an application server.

[0048] As Figure 1 shown, this process may include the following steps:

[0049] Step 101: Obtain the input information of the user.

[0050] In this solution, obtaining the input information of the user can be a single type of information or multiple types of information. The types of information can include Chinese and pinyin.

[0051] After the terminal obtains the information input by the user in the input box, it can perform retrieval on the terminal or send the input information to the server for retrieval.

[0052] Step 102: Decompose the input information into multiple tokens and identify the token category of each token, where the token is the smallest semantic unit.

[0053] In this solution, tokens and token types need to be defined before decomposing the input information into tokens. Among them, a token (Lexeme) is the smallest semantic unit, and the token types can include: Chinese, full pinyin, abbreviated pinyin (the first letter of the pinyin). Among them, the level of the token type: Chinese > full pinyin > abbreviated pinyin, and the sentence (input information) type is the lowest-level token type in the sentence.

[0054] In addition, regular expressions can be used to decompose the target into tokens. If the input information of the user is: zhongg航t, use regular expressions to decompose the target into tokens and identify the token types, as shown in Table 2 below.

[0055] Table 2

[0056] Lexicon Lexical types zhong full pinyin g Simplified Pinyin Flight Chinese t Simplified Pinyin

[0057] It should be noted that using regular expressions may extract non-existent pinyin combinations. For example, "jvao" should be directly decomposed into "j", "v", "a", "o". The solution is as follows: Enumerate all pinyin combinations and determine whether the decomposed pinyin belongs to the combination. The pinyin table is shown in Table 3.

[0058] Table 3 Pinyin Table

[0059]

[0060]

[0061] The above technical features can be summarized as follows: decomposing the input information into multiple lexemes, specifically including:

[0062] Using regular expressions to decompose the input information into multiple lexemes;

[0063] Judging whether the lexeme with the type of full pinyin is correct. If not, continue to decompose the lexeme with the type of full pinyin.

[0064] Step 103: Analyze the lexemes according to the lexeme category to obtain multiple lexeme analysis results.

[0065] In this solution, Step 103 may specifically include:

[0066] Step A: Define query conditions, where the query conditions include: wholeSearch (full pinyin search), chineseSearch (Chinese search), acronymSearch (abbreviated pinyin search), chineseFilter (Chinese filter), and pinyinFilter (pinyin filter); lexemeType is the specified lexeme type for search and must be searched according to the lowest-level lexeme type of the sentence.

[0067] For example: '修改' --> Chinese (word type of the input information); 'xiu改' --> full pinyin (word type of the input information); '修g' --> abbreviated pinyin (word type of the input information); 'zhongg航t' --> abbreviated pinyin (word type of the input information)

[0068] Among them, the three parameters ending with "Search" are used for like operations in SQL and can hit the index. Since the sentence entered by the user may contain Chinese or pinyin, filtering needs to be performed on these two types. chineseFilter and pinyinFilter correspond to the Chinese filter and the pinyin filter respectively.

[0069] For example: If the user enters 'xiug', and searches using the lowest word level, it is like '%xg%'. When searching like this, there is no way to exclude 'Shoe Cabinet', so the chineseFilter and pinyinFilter need to be used for filtering. That is, append '%xiu%' to the chineseFilter and '%xiu%' to the pinyinFilter, so that 'Shoe Cabinet' will not be searched.

[0070] Step B: Generate multiple query fields according to the token type of the token and the query condition, specifically including:

[0071] When the token category of the token is simple pinyin, search the token using the simple pinyin search.

[0072] When the token category of the token is full pinyin, search the token using the full pinyin search and the simple pinyin search respectively, and filter the token using the full pinyin filter.

[0073] When the token category of the token is Chinese, search the token using the full pinyin search, the Chinese search and the simple pinyin search respectively, and at the same time filter the token using the Chinese filter.

[0074] Specifically, when the current token type is Chinese, append the token content to the chineseSearch field, append the token content to the chineseFilter field, convert the token content to full pinyin, and append it to the wholeSearch field, and append the first letter of the full pinyin to the acronymSearch field. [[ID=?]] [[ID=?]]

[0075] When the current token type is full pinyin, append the token content to the wholeSearch field, append the token content to the pinyinFilter field, and append the first letter of the full pinyin to the acronymSearch field.

[0076] When the current token type is simple pinyin, append the token content to the acronymSearch field.

[0077] Continuing with the above example, the token analysis for "zhongg hang t" is as follows:

[0078] "zhong" is full pinyin, the wholeSearch field is "%zhong%", the pinyinFilter field is "%zhong%", and the acronymSearch field is "%z%";

[0079] "g" is simple pinyin, and the acronymSearch field is "%zg%";

[0080] "Hang" is in Chinese, and its full pinyin is "hang". The chineseSearch field is "%hang%", the chineseFilter field is "%hang%", the wholeSearch field is "%zhong%hang%", and the acronymSearch field is "%zgh%";

[0081] "t" is in simplified pinyin, and the acronymSearch field is "%zght%".

[0082] Step 104: Generate a query statement based on the multiple token analysis results, which may specifically include:

[0083] Determine the type of the input information. The type of the input information is the lowest-level token type in the input information. Among them, the level of the Chinese is higher than that of the full pinyin and the simplified pinyin, and the level of the full pinyin is higher than that of the simplified pinyin;

[0084] Generate a query statement according to the type of the input information and the multiple token analysis results.

[0085] It can be divided into the following three cases:

[0086] The first case: When the type of the input information is Chinese, query according to the Chinese search. The query statement can be: select*from pinyinword where 1=1and chinese like#{chineseSearch}.

[0087] The second case: When the type of the input information is full pinyin, query according to the full pinyin search and the Chinese filter. The query statement can be: select*from pinyinword where 1=1and whole like#{wholeSearch}and chinese like#{chineseFilter}.

[0088] The third case: When the type of the input information is simplified pinyin, query according to the simplified pinyin search, the full pinyin filter and the Chinese filter. The query statement can be: select*from pinyinword where 1=1andacronym like#{acronymSearch}and chinese like#{chineseFilter}and whole like#{pinyinFilter}.

[0089] Continuing with the above example, if the lowest-level token type of the SQL statement for "zhongg hangt" is an abbreviated spelling, then the query statement is: select * from pinyinword where 1 = 1 and acronym like '%zght%' and chineselike '%hang%' and whole like '%zhong%hang%'.

[0090] Step 105: Query from the data table according to the query statement to obtain a query result. The data table includes multiple pieces of data, and each piece of data includes keywords of multiple token types.

[0091] According to the query statement in step 104, the query result for "zhongg hangt" is shown in the following table.

[0092] serial number Chinese keyword (chinese) Full Pinyin Keyword (whole) Abbreviated keyword (acronym) 1 China Aerospace China Hangtian zght

[0093] Figure 1 The method, through token decomposition and analysis of the input information, and then mixed retrieval based on different tokens, supports mixed input of Chinese, full spelling, and initials, and is not case-sensitive, improving the retrieval hit rate.

[0094] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method. Figure 2 For the embodiments of this specification, corresponding to Figure 1 is a structural schematic diagram of a hybrid information retrieval device. As Figure 2 shown, the device may include:

[0095] An information acquisition module 201 for acquiring the input information of the user;

[0096] A token decomposition module 202 for decomposing the input information into multiple tokens and identifying the token category of each token, where the token is the smallest semantic unit;

[0097] A token analysis module 203 for analyzing the tokens according to the token category to obtain multiple token analysis results;

[0098] A query statement generation module 204 for generating a query statement according to the multiple token analysis results; <00​​​​The device decomposes and analyzes the input information into lexical units, and then performs mixed retrieval based on different lexical units. It supports Chinese, full pinyin, and mixed initial letter input, and does not distinguish between uppercase and lowercase letters, thus improving the retrieval hit rate.

[0101] In one or more embodiments of the present invention, the word type includes Chinese characters, full pinyin, and abbreviated pinyin.

[0102] In one or more embodiments of the present invention, the word decomposition module 202 may specifically include:

[0103] A lexical decomposition unit is used to decompose the input information into multiple lexical units using regular expressions;

[0104] The judgment unit is used to determine whether the word element type of the full pinyin is correct. If not, the word element type of the full pinyin is further decomposed.

[0105] In one or more embodiments of the present invention, the lexical analysis module 203 may specifically include:

[0106] The query condition definition unit is used to define query conditions, which include: full pinyin search, Chinese search, abbreviated pinyin search, Chinese filtering, and full pinyin filtering.

[0107] The query field generation unit is used to generate multiple query fields based on the lexical type of the lexical and the query conditions.

[0108] In one or more embodiments of the present invention, the query field generation unit may specifically include:

[0109] The first search subunit is used to search for the word element using the abbreviation search when the word element category is abbreviation.

[0110] The second search subunit is used to search for the word element using both full-spelling search and abbreviated-spelling search when the word element category is full-spelling, and to filter the word element using full-spelling filtering.

[0111] The third search subunit is used to search for the word element using the full pinyin search, the Chinese search, and the abbreviated pinyin search respectively when the word element category is Chinese, and to filter the word element using the Chinese filter.

[0112] In one or more embodiments of the present invention, the query statement generation module 204 may specifically include:

[0113] A type determination unit is used to determine the type of the input information, wherein the type of the input information is the lowest level word type in the input information, wherein the level of the Chinese characters is higher than that of the full pinyin and the abbreviated pinyin, and the level of the full pinyin is higher than that of the abbreviated pinyin;

[0114] The query statement generation unit is used to generate a query statement based on the type of the input information and the results of the multiple lexical analysis.

[0115] In one or more embodiments of the present invention, the query statement generation unit may specifically include:

[0116] The first query subunit is used to perform a query based on the Chinese search when the type of the input information is Chinese.

[0117] In one or more embodiments of the present invention, generating a query statement based on the type of the input information and the results of the plurality of lexical analysis specifically includes:

[0118] The second query subunit is used to perform a query based on the full pinyin search and the Chinese filtering when the type of the input information is full pinyin.

[0119] In one or more embodiments of the present invention, generating a query statement based on the type of the input information and the results of the plurality of lexical analysis specifically includes:

[0120] The third query subunit is used to perform a query based on the abbreviated pinyin search, the full pinyin filter, and the Chinese character filter when the type of the input information is abbreviated pinyin.

[0121] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0122] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0123] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0124] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0125] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0126] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0127] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0128] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0129] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A hybrid information retrieval method, characterized in that, The method includes: Obtain user input information; The input information is decomposed into multiple word units, and the word unit category of each word unit is identified, wherein the word unit is the smallest semantic unit; The lexical elements are analyzed according to the lexical category to obtain multiple lexical analysis results; Generate a query statement based on the results of the multiple word element analysis; The query statement is used to query the data table to obtain the query results. The data table includes multiple data entries, and each data entry includes keywords of various word types. The word types include Chinese, full pinyin, and simplified pinyin; The analysis of the lexical units based on the lexical unit category specifically includes: Define query conditions, which include: full Pinyin search, Chinese search, abbreviated Pinyin search, Chinese filtering, and full Pinyin filtering; Based on the lexical type and the query conditions, generate multiple query fields; The step of generating multiple query fields based on the lexical type and the query conditions specifically includes: When the word element category is abbreviation, the abbreviation search is used to search for the word element; When the word element category is full pinyin, the word element is searched using the full pinyin search and the abbreviated pinyin search respectively, and the word element is filtered using the full pinyin filter; When the word element category is Chinese, the word element is searched using the full pinyin search, the Chinese search, and the simplified pinyin search respectively, and the word element is filtered using the Chinese filter. The step of generating a query statement based on the results of the multiple lexical analysis specifically includes: The type of the input information is determined, and the type of the input information is the lowest level word type in the input information, wherein the level of the Chinese is higher than that of the full pinyin and the abbreviated pinyin, and the level of the full pinyin is higher than that of the abbreviated pinyin; A query statement is generated based on the type of input information and the results of the multiple lexical analysis.

2. The method as described in claim 1, characterized in that, The step of decomposing the input information into multiple word units specifically includes: The input information is decomposed into multiple tokens using regular expressions; Determine whether the word type is full pinyin. If not, continue to decompose the word type full pinyin.

3. The method as described in claim 1, characterized in that, The step of generating a query statement based on the type of input information and the results of multiple lexical analysis specifically includes: When the type of input information is Chinese, a query is performed based on the Chinese search.

4. The method as described in claim 1, characterized in that, The step of generating a query statement based on the type of input information and the results of multiple lexical analysis specifically includes: When the type of input information is Pinyin, a query is performed based on the Pinyin search and the Chinese filtering.

5. The method as described in claim 1, characterized in that, The step of generating a query statement based on the type of input information and the results of multiple lexical analysis specifically includes: When the type of input information is abbreviated pinyin, the query is performed based on the abbreviated pinyin search, the full pinyin filter, and the Chinese character filter.

6. A hybrid information retrieval device, characterized in that, The device includes: The information acquisition module is used to acquire user input information; The lexical decomposition module is used to decompose the input information into multiple lexical units and identify the lexical unit category of each lexical unit, wherein the lexical unit is the smallest semantic unit; The lexical analysis module is used to analyze the lexical based on the lexical category and obtain multiple lexical analysis results; The query statement generation module is used to generate query statements based on the results of the multiple word element analysis. The information query module is used to query the data table according to the query statement and obtain the query results. The data table includes multiple data entries, and each data entry includes keywords of various word types. The word types mentioned include Chinese, full pinyin, and simplified pinyin; The analysis of the lexical units based on the lexical unit category specifically includes: Define query conditions, which include: full Pinyin search, Chinese search, abbreviated Pinyin search, Chinese filtering, and full Pinyin filtering; Based on the lexical type and the query conditions, generate multiple query fields; The step of generating multiple query fields based on the lexical type and the query conditions specifically includes: When the word element category is abbreviation, the abbreviation search is used to search for the word element; When the word element category is full pinyin, the word element is searched using the full pinyin search and the abbreviated pinyin search respectively, and the word element is filtered using the full pinyin filter; When the word element category is Chinese, the word element is searched using the full pinyin search, the Chinese search, and the simplified pinyin search respectively, and the word element is filtered using the Chinese filter. The step of generating a query statement based on the results of the multiple lexical analysis specifically includes: The type of the input information is determined, and the type of the input information is the lowest level word type in the input information, wherein the level of the Chinese is higher than that of the full pinyin and the abbreviated pinyin, and the level of the full pinyin is higher than that of the abbreviated pinyin; A query statement is generated based on the type of input information and the results of the multiple lexical analysis.

Images