Data search method and device, computer device and computer readable storage medium

Abstract

The application discloses a data search method and device, computer equipment and a computer readable storage medium. The method comprises the following steps: obtaining query data, the query data comprising at least one query word; performing semantic analysis on the query data, and constructing a syntax tree corresponding to the query data according to the analysis result; calculating the weight value of each query word in a plurality of candidate documents contained in a candidate document library, and constructing a first inverted index list corresponding to the query data based on the weight value; updating the first inverted index list according to a logical node in the syntax tree to obtain a second inverted index list; and querying a preset number of target candidate documents matched with the query data based on the second inverted index list. The method can improve the accuracy of the data search result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a data search method and device, computer equipment and computer readable storage medium. BACKGROUND

[0002] In recent years, the Internet technology develops rapidly, which greatly reduces the cost of people obtaining information, and human society enters the era of knowledge explosion.

[0003] The search engine is one of the main sources of people obtaining information in the Internet era, and the search engine can retrieve the document information expected by the user from the Internet according to the query data input by the user by using a certain algorithm and side crack.

[0004] At present, the search engine has low accuracy in data search according to the query data input by the user. SUMMARY

[0005] The embodiment of the present application provides a data search method, device, computer readable storage medium and computer equipment, which can effectively improve the accuracy of data search.

[0006] The first aspect of the present application provides a data search method, which comprises:

[0007] Obtaining query data, wherein the query data comprises at least one query word;

[0008] Performing semantic analysis on the query data, and constructing a syntax tree corresponding to the query data according to the analysis result;

[0009] Calculating the weight value of each query word in a plurality of candidate documents contained in a candidate document library, and constructing a first inverted index list corresponding to the query data based on the weight value;

[0010] Updating the first inverted index list according to the logical node in the syntax tree to obtain a second inverted index list;

[0011] Querying a preset number of target candidate documents matched with the query data based on the second inverted index list.

[0012] Correspondingly, the second aspect of the present application provides a data search device, which comprises:

[0013] An acquisition unit configured to acquire query data, wherein the query data comprises at least one query word;

[0014] An analysis unit configured to perform semantic analysis on the query data, and construct a syntax tree corresponding to the query data according to the analysis result;

[0015] The constructing unit is configured to calculate a weight value of each query word in a plurality of candidate documents included in a candidate document library, and construct a first inverted index list corresponding to the query data based on the weight value;

[0016] The updating unit is configured to update the first inverted index list according to the logical nodes in the syntax tree, to obtain a second inverted index list;

[0017] The querying unit is configured to query a preset number of target candidate documents matching the query data based on the second inverted index list.

[0018] In some embodiments, the analyzing unit comprises:

[0019] The first analyzing subunit is configured to perform semantic analysis on the query data, to determine phrase data in the query data and synonyms of each query word, the phrase data comprising at least two associated query words.

[0020] The first constructing subunit is configured to construct a syntax tree with the phrase data as an and node, and each query word and the corresponding synonym as an or node.

[0021] In some embodiments, the first inverted index list is a block inverted index list, and the updating unit comprises:

[0022] The first determining subunit is configured to determine, in the first inverted index list, a plurality of first block inverted indexes corresponding to each logical node in the syntax tree, the block inverted index comprising a plurality of inverted blocks.

[0023] The second determining subunit is configured to determine, according to the logical relationship of each logical node and the corresponding plurality of first block inverted indexes, a second block inverted index corresponding to each logical node.

[0024] The replacing subunit is configured to replace, in the first inverted index list, the plurality of first block inverted indexes corresponding to each logical node with the corresponding second block inverted index, to obtain the second inverted index list.

[0025] In some embodiments, the second determining subunit comprises:

[0026] The first determining module is configured to, when the logical relationship of a logical node is an and logical, determine a candidate block inverted index with the least number of documents in the plurality of first block inverted indexes.

[0027] The second determining module is configured to determine a first inverted block in the candidate block inverted index, and determine, based on the first inverted block, an associated inverted block in other block inverted indexes, the other block inverted indexes being block inverted indexes other than the candidate block inverted index in the plurality of first block inverted indexes.

[0028] The first operation module is configured to perform an AND operation on the parameters of the first inverted block and the parameters of the associated inverted block to obtain a second inverted block corresponding to the first inverted block.

[0029] The third determination module is configured to determine a second sub-block inverted index corresponding to the logical node according to the second inverted block.

[0030] In some embodiments, the data search device provided in the present application further comprises:

[0031] The fourth determination module is configured to determine a third inverted block in each first sub-block inverted index when the logical relationship of the logical node is OR logic, to obtain a plurality of third inverted blocks.

[0032] The second operation module is configured to perform an OR operation on the parameters of the plurality of third inverted blocks to obtain a fourth inverted block.

[0033] The fifth determination module is configured to determine a second sub-block inverted index corresponding to the logical node according to the fourth inverted block.

[0034] In some embodiments, the computing unit comprises:

[0035] The first obtaining sub-unit is configured to obtain the number of times that the target query word appears in each candidate document and the document length of each candidate document.

[0036] The first computing sub-unit is configured to calculate a first weight parameter of the target query word in each candidate document according to the number of times that the target query word appears in each candidate document and the document length of each candidate document.

[0037] The second obtaining sub-unit is configured to obtain a first number of candidate documents and a second number of candidate documents containing the target query word in the candidate documents.

[0038] The second computing sub-unit is configured to calculate a second weight parameter of the target query word in each candidate document according to the first number and the second number.

[0039] The third computing sub-unit is configured to calculate a weight value of the target query word in each candidate document according to the first weight parameter and the second weight parameter.

[0040] The first traversal sub-unit is configured to traverse each query word to obtain a weight value of each query word in each candidate document.

[0041] In some embodiments, the computing unit comprises:

[0042] The third determining sub-unit is configured to determine a plurality of target candidate documents associated with the target query word from the plurality of candidate documents, and create a document number for each target candidate document in descending order;

[0043] The fourth determining sub-unit is configured to determine a target weight value of the target query word in each target candidate document according to the weight value;

[0044] The second constructing sub-unit is configured to construct an inverted index of the target query word based on the document number of each target candidate document and the target weight value;

[0045] The second traversing sub-unit is configured to traverse each query word to generate an inverted index corresponding to each query word, and obtain a first inverted index list.

[0046] In some embodiments, the data search device provided in the present application further comprises:

[0047] The dividing sub-unit is configured to divide the inverted index corresponding to each query word into a plurality of inverted blocks;

[0048] The fifth determining sub-unit is configured to determine an inverted block parameter of each inverted block based on the document number of the document included in each inverted block and the weight value of the query word in the document;

[0049] The first updating sub-unit is configured to update the first inverted index list according to the inverted block parameter of each inverted block.

[0050] In some embodiments, the data search device provided in the present application further comprises:

[0051] The sixth determining sub-unit is configured to determine an associated query word included in the new candidate document based on the at least one query word when detecting that the new candidate document is added to the candidate document library, and the associated query word belongs to the at least one query word;

[0052] The fourth calculating sub-unit is configured to calculate a weight value of each associated query word in the new candidate document;

[0053] The second updating sub-unit is configured to number the new candidate document, and update the first inverted index list based on the number of the new candidate document and the weight value of each associated query word in the new candidate document.

[0054] In some embodiments, the query unit comprises:

[0055] The splitting sub-unit is configured to split each inverted index in the second inverted index list into a plurality of sub-inverted indexes, and obtain a plurality of sub-inverted index lists;

[0056] The first query subunit is configured to simultaneously query candidate documents matching the query data based on each sub-inverted index list by using multiple threads to obtain multiple query results.

[0057] The sixth determination subunit is configured to determine a preset number of target candidate documents matching the query data according to the multiple query results.

[0058] In some embodiments, the query unit is configured to:

[0059] The second analysis subunit is configured to perform semantic analysis on the query data to determine keywords contained in the query data.

[0060] The second query subunit is configured to query a preset number of target candidate documents matching the query data based on the keywords and the second inverted index list.

[0061] The third aspect of the present application further provides a computer readable storage medium, which stores a plurality of instructions, and the instructions are adapted to be loaded by a processor to execute the steps in the data search method provided in the first aspect of the present application.

[0062] The fourth aspect of the present application provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the steps in the data search method provided in the first aspect of the present application when executing the computer program.

[0063] The fifth aspect of the present application provides a computer program product, which includes computer programs / instructions, and the computer programs / instructions implement the steps in the data search method provided in the first aspect when executed by a processor.

[0064] The data search method provided in the embodiments of the present application includes the following steps: obtaining query data, the query data including at least one query word; performing semantic analysis on the query data, and constructing a syntax tree corresponding to the query data according to an analysis result; calculating a weight value of each query word in a plurality of candidate documents contained in a candidate document library, and constructing a first inverted index list corresponding to the query data based on the weight value; updating the first inverted index list according to a logical node in the syntax tree to obtain a second inverted index list; and querying a preset number of target candidate documents matching the query data based on the second inverted index list.

[0065] According to the data search method, the semantic analysis can be performed on the obtained query data, and a syntax tree is constructed according to the result of the semantic analysis, and the inverted index list is updated according to the nodes in the syntax tree, so that the semantic information of the query data can be fully considered when the index list is recalled, the recall deviation is avoided, and more accurate documents can be queried. The method can effectively improve the accuracy of data search. BRIEF DESCRIPTION OF DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0067] Figure 1 is a scene diagram of data search in the present application;

[0068] Figure 2 is a flow diagram of the data search method provided by the present application;

[0069] Figure 3 is a schematic diagram of the inverted index list;

[0070] Figure 4 is a schematic diagram of the block inverted index list;

[0071] Figure 5 is another flow diagram of the data search method provided by the present application;

[0072] Figure 6 is a schematic diagram of the real-time stream inverted index list;

[0073] Figure 7 is a schematic diagram of the data structure corresponding to the inverted list;

[0074] Figure 8 is a schematic diagram of generating a node virtual inverted block;

[0075] Figure 9 is a schematic diagram of generating an or node virtual inverted block;

[0076] Figure 10 is a schematic diagram of the data search device provided by the present application;

[0077] Figure 11 is a schematic diagram of the computer device provided by the present application. DETAILED DESCRIPTION

[0078] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0079] The embodiments of the present application provide a data search method, a data search device, a computer device and a computer readable storage medium. The data search method can be used in the data search device. The data search device can be integrated in the computer device, which can be a terminal or a server. The terminal can be a mobile phone, a tablet computer, a notebook computer, a smart television, a wearable smart device, a personal computer (PC) and a vehicle terminal, etc. The server can be a physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The server can be a node in a blockchain.

[0080] Please refer to Figure 1 A scene diagram of the data search method provided by the present application is shown. As shown in the figure, the server A receives the query request sent by the terminal B, then obtains the query data from the query request, the query data includes at least one query word; the query data is analyzed semantically, and a syntax tree corresponding to the query data is constructed according to the analysis result; the weight value of each query word in the plurality of candidate documents included in the candidate document library is calculated, and a first inverted index list corresponding to the query data is constructed based on the weight value; the first inverted index list is updated according to the logical nodes in the syntax tree, and a second inverted index list is obtained; the second inverted index list is used to query a preset number of target candidate documents matching the query data.

[0081] It should be noted that Figure 1 The data search scene diagram shown is only an example, and the data search scene described in the embodiments of the present application is used to more clearly illustrate the technical solutions of the present application, and does not limit the technical solutions provided by the present application. Those skilled in the art can know that with the evolution of data search scene and the emergence of new business scene, the technical solutions provided by the present application are also applicable to similar technical problems.

[0082] Based on the above implementation scene, the following will be described in detail.

[0083] In the related art, when a search engine queries a corresponding recommended document according to query data input by a user, only a plurality of query words contained in the query data are considered, and a plurality of candidate documents with higher scores are recalled according to the query words and recommended to the user. The logical relationship between the query words is not considered, for example, some query words have a phrase relationship and cannot be separated, and some query words have a synonym relationship and only one needs to be recalled. Therefore, the existing data search method may cause recall bias, and thus the accuracy of data search is poor. The present application provides a data search method, which can improve the accuracy of data search.

[0084] Embodiments of the present application will be described from the perspective of a data search device, which can be integrated in a computer device. The computer device can be a terminal or a server. The terminal can be a mobile phone, a tablet computer, a notebook computer, a smart television, a wearable smart device, a personal computer (PC), and a vehicle-mounted terminal, etc. The server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms, etc. basic cloud computing services. As shown in the figure, the flowchart of the data search method provided by the present application includes the following steps: Figure 2

[0085] Step 101, obtaining query data.

[0086] The query data can be data input by a user to query related documents. Specifically, the user can input the query data in the search engine, and the search engine generates a query request based on the query data input by the user and sends it to the data search device. The data search device parses the query request after receiving the query request to obtain the query data in the query request. Then the data search device further searches the data according to the query data.

[0087] The query data can be a single word, a word, a sentence, or a paragraph. The query data can also be a search formula composed of a specific logical language. The query data can be Chinese query data, English query data, Japanese query data, etc. In the embodiments of the present application, the query data is taken as Chinese for specific introduction.

[0088] ​When the query data is a sentence or a paragraph, if a sentence or a paragraph is taken as a target for document matching, the number of documents that can be recalled is small, resulting in poor recall effect. Generally, the sentence or paragraph can be divided into words, and then the data search and recall can be further performed according to the plurality of query words.

[0089] In step 102, semantic analysis is performed on the query data, and a syntax tree corresponding to the query data is constructed according to the analysis result.

[0090] In the present application, instead of directly querying the candidate documents according to the query words in the query data, semantic analysis is first performed on the query data, and the process of querying the documents is improved according to the semantic analysis result. Specifically, semantic analysis can be first performed on the query data, and a syntax tree corresponding to the query data is constructed according to the semantic analysis result.

[0091] The semantic analysis of the query data can be performed by using a preset semantic analysis neural network model. The semantic analysis neural network model can determine the specific meaning expressed by each query word in combination with the context of each query word, and then output the logical relationship between the query words contained in the query data. For example, a single independent query word, a phrase formed by query words, and query words that are synonyms of each other are output. For example, when the query data is "China bank interest rate, China bank interest rate", "China bank" and "China bank" are synonyms, and any one of the recall results can be included. And "China" and "bank" form the phrase "China bank", so the recall result must include both words and cannot be split. If it is split, it will cause semantic drift, resulting in inaccurate query results. There is no synonym for "interest rate" and no phrase, so it is an independent query word.

[0092] The syntax tree is a graphical representation of the structure of a sentence, which represents the derivation result of the sentence and is conducive to understanding the hierarchy of the sentence syntax structure. Simply put, the syntax tree is a tree formed when derivation is performed according to a certain rule. The nodes in the syntax tree can include independent query words, phrases with a relationship, and synonyms with the same meaning.

[0093] In some embodiments, the semantic analysis of the query data and the construction of the syntax tree corresponding to the query data according to the analysis result include:

[0094] 1. Perform semantic analysis on the query data to determine phrase data in the query data and synonyms of each query word, the phrase data containing at least two associated query words;

[0095] 2. Construct a syntax tree with the phrase data as an and node and each query word and the corresponding synonym as an or node.

[0096] In the embodiments of the present application, the syntax tree can be constructed by taking the phrase data as the AND node, and the query words and the corresponding synonyms as the OR node. The AND node and the OR node are used interchangeably. The two or more query words corresponding to the AND node need to be recalled simultaneously, that is, the recall result must contain the two or more query words in the AND node. The two or more query words corresponding to the OR node only need to be recalled once, that is, the recall result only needs to contain any one of the two or more query words corresponding to the OR node.

[0097] The phrase data and the synonyms are obtained by performing semantic analysis on the query data. The semantic analysis can be performed by using the aforementioned semantic analysis neural network model or by using a preset semantic analysis algorithm. The specific model structure of the semantic analysis neural network model can be any suitable model structure, which is not limited herein.

[0098] In step 103, the weight value of each query word in the plurality of candidate documents included in the candidate document library is calculated, and a first inverted index list corresponding to the query data is constructed based on the weight value.

[0099] In the present application, the data search can be performed by using the inverted index method. The inverted index is to score the documents according to the query words input by the user, and then recall the top preset number of documents. The inverted index is derived from the need to find records according to the value of an attribute in practical applications. Each item in this index table includes an attribute value and the address of each record with the attribute value. Since the attribute value is not determined by the record, but the position of the record is determined by the attribute value, it is called inverted index. Specifically, in the field of data search, the inverted index is to construct an inverted index list for data search, which contains the inverted index corresponding to each query word. Each inverted index contains the document ID (Identity document) of a plurality of documents and the weight of the corresponding query word in each document.

[0100] In some embodiments, the inverted index list corresponding to the query words can be pre-constructed, specifically, the inverted index list corresponding to the query words can be constructed based on all candidate documents in the candidate document library, and the inverted index list can be maintained. When a new candidate document is added to the candidate document library, the new candidate document can be numbered, and it is determined whether the new candidate document contains any of the query words in the inverted index list. If so, the document ID is added to the inverted list corresponding to the query word to update the inverted list. Then, when the query data input by the user is received and the query words are parsed, the inverted index list corresponding to the query words can be obtained from the pre-constructed inverted index list. In this way, the efficiency of document query can be greatly improved.

[0101] In some embodiments, the weight value of each query word in the plurality of candidate documents is calculated, including:

[0102] 1. The number of times the target query word appears in each candidate document and the document length of each candidate document are obtained.

[0103] 2. The first weight parameter of the target query word in each candidate document is calculated according to the number of times it appears in each candidate document and the document length of each candidate document.

[0104] In the embodiments of the present application, the method of calculating the term frequency (TF) of the query word in the document can be used to calculate the weight of the query word in the document. Specifically, for any target query word, the number of times it appears in each candidate document can be obtained. Then, the document length of each candidate document is obtained, and the ratio of the number of times the target query word appears in each candidate document to the corresponding document length is calculated, and the weight of the target query word in each candidate document can be obtained. Similarly, by traversing each query word, the weight of each query word in each candidate document can be obtained.

[0105] That is, in the present application, the weight of the query word in the document is proportional to the number of times the query word appears in the document, and inversely proportional to the document length of the document corresponding to the query word.

[0106] In some embodiments, the method of calculating the weight of the query word in the document can further include:

[0107] 3. The first number of candidate documents and the second number of candidate documents containing the target query word are obtained.

[0108] 4. The second weight parameter of the target query word in each candidate document is calculated according to the first number and the second number.

[0109] 5. Calculate the weight value of the target query term in each candidate document based on the first weight parameter and the second weight parameter;

[0110] 6. Iterate through each query term and obtain the weight value of each query term in each candidate document.

[0111] In this embodiment, when calculating the weight of a query term in a document, the inverse document frequency (IDF) corresponding to the query term can be further considered. Specifically, for any target query term, the total number of candidate documents (referred to here as the first number) and the number of candidate documents containing the target query term (referred to here as the second number) can be obtained. Then, the logarithm of the ratio of the first number and the second number can be used to determine the inverse document frequency corresponding to the target query term. That is, the IDF value of the target query term is inversely proportional to the total number of candidate documents and directly proportional to the number of candidate documents containing the target query term. Further, the product of the term frequency and the inverse document frequency of the target query term can be determined as the weight of the target query term in each candidate document. Further, by traversing each query term, the weight of each query term in each candidate document can be obtained.

[0112] It's understandable that the term frequency of a query term in a specific candidate document is fixed. However, the inverse text frequency (IMSF) of that query term changes with the number of candidate documents. When the number of candidate documents in the candidate document library increases, but the number of documents containing the query term does not increase, the IMSF of the query term will decrease. Conversely, if newly added documents to the candidate document library contain the query term, the IMSF of the query term will increase. Therefore, the term frequency of each query term in the candidate documents can be pre-calculated as an initial weight. During query retrieval, the IMSF of each query term can be calculated in real time, and the weight of the query term in the candidate documents can be updated based on the real-time calculated IMSF. This can improve the efficiency of data search.

[0113] like Figure 3 The diagram shows a schematic of an inverted index list. As shown, the inverted index list 10 contains multiple inverted indexes 11 corresponding to query term 111. Each inverted index 11 contains the document IDs of multiple documents 112 corresponding to query term 111 and the weight of query term 111 in document 112.

[0114] Based on the inverted index list, each document can be scored. Specifically, the MaxScore, WeakAND, Block-Max WeakAND, and the like can be used to score each document to obtain the score of each document, and then a preset number of documents with higher scores are selected as target documents for recall.

[0115] In some embodiments, the first inverted index list corresponding to the query data is constructed based on the weight values, including:

[0116] A. Among the plurality of candidate documents, a plurality of target candidate documents associated with the target query word are determined, and a document number is created for each target candidate document in descending order;

[0117] B. The target weight value of the target query word in each target candidate document is determined according to the weight value;

[0118] C. The inverted index of the target query word is constructed based on the document number and the target weight value of each target candidate document;

[0119] D. Each query word is traversed to generate an inverted index corresponding to each query word to obtain the first inverted index list.

[0120] In the embodiments of the present application, a specific method for creating an inverted index list is given. As described above, the inverted index list contains inverted indexes corresponding to a plurality of query words. Here, the creation process of the inverted index corresponding to an arbitrary target query word is introduced as an example.

[0121] Please continue to refer to Figure 3 For the target query word "China", three documents containing the query word "China" are found in the candidate documents, so the three documents can be determined as the target candidate documents corresponding to the target query word "China". At this time, the three target candidate documents can be numbered in descending order, i.e., 01 first, then 02, and so on. The specific target candidate document numbered 01 and the specific target candidate document numbered 02 can be freely set. In the present application, the numbering can be performed according to the time sequence of the query. When the first target candidate document containing the target query word "China" is found, it is numbered 01. When the next target candidate document containing the target query word "China" is found, it is numbered 02. In this way, the weight of each query word in each candidate document is calculated as described above. At this time, the weight of the target query word "China" in the corresponding three target candidate documents can be obtained. Finally, the inverted index of the target query word "China" is constructed based on the document number and the corresponding weight value of each target candidate document as follows: Figure 3The inverted index shown.

[0122] Then, by traversing each query word, the inverted index corresponding to each query word can be obtained, and then the inverted index list corresponding to the query data is obtained. The creation process of the inverted index can be completed offline before receiving the query data. When querying, the inverted index corresponding to the query data can be found in the inverted list according to the query data, so that the efficiency of document querying can be improved.

[0123] In some embodiments, the method for creating the inverted index list in the present application can further include:

[0124] E. dividing the inverted index corresponding to each query word into multiple inverted blocks;

[0125] F. determining the inverted block parameters of each inverted block based on the document numbers of the documents contained in each inverted block and the weight values of the query words in the documents;

[0126] G. updating the first inverted index list according to the inverted block parameters of each inverted block.

[0127] In the present application, a block inverted index list can be constructed, and then the aforementioned block maximum soft intersection method can be used to query and recall a preset number of target candidate documents. The block maximum soft intersection method does not need to traverse and calculate the scores of each candidate document, and can prune non-target candidate documents according to certain rules, so that score calculation is not needed for non-target candidate documents, thereby greatly improving the efficiency of data searching. Here, the construction method of the block inverted index list is introduced first, and the specific process of recalling target candidate documents by the block maximum soft intersection method corresponding to the block inverted index will be introduced later.

[0128] Specifically, the inverted index corresponding to each query word in the aforementioned constructed inverted index list can be divided first to obtain multiple inverted blocks. Then, for each inverted block, the inverted block parameters of the inverted block are determined. The parameters of the inverted block can include the ID of the starting document, the ID of the ending document, and the maximum value of the weight values of these documents. Then, the aforementioned inverted index list can be updated according to the inverted block parameters of the inverted block.

[0129] As Figure 4 shown, the inverted index list obtained after updating each inverted index in the inverted index list shown in Figure 3 As shown in the figure, in the updated inverted index list 10, the inverted index 11 corresponding to each query word 111 contains multiple inverted blocks 113. The inverted block 113 contains the starting document ID, the ending document ID, and the maximum weight coefficient of all documents in the inverted block.

[0130] In some embodiments, the data searching method provided in the present application further comprises:

[0131] H. When it is detected that a new candidate document is added in the candidate document library, determining the associated query words contained in the new candidate document based on the at least one query word, the associated query words belonging to the at least one query word;

[0132] I. Calculating the weight value of each associated query word in the new candidate document;

[0133] J. Numbering the new candidate document, and updating the first inverted index list based on the number of the new candidate document and the weight value of each associated query word in the new candidate document.

[0134] In the embodiment, the real-time updating of the candidate document can be supported, and the inverted index list can be refreshed according to the updated candidate document library. Specifically, when it is detected that a new candidate document is added in the candidate document library, it is detected whether the query data contains the query words in the new candidate document. If it is detected that one or more query words contained in the query data exist in the new candidate document, the one or more query words are determined as the associated query words. Then, the weight values of the associated query words in the new candidate document are calculated one by one. The specific calculation method can refer to the calculation method of the weight value of the query word in the candidate document, which will not be described herein.

[0135] Then, the inverted index corresponding to each associated query word is found one by one, and the document ID and the corresponding weight of the new candidate document are added to the inverted index of each associated query word, so as to realize the updating of the inverted index list. The document ID of the new candidate document can be obtained by adding 1 to the maximum value of the document ID in the inverted index list. That is, the new candidate document is numbered in ascending order.

[0136] When the inverted index corresponding to the associated query word is the block inverted index, the new candidate document can be added to the last inverted block of the block inverted index of the associated query word. Specifically, the terminal document ID of the last inverted block of the block inverted index of the associated query word can be modified as the ID of the new candidate document, and the weight value of the last inverted block is compared with the weight value of the new candidate document, and the larger one is determined as the weight value of the inverted block.

[0137] Step 104, updating the first inverted index list according to the logical nodes in the syntax tree to obtain a second inverted index list.

[0138] Wherein, after generating the inverted index list corresponding to the query data according to the weight values of the query words in the query data in each candidate document, the generated inverted index list can be further updated according to the logical nodes in the syntax tree to obtain a new inverted index list. Here, in order to distinguish the inverted index list before and after the update, the inverted index list before the update is determined as the first inverted index list, and the inverted index list after the update is determined as the second inverted index list. Wherein, the inverted index list here can be a block inverted index list.

[0139] In some embodiments, updating the first inverted index list according to the logical nodes in the syntax tree to obtain the second inverted index list comprises:

[0140] 1. Determining a plurality of first block inverted indexes corresponding to each logical node in the syntax tree in the first inverted index list, wherein the block inverted index comprises a plurality of inverted blocks;

[0141] 2. Determining a second block inverted index corresponding to each logical node according to the logical relationship of each logical node and the corresponding plurality of first block inverted indexes;

[0142] 3. Replacing the plurality of first block inverted indexes corresponding to each logical node in the first inverted index list with the corresponding second block inverted indexes to obtain the second inverted index list.

[0143] Wherein, updating the first inverted index list according to the logical nodes in the syntax tree can specifically be to first determine a plurality of inverted indexes corresponding to each logical node in the first inverted index list according to the logical nodes in the syntax tree. Then, updating the plurality of inverted indexes according to the logical relationship in the logical nodes to obtain an inverted index corresponding to each logical node. Wherein, in order to distinguish the inverted indexes before and after the update, the plurality of inverted indexes corresponding to the logical nodes in the first inverted index list can be referred to as first inverted indexes, and the inverted indexes obtained by updating the plurality of first inverted indexes according to the logical relationship of the logical nodes can be referred to as second inverted indexes.

[0144] Further, the plurality of first inverted indexes in the first inverted index list can be updated to the corresponding second inverted indexes. Then, each logical node in the syntax tree is traversed, and the plurality of first inverted indexes corresponding to each logical node are all replaced with the corresponding second inverted indexes, thereby realizing the update of the first inverted index list to obtain the updated second inverted index list.

[0145] In some embodiments, determining a second block inverted index corresponding to each logical node according to the logical relationship of each logical node and the corresponding plurality of first block inverted indexes comprises:

[0146] 2.1, when the logical relationship of the logical node is and logic, determining a candidate chunk inverted index with the least number of documents in the plurality of first chunk inverted indexes;

[0147] 2.2, determining a first inverted block in the candidate chunk inverted index, and determining an associated inverted block in other chunk inverted indexes based on the first inverted block, the other chunk inverted indexes being the chunk inverted indexes other than the candidate chunk inverted index in the plurality of first chunk inverted indexes;

[0148] 2.3, performing an AND operation on the parameters of the first inverted block and the parameters of the associated inverted block to obtain a second inverted block corresponding to the first inverted block;

[0149] 2.4, determining a second chunk inverted index corresponding to the logical node according to the second inverted block.

[0150] The logical relationship of the logical node in the syntax tree can be and logic or or logic. When the logical node is and logic, i.e., the query terms constitute a phrase relationship and need to be recalled simultaneously. The inverted index with the least number of documents in the plurality of chunk inverted indexes corresponding to the and logical node is determined as a candidate chunk inverted index. Then, an arbitrary first inverted block is determined in the candidate chunk inverted index, and an associated inverted block corresponding to the first inverted block is determined in the other chunk inverted indexes other than the candidate chunk inverted index in the first chunk inverted index. The associated inverted block can be one or multiple. The minimum value of the start ID in the inverted block parameters of the associated inverted block is not greater than the start ID of the first inverted block, and the maximum value of the end ID in the inverted block parameters of the associated inverted block is not less than the end ID of the first inverted block.

[0151] After the associated inverted block corresponding to the first inverted block is determined, an AND operation is performed on the inverted block parameters of the first inverted block and the inverted block parameters of the associated inverted block to obtain a second inverted block corresponding to the first inverted block. Specifically, the AND operation on the inverted block parameters of the first inverted block and the inverted block parameters of the associated inverted block can be to determine the maximum value of the weight values of the associated inverted blocks in each first chunk inverted index, then add the maximum values of the weight values to the weight value of the first inverted block to obtain the weight value of the second inverted block, and then take the start document ID of the first inverted block as the start document ID of the second inverted block, and take the end document ID of the first inverted block as the end document ID of the second inverted block to generate the second inverted block corresponding to the first inverted block.

[0152] Then, each first inverted block in the candidate chunk inverted index is traversed to obtain a second inverted block corresponding to each first inverted block. All the second inverted blocks are combined to obtain a second chunk inverted index corresponding to the and logical node.

[0153] In some embodiments, the data search method provided by the present application further comprises:

[0154] 2.5、When the logical relationship of the logical node is or logical, a third inverted block is determined in each first sub-block inverted index, and a plurality of third inverted blocks are obtained;

[0155] 2.6, The parameters of the plurality of third inverted blocks are operated by or operation, and a fourth inverted block is obtained;

[0156] 2.7, The second sub-block inverted index corresponding to the logical node is determined according to the fourth inverted block.

[0157] When the logical relationship in the logical node is or logical, a plurality of inverted lists corresponding to the logical node can be determined, and a third inverted block in the inverted lists is determined. The third inverted block can be an inverted block pointed to by the current pointer.

[0158] Further, the plurality of third inverted blocks determined are operated by or operation. Specifically, the minimum value of the start document ID in the inverted block parameters of the third inverted block is taken as the start document ID of the fourth inverted block, the maximum value of the end document ID in the inverted block parameters of the third inverted block is taken as the end document ID of the fourth inverted block, and the maximum value of the weight value in the inverted block parameters of the third inverted block is taken as the weight value of the fourth inverted block. Thus, the fourth inverted block obtained by operating the third inverted block by or operation can be determined. Further, the second sub-block inverted index can be determined according to the fourth inverted block.

[0159] Step 105, querying a preset number of target candidate documents matching the query data based on the second inverted index list.

[0160] After the second inverted index list corresponding to the query data is determined, each candidate document can be scored based on the second inverted index list, and a scoring value of each candidate document is obtained. Then, a preset number of candidate documents with the highest scores are selected as target candidate documents for recall, and the target candidate documents are displayed to the user.

[0161] In some embodiments, querying a preset number of target candidate documents matching the query data based on the second inverted index list includes:

[0162] 1. Splitting each inverted index in the second inverted index list into a plurality of sub-inverted indexes, and obtaining a plurality of sub-inverted index lists;

[0163] 2. Querying candidate documents matching the query data based on each sub-inverted index list simultaneously using a plurality of threads, and obtaining a plurality of query results;

[0164] 3. Determining a preset number of target candidate documents matching the query data according to the plurality of query results.

[0165] In the present application, multi-threading can be used for inverted index query. Specifically, each inverted index in the second inverted index list corresponding to the query data can be split into multiple sub-inverted indexes, obtaining multiple sub-inverted indexes. Specifically, the splitting method can be split according to the document ID, so as to split one inverted index list into several independent sub-inverted index lists. Alternatively, it can also be understood that the candidate document library is split into several sub-candidate document libraries, and then the sub-inverted index list corresponding to the query data is generated for each sub-candidate document library. Then, multiple threads are used to perform index query on each sub-inverted index list to obtain multiple recall results corresponding to each sub-inverted index list.

[0166] Further, the top preset number of candidate documents with the highest scores in the multiple recall results corresponding to each sub-inverted list can be determined as target candidate documents. This method can use multiple threads for sub-thread query, which can greatly improve the efficiency of data search.

[0167] In some embodiments, querying the second inverted index list to obtain the preset number of target candidate documents matching the query data includes:

[0168] A. performing semantic analysis on the query data to determine the keywords contained in the query data;

[0169] B. querying the second inverted index list to obtain the preset number of target candidate documents matching the query data based on the keywords.

[0170] In the embodiments of the present application, the accuracy of data search can be further improved according to the keywords in the query data. Specifically, in some query data, there are keywords that must be recalled in the query recall, that is, the target candidate documents obtained by the query must contain the keywords. The keywords can be determined according to the results of semantic analysis. For example, in some cases, the keyword can be the word corresponding to the object in the sentence. For example, "interest rate" in "China bank interest rate".

[0171] Since the target candidate documents that need to be recalled must contain the keyword, the inverted index corresponding to the keyword can be found in the second inverted index list first. Then further update the second inverted index list according to the documents contained in the inverted index corresponding to the keyword, that is, delete the document information in the second inverted index list that is not in the inverted index corresponding to the keyword, to obtain a new inverted index list. Then recall according to the new inverted index list.

[0172] Or in some cases, after determining the keyword in the query data, the candidate documents can be filtered according to the keyword first, and only the documents containing the keyword are retained as candidate documents. Then the inverted index list is reconstructed for query, and the preset number of target candidate documents matching the query data are obtained.

[0173] According to the above description, the data search method provided by the embodiment of the application can obtain query data, the query data including at least one query word; perform semantic analysis on the query data, and construct a syntax tree corresponding to the query data according to the analysis result; calculate the weight value of each query word in a plurality of candidate documents included in a candidate document library, and construct a first inverted index list corresponding to the query data based on the weight value; update the first inverted index list according to the logical nodes in the syntax tree to obtain a second inverted index list; and query a preset number of target candidate documents matching the query data based on the second inverted index list.

[0174] Therefore, the data search method provided by the application can perform semantic analysis on the obtained query data, construct a syntax tree according to the result of the semantic analysis, and update the inverted index list according to the nodes in the syntax tree, so that the semantic information of the query data can be fully considered when the index list is recalled, the recall deviation is avoided, and more accurate documents can be queried. The method can effectively improve the accuracy of data search.

[0175] The application also provides a data search method, which can be used in a computer device, which can be a terminal or a server. In the application, the computer device is taken as an example of a server for detailed description. As shown in Figure 5 The method specifically includes the following steps:

[0176] In step 201, the server receives a data search request sent by a terminal.

[0177] The terminal can be a smart phone, a tablet, a telephone watch or a personal computer, etc. The data search request can be a data search request input in a search engine, or a data search request input in a data search tool provided in an instant messaging application. The data search request includes data search data input by a user.

[0178] In step 202, the server parses the data search request to obtain search data.

[0179] The server can analyze the data search request to obtain search data included in the data search request after receiving the data search request sent by the terminal. The search data can also be referred to as query data. The query data can be a word, a phrase, a sentence or a paragraph. The query data can be text format query data or voice form query data. When the query data is voice form query data, the server can perform voice recognition on the voice data and convert it into text format query data.

[0180] In step 203, the server performs semantic analysis on the query data to obtain phrase data, synonyms and must-have words.

[0181] After obtaining the query data by analysis, the server can perform semantic analysis on the query data to determine whether there is a certain logical relationship between multiple query words included in the query data. The logical relationship can be an association relationship, a synonym relationship, etc. When the query words have an association relationship, the multiple query words having the association relationship form phrase data. When the query words have a synonym relationship, the multiple query words having the synonym relationship form synonyms.

[0182] In addition, the server can also determine must-have words in the query data according to semantic analysis. The must-have words can be understood as the most critical words in the query data, or can be understood as words that must be included in the recalled documents.

[0183] In step 204, the server constructs a syntax tree according to the phrase data, synonyms and must-have words.

[0184] After analyzing the query data to determine the query words, phrase data, synonyms and must-have words, the server can further construct a syntax tree corresponding to the query data by taking the query words, phrase data, synonyms and must-have words as nodes of the syntax tree. Then the server further performs data search according to the syntax tree.

[0185] In step 205, the server constructs a real-time stream inverted index according to the query data.

[0186] In this application, the server pulls an inverted list corresponding to the query word according to the query word included in the query data input by the user. In order to find the inverted index list corresponding to the query word more quickly, this application uses a hash bucket to store the word ID of the query word. A hash bucket can include multiple word IDs. For example, Figure 6As shown in the figure, the real-time stream inverted index list provided by the present application is shown. One hash bucket can correspond to one query word or multiple query words, and one query word corresponds to one inverted index list (also referred to as an inverted list). Therefore, one hash bucket can correspond to one or more inverted lists. For multiple inverted lists in one hash bucket, a pointer can be set at the end of a previous inverted list, and the pointer points to the next inverted list. For the last inverted list in the hash bucket, the pointer at the end of the inverted list points to null.

[0187] As shown in the figure, the data structure corresponding to one inverted list is shown. The data structure corresponding to the inverted list can be the data structure of the inverted list corresponding to any query word. Specifically, as shown in the figure, a plurality of memory blocks can be used to store the data corresponding to the inverted list. Specifically, the first memory block can be used to store the information of one inverted block in the inverted list, wherein the information of one inverted block includes the head information of the inverted block, the document ID of each document in the inverted block, and the TF value of the document, the head information of the inverted block includes the start ID of the start document in the inverted block, the end ID of the end document, and the maximum value of the weight values of all documents in the inverted block, and the maximum value is represented by the UB value. Figure 7

[0188] When the real-time stream inverted index is constructed, if a new document is added to the candidate document library, the new document can be assigned an ID, and specifically, the document can be assigned an ID in ascending order. For example, the document ID of the new document can be obtained by adding 1 to the maximum value of all document IDs in the inverted list. Then, the query words contained in the newly added document can be obtained, and the inverted lists corresponding to the query words are found, and the document is added to the end of each inverted list corresponding to the query word. When the new document is added to the end of an inverted block, the UB value and the end ID of the inverted block can be updated. If the newly added document is the only document in the inverted block, the document ID is set as the start ID of the inverted block, and the weight of the query word in the new document is set as the UB value of the inverted block.

[0189] In the present application, the TF / IDF scoring method can be used to calculate the weight of each query word in the document. Specifically, the following formulas (1) to (3) can be used for specific calculation:

[0190] Score(d,t)=TF t∈d ×IDF t Formula (1)

[0191]

[0192] ​

[0193] where Score(d, t) denotes the weight of query term t in document d. TF t∈d is the term frequency of query term t in document d. fre(d, t) is the number of occurrences of query term t in document d, and len(d) is the length of document d. IDF t is the inverse document frequency of query term t, |D| is the total number of candidate documents in the candidate document library, and |d t | is the number of documents in the candidate document library that contain query term t.

[0194] where for a certain query term and candidate document, the term frequency is fixed, but the inverse document frequency varies with the number of documents in the candidate document library. Therefore, the weight of the same query term in the same document is different when querying at different times.

[0195] Step 206, the server searches data according to the syntax tree and the real-time stream inverted index to obtain the recalled target document.

[0196] wherein in the present application, the block maximum soft intersection method can be used to search data from the real-time stream inverted index. Here, the block maximum soft intersection method will be introduced first.

[0197] The block maximum soft intersection method is based on block inverted index. This method calculates the following data and performs query according to these data:

[0198] 1. Key-value pair postings, the key data contained in the key-value pair is a query term q t , and the value data is the inverted list corresponding to the query term q t ;

[0199] 2. The document fd pointed to by the first inverted pointer in postings;

[0200] 3. Result heap H, which stores the highest score of the preset number of results found by the algorithm so far;

[0201] 4. The lowest score θ of all documents in result heap H.

[0202] In the query using the block maximum soft intersection method, each word corresponds to a pointer in the block inverted list, which indicates the progress of the algorithm, i.e. the current calculated document. First, the IDs of the documents pointed to by the pointers in the multiple inverted lists in the postings are sorted in ascending order, and then the UB values of the inverted blocks where each pointer is located are added one by one in the sorted order. When the accumulated UB value is greater than θ, the query word corresponding to the inverted block whose accumulated value is just greater than θ is set as the axis keyword (pterm), and the document pointed to by the pointer in the inverted list where the inverted block is located is set as the pivot. At this time, if fd is equal to pivot, it is determined that pivot is a candidate document, which is inserted into the result heap H, and θ is updated. If fd is less than pivot, fd is jumped to the starting position of the next block greater than or equal to pivot or the next block, and the algorithm enters the next loop. If the accumulated UB value of all inverted blocks is still less than θ, fd is jumped to the next inverted block to enter the next loop.

[0203] The method can effectively prune the query process, and does not need to calculate the scores of all non-target documents, thereby greatly improving the efficiency of data search.

[0204] In the present application, when the block maximum soft intersection method is used for data search, the inverted blocks need to be processed according to the logical nodes in the syntax tree to obtain the virtual inverted blocks corresponding to the logical nodes. And according to the virtual inverted blocks, the data search is performed to obtain the corresponding recall results. The method of the present application fully considers the semantic information in the query data, avoids recall bias, and improves the accuracy of data search.

[0205] Specifically, the nodes in the syntax tree include and nodes, or nodes, word nodes and must-keep words. The and node indicates that the recall result needs to contain all query words in the node, the or node indicates that only any one of the query words in the node is required, and the must-keep word indicates that the recall result must contain the query word.

[0206] As Figure 8The diagram illustrates the processing of the inverted block corresponding to a node during the calculation process. Specifically, as shown in the figure, when a node contains three query terms: "China," "People," and "Bank," the maximum soft intersection algorithm requires the start ID, end ID, and UB value of the inverted block corresponding to that node to be calculated. To recall the query terms contained in the node, the inverted list with the fewest documents in the node can be used as a candidate inverted list, i.e., the inverted list corresponding to the query term "People." Then, the start ID and end ID of the inverted block pointed to by the current pointer are determined from this inverted list as the start ID and end ID of the virtual inverted block corresponding to the node. For example, if the start ID of the inverted block pointed to by the pointer in the candidate inverted list is 100 and the end ID is 1000, then the start ID of the virtual inverted block corresponding to the node is 100 and the end ID is 1000. Further, the UB value of the virtual inverted block corresponding to the node is determined. First, search other inverted lists for all inverted blocks that cover the range of the virtual inverted block. Specifically, find two inverted blocks in the inverted list corresponding to the query term "China" and three inverted blocks in the inverted list corresponding to the query term "bank". Then, add the maximum UB values ​​of the inverted blocks selected from each inverted list, i.e., 0.3 + 0.36 + 0.37 = 1.03, and use this as the UB value of the virtual inverted block corresponding to the node.

[0207] like Figure 9 The diagram illustrates the processing of inverted blocks corresponding to OR nodes during the calculation process. For a block inverted list corresponding to multiple query terms for an OR term, the minimum value of the starting ID of the inverted block currently pointed to by the pointer in each inverted list is obtained as the starting ID of the virtual inverted block of the OR node, and the maximum value of the ending ID of the inverted block currently pointed to by the pointer in each inverted list is obtained as the ending ID of the virtual inverted block of the OR node. Then, the maximum value of the UB value of the inverted block currently pointed to by the pointer in each inverted list is used as the UB value of the virtual inverted block of the OR node. Thus, when the block maximization soft intersection algorithm is used to calculate the inverted block corresponding to the above logical node, the above virtual inverted block is used for calculation, and the query terms corresponding to the above logical node can be retrieved.

[0208] For essential keywords in the query, since these keywords must be included in the recall results, the block-maximization soft intersection algorithm requires accumulating UB values ​​until they exceed θ to select candidate documents. This can lead to the algorithm discarding any query keyword in the recall results. To ensure that the recall results include essential keywords, this application requires that when accumulating UB values, not only is the accumulated value greater than θ, but the UB values ​​of the inverted blocks corresponding to the essential keywords must also be accumulated. This guarantees that the candidate documents will always contain essential keywords.

[0209] In the embodiment, in order to support efficient parallelization calculation, the application can split the real-time stream inverted index to obtain multiple real-time stream inverted indexes. Then multiple threads are used for parallel optimization. In this way, when a new document is added to the candidate document library, the ID value given to it is calculated according to the number of threads, and it is inserted into the split inverted index corresponding to the calculation result. When querying, multiple threads respectively perform block maximum soft intersection calculation on multiple real-time stream inverted indexes, and find a certain number of results respectively. Then, the results found respectively are summarized and scored and ranked, and the highest number of target documents are selected as target documents. For example, three threads are used for calculation, and each thread queries 150 documents, and a total of 300 documents are obtained. Then the 300 documents can be used as target documents for recall.

[0210] In some cases, two calculations can be performed using multiple threads. After a certain number of documents are found by the first multiple threads using the block maximum soft intersection algorithm, the score of the lowest scoring document in each thread is compared with the lowest score in the found target documents. For example, the score of the 150th document in thread A is compared with the score of the 300th document in the aforementioned 300 target documents. If the lowest score queried by the thread is less than the lowest score in the aforementioned target documents, the thread can be terminated. If the lowest score queried by the thread is greater than the lowest score in the aforementioned target documents, the thread can be calculated a second time, and the number of documents recalled by the thread is expanded in the second calculation.

[0211] Step 207, the server sends the target documents to the terminal for display.

[0212] When a preset number of target documents matching the query data are queried, the server can send the target documents to the terminal for display. Thus, the data search process is completed.

[0213] According to the above description, the data search method provided by the application can obtain query data, the query data including at least one query word; perform semantic analysis on the query data, and construct a syntax tree corresponding to the query data according to the analysis result; calculate the weight value of each query word in the multiple candidate documents included in the candidate document library, and construct a first inverted index list corresponding to the query data based on the weight value; update the first inverted index list according to the logical nodes in the syntax tree to obtain a second inverted index list; and query a preset number of target candidate documents matching the query data based on the second inverted index list.

[0214] Therefore, the data search method provided in this application can perform semantic analysis on the acquired query data, construct a syntax tree based on the results of the semantic analysis, and update the inverted index list based on the nodes in the syntax tree. This allows the semantic information of the query data to be fully considered when recalling data based on the index list, avoiding recall bias and thus enabling the retrieval of more accurate documents. This method can effectively improve the accuracy of data search.

[0215] To better implement the above methods, this application also provides a data search device that can be integrated into a terminal or server.

[0216] For example, such as Figure 10 The diagram shown is a structural schematic of a data search device provided in an embodiment of this application. The data search device may include an acquisition unit 301, an analysis unit 302, a construction unit 303, an update unit 304, and a query unit 305, as follows:

[0217] The acquisition unit 301 is used to acquire query data, which includes at least one query term.

[0218] Analysis unit 302 is used to perform semantic analysis on the query data and construct a syntax tree corresponding to the query data based on the analysis results;

[0219] Construction unit 303 is used to calculate the weight value of each query term in multiple candidate documents contained in the candidate document library, and construct the first inverted index list corresponding to the query data based on the weight value;

[0220] Update unit 304 is used to update the first inverted index list based on the logical nodes in the syntax tree to obtain the second inverted index list;

[0221] The query unit 305 is used to query a preset number of target candidate documents that match the query data based on the second inverted index list.

[0222] In some embodiments, the analysis unit includes:

[0223] The first analysis subunit is used to perform semantic analysis on the query data to determine the phrase data in the query data and the synonyms of each query term. The phrase data contains at least two related query terms.

[0224] The first construction subunit is used to construct a syntax tree with phrase data as AND nodes and each query term and its corresponding synonym as OR nodes.

[0225] In some embodiments, the first inverted index list is a block inverted index list, and the update unit includes:

[0226] The first determining sub-unit is configured to determine, in the first inverted index list, a plurality of first block inverted indexes corresponding to each logical node in the syntax tree, wherein each block inverted index comprises a plurality of inverted blocks.

[0227] The second determining sub-unit is configured to determine, according to the logical relationship of each logical node and the corresponding plurality of first block inverted indexes, a second block inverted index corresponding to each logical node.

[0228] The replacing sub-unit is configured to replace the plurality of first block inverted indexes corresponding to each logical node in the first inverted index list with the corresponding second block inverted index, to obtain a second inverted index list.

[0229] In some embodiments, the second determining sub-unit comprises:

[0230] The first determining module is configured to, when the logical relationship of the logical node is AND, determine a candidate block inverted index with the least number of documents in the plurality of first block inverted indexes.

[0231] The second determining module is configured to determine a first inverted block in the candidate block inverted index, and determine an associated inverted block in other block inverted indexes based on the first inverted block, wherein the other block inverted indexes are block inverted indexes other than the candidate block inverted index in the plurality of first block inverted indexes.

[0232] The first operation module is configured to perform AND operation on a parameter of the first inverted block and a parameter of the associated inverted block, to obtain a second inverted block corresponding to the first inverted block.

[0233] The third determining module is configured to determine the second block inverted index corresponding to the logical node according to the second inverted block.

[0234] In some embodiments, the data searching apparatus provided in the present application further comprises:

[0235] The fourth determining module is configured to, when the logical relationship of the logical node is OR, determine a third inverted block in each first block inverted index, to obtain a plurality of third inverted blocks.

[0236] The second operation module is configured to perform OR operation on parameters of the plurality of third inverted blocks, to obtain a fourth inverted block.

[0237] The fifth determining module is configured to determine the second block inverted index corresponding to the logical node according to the fourth inverted block.

[0238] In some embodiments, the computing unit comprises:

[0239] The first obtaining sub-unit is configured to obtain a number of times that a target query word appears in each candidate document and a document length of each candidate document.

[0240] The first calculation sub-unit is configured to calculate a first weight parameter of the target query word in each candidate document according to the number of occurrences of the target query word in each candidate document and the document length of each candidate document.

[0241] The second acquisition sub-unit is configured to acquire a first number of candidate documents and a second number of candidate documents containing the target query word in the candidate documents.

[0242] The second calculation sub-unit is configured to calculate a second weight parameter of the target query word in each candidate document according to the first number and the second number.

[0243] The third calculation sub-unit is configured to calculate a weight value of the target query word in each candidate document according to the first weight parameter and the second weight parameter.

[0244] The first traversal sub-unit is configured to traverse each query word to obtain the weight value of each query word in each candidate document.

[0245] In some embodiments, the calculation unit comprises:

[0246] The third determination sub-unit is configured to determine a plurality of target candidate documents associated with the target query word in the plurality of candidate documents, and create a document number for each target candidate document in descending order.

[0247] The fourth determination sub-unit is configured to determine a target weight value of the target query word in each target candidate document according to the weight value.

[0248] The second construction sub-unit is configured to construct an inverted index of the target query word based on the document number and the target weight value of each target candidate document.

[0249] The second traversal sub-unit is configured to traverse each query word to generate an inverted index corresponding to each query word to obtain a first inverted index list.

[0250] In some embodiments, the data search apparatus provided by the present application further comprises:

[0251] The division sub-unit is configured to divide the inverted index corresponding to each query word into a plurality of inverted blocks.

[0252] The fifth determination sub-unit is configured to determine an inverted block parameter of each inverted block based on the document number of the document contained in each inverted block and the weight value of the query word in the document.

[0253] The first update sub-unit is configured to update the first inverted index list according to the inverted block parameter of each inverted block.

[0254] In some embodiments, the data search apparatus provided by the present application further comprises:

[0255] The sixth determining sub-unit is configured to determine, when it is detected that a new candidate document is added to the candidate document library, an associated query word contained in the new candidate document based on the at least one query word, the associated query word belonging to the at least one query word;

[0256] The fourth calculating sub-unit is configured to calculate a weight value of each associated query word in the new candidate document.

[0257] The second updating sub-unit is configured to number the new candidate document, and update the first inverted index list based on the number of the new candidate document and the weight value of each associated query word in the new candidate document.

[0258] In some embodiments, the query unit comprises:

[0259] The splitting sub-unit is configured to split each inverted index in the second inverted index list into a plurality of sub-inverted indexes, to obtain a plurality of sub-inverted index lists.

[0260] The first query sub-unit is configured to simultaneously query, based on each sub-inverted index list, a candidate document matching the query data by using a plurality of threads, to obtain a plurality of query results.

[0261] The sixth determining sub-unit is configured to determine, from the plurality of query results, a preset number of target candidate documents matching the query data.

[0262] In some embodiments, the query unit is configured to:

[0263] The second analyzing sub-unit is configured to perform semantic analysis on the query data, to determine a keyword contained in the query data.

[0264] The second query sub-unit is configured to query, based on the keyword and the second inverted index list, a preset number of target candidate documents matching the query data.

[0265] In implementation, the above various units can be implemented as independent entities, or can be combined as the same or several entities, and the specific implementation of the above various units can be referred to the method embodiments above, which will not be described herein.

[0266] As described above, the data search device provided in this application embodiment acquires query data through an acquisition unit 301, the query data including at least one query term; an analysis unit 302 performs semantic analysis on the query data and constructs a syntax tree corresponding to the query data based on the analysis results; a construction unit 303 calculates the weight value of each query term in multiple candidate documents contained in the candidate document library and constructs a first inverted index list corresponding to the query data based on the weight value; an update unit 304 updates the first inverted index list according to the logical nodes in the syntax tree to obtain a second inverted index list; and a query unit 305 queries a preset number of target candidate documents that match the query data based on the second inverted index list.

[0267] Therefore, the data search method provided in this application can perform semantic analysis on the acquired query data, construct a syntax tree based on the results of the semantic analysis, and update the inverted index list based on the nodes in the syntax tree. This allows the semantic information of the query data to be fully considered when recalling data based on the index list, avoiding recall bias and thus enabling the retrieval of more accurate documents. This method can effectively improve the accuracy of data search.

[0268] This application also provides a computer device, which can be a terminal or a server, such as... Figure 11 The diagram shown is a structural schematic of the computer device provided in this application. Specifically:

[0269] The computer device may include components such as a processing unit 401 with one or more processing cores, a storage unit 402 with one or more storage media, a power module 403, and an input module 404. Those skilled in the art will understand that... Figure 11 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0270] The processing unit 401 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the storage unit 402, and by calling data stored in the storage unit 402. Optionally, the processing unit 401 may include one or more processing cores; preferably, the processing unit 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processing unit 401.

[0271] The storage unit 402 can be used to store software programs and modules, and the processing unit 401 executes various functional applications and data processing by running the software programs and modules stored in the storage unit 402. The storage unit 402 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function (such as a sound playing function, an image playing function, a webpage access, etc.), and the like; and the data storage area can store data created according to the use of the computer device, etc. In addition, the storage unit 402 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the storage unit 402 can also include a memory controller to provide access of the processing unit 401 to the storage unit 402.

[0272] The computer device further includes a power module 403 for powering various components. Preferably, the power module 403 can be logically connected to the processing unit 401 through a power management system, so as to realize functions of managing charging, discharging, and power consumption management, etc. through the power management system. The power module 403 can also include one or more than one direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power state indicator, and the like.

[0273] The computer device can further include an input module 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0274] Although not shown, the computer device can also include a display unit, etc., which will not be described here. Specifically, in the present embodiment, the processing unit 401 in the computer device loads one or more than one executable file corresponding to the process of an application program into the storage unit 402 according to the following instructions, and runs the application program stored in the storage unit 402 by the processing unit 401, so as to realize various functions, as follows:

[0275] Obtaining query data, the query data including at least one query word; performing semantic analysis on the query data, and constructing a syntax tree corresponding to the query data according to an analysis result; calculating a weight value of each query word in a plurality of candidate documents included in a candidate document library, and constructing a first inverted index list corresponding to the query data based on the weight value; updating the first inverted index list according to a logical node in the syntax tree, to obtain a second inverted index list; and querying a preset number of target candidate documents matching the query data based on the second inverted index list.

[0276] It should be explained that the computer device provided by the embodiments of the present application and the method in the above embodiments belong to the same concept, and the specific implementation of the above operations can be referred to the previous embodiments, which will not be repeated here.

[0277] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by related hardware controlled by the instructions, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0278] Therefore, the embodiments of the present application provide a computer readable storage medium, which stores a plurality of instructions capable of being loaded by a processor to execute the steps in any method provided by the embodiments of the present application. For example, the instructions can execute the following steps:

[0279] Obtaining query data, the query data comprising at least one query word; performing semantic analysis on the query data, and constructing a syntax tree corresponding to the query data according to the analysis result; calculating a weight value of each query word in a plurality of candidate documents included in a candidate document library, and constructing a first inverted index list corresponding to the query data based on the weight value; updating the first inverted index list according to a logical node in the syntax tree to obtain a second inverted index list; and querying a preset number of target candidate documents matching the query data based on the second inverted index list.

[0280] The specific implementation of the above operations can be referred to the previous embodiments, which will not be repeated here.

[0281] The computer readable storage medium can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0282] Since the instructions stored in the computer readable storage medium can execute the steps in any method provided by the embodiments of the present application, the beneficial effects that can be achieved by any method provided by the embodiments of the present application can be achieved, which will be described in detail in the previous embodiments, and will not be repeated here.

[0283] According to an aspect of the present application, a computer program product or a computer program is provided, which includes computer instructions stored in a storage medium. The processor of the computer device reads the computer instructions from the storage medium, and the processor executes the computer instructions to make the computer device execute the method provided in any optional implementation manner of the above data search method.

[0284] The data search method, device, computer device and computer readable storage medium provided by the embodiment of the present application are described in detail above, the principle and implementation manner of the present application are described by applying specific examples in this paper, and the above embodiment is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed, and on the basis of the above, the content of the specification should not be understood as the limitation of the present application.

Claims

1. A data search method characterized by, The method comprises: acquiring query data, the query data comprising at least one query word; performing semantic analysis on the query data to determine phrase data in the query data and synonyms of each query word, the phrase data comprising at least two associated query words; constructing a syntax tree with the phrase data as an and node and each query word and corresponding synonym as an or node, wherein the multiple query words corresponding to the and node need to be recalled simultaneously in recall, and the multiple query words corresponding to the or node only need one to be recalled in recall; calculating a weight value of each query word in a plurality of candidate documents contained in a candidate document library, and constructing a first inverted index list corresponding to the query data based on the weight value, the first inverted index list being a block inverted index list; determining, in the first inverted index list, a plurality of first block inverted indexes corresponding to each logical node in the syntax tree, the block inverted index comprising a plurality of inverted blocks; determining a second block inverted index corresponding to each logical node according to a logical relationship of each logical node and the corresponding plurality of first block inverted indexes; replacing the plurality of first block inverted indexes corresponding to each logical node in the first inverted index list with the corresponding second block inverted index to obtain a second inverted index list; querying a preset number of target candidate documents matching the query data based on the second inverted index list.

2. The method of claim 1, wherein, The method further comprises: when the logical relationship of the logical node is or logic, determining a third inverted block in each first block inverted index to obtain a plurality of third inverted blocks; performing or operation on parameters of the plurality of third inverted blocks to obtain a fourth inverted block; determining the second block inverted index corresponding to the logical node according to the fourth inverted block. The method further comprises:

3. The method of claim 2, wherein, when the logical relationship of the logical node is or logic, determining a third inverted block in each first block inverted index to obtain a plurality of third inverted blocks; performing or operation on parameters of the plurality of third inverted blocks to obtain a fourth inverted block; determining the second block inverted index corresponding to the logical node according to the fourth inverted block. The method further comprises:

4. The method of claim 1, wherein, acquiring a number of times that the target query word appears in each candidate document and a document length of each candidate document; calculating a first weight parameter of the target query word in each candidate document according to the number of times that the target query word appears in each candidate document and the document length of each candidate document; acquiring a first number of candidate documents and a second number of candidate documents containing the target query word in the candidate documents; calculating a second weight parameter of the target query word in each candidate document according to the first weight parameter and the second number of candidate documents containing the target query word in the candidate documents. calculating a second weight parameter of the target query word in each candidate document according to the first quantity and the second quantity; calculating a weight value of the target query word in each candidate document according to the first weight parameter and the second weight parameter; traversing each query word to obtain a weight value of each query word in each candidate document.

5. The method of claim 1, wherein, The constructing the first inverted index list corresponding to the query data based on the weight value comprises: determining a plurality of target candidate documents associated with a target query word in the plurality of candidate documents, and creating a document number for each target candidate document in descending order; determining a target weight value of the target query word in each target candidate document according to the weight value; constructing an inverted index of the target query word based on the document number of each target candidate document and the target weight value; traversing each query word to generate an inverted index corresponding to each query word to obtain a first inverted index list.

6. The method of claim 5, wherein, The method further comprises: dividing the inverted index corresponding to each query word into a plurality of inverted blocks; determining an inverted block parameter of each inverted block based on the document number of the document contained in each inverted block and the weight value of the query word in the document; updating the first inverted index list according to the inverted block parameter of each inverted block.

7. The method of claim 5, wherein, The method further comprises: when detecting that a new candidate document is added to the candidate document library, determining associated query words contained in the new candidate document based on the at least one query word, the associated query words belonging to the at least one query word; calculating a weight value of each associated query word in the new candidate document; numbering the new candidate document and updating the first inverted index list based on the number of the new candidate document and the weight value of each associated query word in the new candidate document.

8. The method of claim 1, wherein, The querying a preset number of target candidate documents matching the query data based on the second inverted index list comprises: splitting each inverted index in the second inverted index list into a plurality of sub-inverted indexes to obtain a plurality of sub-inverted index lists; simultaneously querying candidate documents matching the query data based on each sub-inverted index list using a plurality of threads to obtain a plurality of query results; determining a preset number of target candidate documents matching the query data according to the plurality of query results.

9. The method of claim 1, wherein, The querying a preset number of target candidate documents matching the query data based on the second inverted index list comprises: performing semantic analysis on the query data to determine keywords contained in the query data; querying a preset number of target candidate documents matching the query data based on the keywords and the second inverted index list.

10. A data search apparatus characterized by comprising: The device comprises: an acquisition unit configured to acquire query data, the query data comprising at least one query word; An analysis unit is configured to perform semantic analysis on the query data, determine phrase data in the query data and synonyms of each query word, the phrase data comprising at least two associated query words; construct a syntax tree with the phrase data as an and node and each query word as an or node, wherein the query words corresponding to the and node need to be recalled simultaneously in recall, and the query words corresponding to the or node need to be recalled only one in recall; A construction unit is configured to calculate a weight value of each query word in a plurality of candidate documents included in a candidate document library, and construct a first inverted index list corresponding to the query data based on the weight value, the first inverted index list being a block inverted index list; An updating unit is configured to determine a plurality of first block inverted indexes corresponding to each logical node in the syntax tree in the first inverted index list, the block inverted index comprising a plurality of inverted blocks; determine a second block inverted index corresponding to each logical node according to a logical relationship of each logical node and the plurality of first block inverted indexes; and replace the plurality of first block inverted indexes corresponding to each logical node in the first inverted index list with the corresponding second block inverted index, to obtain a second inverted index list; A query unit is configured to query a preset number of target candidate documents matching the query data based on the second inverted index list.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a plurality of instructions, which are adapted to be loaded by a processor to perform the steps in the data search method of any one of claims 1 to 9.

12. A computer device, comprising: A computer program product includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps in the data search method of any one of claims 1 to 9.

13. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the steps in the data search method of any one of claims 1 to 9.

Images