Search engine query optimization method, device, and electronic device

Abstract

The present application provides a search engine query optimization method, device, electronic device, and computer-readable storage medium; the method includes: obtaining a search engine's historical query record, the historical query record including multiple query statements, generating a syntax tree based on the terms included in the query statements; adjusting the execution order of the terms in the syntax tree based on the amount of information included in the inverted list corresponding to each term in the syntax tree; searching based on the syntax tree after the execution order is adjusted, and determining the execution order of each term in the syntax tree based on the search results, and the execution order minimizes the overall search cost; wherein the syntax tree is used for the search engine to search when receiving the query statement. Through the present application, search consumption can be reduced.

Description

Technical Field

[0001] The present application relates to search technology, and in particular to a search engine query optimization method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to achieve the best results.

[0003] Artificial intelligence is widely used in search engines. However, as the amount of information continues to grow, the amount of information that needs to be traversed increases accordingly, and search time increases accordingly. In this case, related technologies use complex search strategies, which will increase search time and affect the user experience of search engines. Summary of the Invention

[0004] Embodiments of the present application provide a search engine query optimization method, device, electronic device, and computer-readable storage medium, which can reduce search consumption.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present invention provides a search engine query optimization method, comprising:

[0007] Obtaining historical query records of a search engine, the historical query records including multiple query statements, and generating a syntax tree based on the terms included in the query statements;

[0008] Adjusting the execution order of the words in the syntax tree based on the amount of information included in the inverted list corresponding to each word in the syntax tree;

[0009] Searching based on the syntax tree after adjusting the execution order, and determining the execution order of each word in the syntax tree according to the search results, wherein the execution order minimizes the overall search cost;

[0010] The syntax tree is used by the search engine to search when receiving the query statement.

[0011] The present invention provides a search engine query optimization device, comprising:

[0012] A generation module is used to obtain historical query records of a search engine, wherein the historical query records include multiple query statements, and generate a syntax tree based on the terms included in the query statements;

[0013] an adjusting module, configured to adjust the execution order of the words in the syntax tree based on the amount of information included in the inverted list corresponding to each word in the syntax tree;

[0014] A determination module is used to search based on the syntax tree after adjusting the execution order, and determine the execution order of each word in the syntax tree according to the search results, and the execution order minimizes the overall search cost; wherein, the syntax tree is used for the search engine to search when receiving the query statement.

[0015] In the above solution, the generating module is further used to:

[0016] Performing word segmentation on the query statement to obtain at least one word element;

[0017] Performing assimilation processing on the at least one word to obtain at least one word;

[0018] Performing grammatical analysis on the at least one word to obtain a logical operator representing a logical relationship between the words;

[0019] The syntax tree is generated based on the at least one term and the logical operator.

[0020] In the above solution, when the syntax tree includes at least two adjacent AND operators, the adjustment module is further configured to:

[0021] In the syntax tree, determining the amount of information corresponding to the at least two adjacent words connected by the operator;

[0022] The execution order of the word with the least amount of corresponding information is adjusted to the first place.

[0023] In the above solution, when the syntax tree includes at least one non-operator, the adjustment module is further configured to:

[0024] In the syntax tree, determining the number of word corresponding information of the at least one non-operator connection;

[0025] Adjusting the execution order of the at least one non-operator-connected word to after words connected by other logical operators, wherein the other logical operators include an AND operator and an OR operator;

[0026] When the syntax tree includes at least two non-operators, the execution order of the words connected by the at least two non-operators is consistent with the order of the quantity of information corresponding to the corresponding words.

[0027] In the above solution, the determining module is further used to:

[0028] Adjusting the execution order of other words in the syntax tree to obtain multiple combinations consisting of all words in the syntax tree, wherein the other words are words in the syntax tree other than the word with the least amount of corresponding information;

[0029] Searching the multiple combinations respectively to obtain corresponding multiple search results and a search volume corresponding to each search result;

[0030] Determine the order in which each term in the combination that results in the smallest search volume should be executed.

[0031] In the above solution, the determining module is further used to:

[0032] Adjusting the execution order of other words in the syntax tree to obtain multiple combinations consisting of all words in the syntax tree, wherein the other words are words in the syntax tree other than the word with the least amount of corresponding information;

[0033] Searching the multiple combinations respectively to obtain corresponding multiple search results and the search time corresponding to each search result;

[0034] Determine the execution order of each word in the combination corresponding to the minimum search time.

[0035] In the above solution, the query optimization device of the search engine further includes a search module for:

[0036] Receiving a search request carrying a query statement sent by the search engine;

[0037] Determining a syntax tree that matches the query statement in the search request;

[0038] Searching based on the execution order of each word in the matching syntax tree to obtain at least one search result;

[0039] The at least one search result is sorted in descending order of relevance to the query statement and returned to the search engine.

[0040] In the above solution, the search module is further used to:

[0041] Obtaining multiple filter conditions corresponding to the query statements in the historical query records;

[0042] reorganizing the plurality of filtering conditions, and constructing a plurality of filters based on the reorganized plurality of filtering conditions;

[0043] Searching the query statement based on the multiple filters respectively to obtain the search time corresponding to each filter;

[0044] The filter with the shortest search time is used as the filter of the query statement, and the filter is used to perform a search when a search request carrying the query statement is received.

[0045] In the above solution, the search module is further used to:

[0046] Obtaining multiple filter conditions corresponding to the query statements in the historical query records;

[0047] Determine the amount of filtered information corresponding to each filtering condition;

[0048] The multiple filter conditions are sorted in ascending order according to the amount of filtered information, and a filter is constructed based on the multiple filter conditions sorted in ascending order. The filter is used to perform a search when a search request carrying the query statement is received.

[0049] An embodiment of the present application further provides an electronic device, including:

[0050] a memory for storing executable instructions;

[0051] The processor is configured to implement the search engine query optimization method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0052] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute instructions to implement a search engine query optimization method provided in an embodiment of the present application.

[0053] The embodiments of the present application have the following beneficial effects:

[0054] A syntax tree is generated based on the query statements in the historical query records, and the execution order of the words in the syntax tree is adjusted based on the amount of information included in the inverted list corresponding to each word in the syntax tree. Because the execution order of words in the syntax tree is different, the corresponding search volume is also different, so adjusting the execution order of words in the syntax tree can reduce the search volume; then, a search is performed based on the syntax tree after the execution order is adjusted, and the execution order of each word in the syntax tree is adjusted again according to the search results. In this way, the execution order of each word that minimizes the overall search cost can be determined based on the feedback of the actual search results. When searching based on the syntax tree obtained by the final adjustment, the search consumption is effectively reduced and the search efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1A 1 is a schematic diagram of the architecture of the search system 10 provided in an embodiment of the present application;

[0056] Figure 1B 1 is a schematic diagram of the architecture of the search system 10 provided in an embodiment of the present application;

[0057] Figure 2A This is a flow chart of a search engine query optimization method provided by an embodiment of the present application;

[0058] Figure 2B This is a flow chart of a search engine query optimization method provided by an embodiment of the present application;

[0059] Figure 3 is a schematic diagram of a syntax tree provided in an embodiment of the present application;

[0060] Figure 4 Schematic diagram of a syntax tree after adjusting the execution order provided by an embodiment of the present application;

[0061] Figure 5 This is a schematic diagram of the interactive flow of the search engine query optimization method provided by the embodiment of the present application;

[0062] Figure 6 It is a structural diagram of the server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0064] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0066] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0067] 1) Search recall: that is, searching through query statements and returning matching search results.

[0068] 2) Posted List: A posting list contains all the information in which a certain word appears, along with the position of the word within that information. Each entry in the list is called a posting item. The information in the list can be documents, web pages, etc. Using the posting list, you can determine which information contains a certain word.

[0069] Query optimizers are categorized as rule-based and cost-based. Rule-based optimizers use pre-defined rules to simplify query statements, thereby improving query efficiency. Cost-based optimizers use a variety of optimization methods, such as statistics, cardinality estimation, transformation rules, cost models, and plan space search engines, to improve query efficiency.

[0070] In search engines provided by related technologies, the order in which terms in a syntax tree are executed, or the order in which filter conditions are executed, is in the order specified by the business. Some well-optimized search engines incorporate rule-based optimizers to pre-merge multiple filter conditions in a filter, or to convert deep syntax trees into shallow syntax trees. This means that complex query expressions corresponding to deep syntax trees can be replaced with simpler query expressions (corresponding to shallow syntax trees) without changing the semantics.

[0071] When searching for words in a syntax tree, search efficiency is affected by the order in which the words are executed. For example, if the query expression is (a or b) and c and d, and the amount of information included in the inverted list corresponding to d is very small, the first two searches, namely searching for "a or b" and "(a or b) and c" respectively, will cause a lot of meaningless consumption, such as consumption of resources such as the Central Processing Unit (CPU). In this case, because the rule-based optimizer cannot consider the amount of information corresponding to each word, it cannot determine the execution order of each word to minimize consumption, and cannot solve the problem of high consumption. Moreover, the rule-based optimizer cannot solve the problems at the non-grammatical expression level in the search, for example, the query statement requires finding images with a resolution higher than 480×800.

[0072] In order to solve the technical problem of high resource consumption during search in the related art, the embodiments of the present application provide a search engine query optimization method, device, electronic device and computer-readable storage medium, which can reduce search consumption and improve search efficiency.

[0073] The search engine query optimization method provided in the embodiments of the present application can be implemented by various electronic devices, for example, it can be implemented by a terminal alone, or it can be implemented by a server and a terminal in collaboration. For example, the server alone executes the search engine query optimization method described below, or the terminal and the server collaborate to execute the search engine query optimization method described below, for example, the terminal sends a search request carrying a query statement to the server, the server determines the execution order of each word in the syntax tree based on the search request, searches based on the syntax tree, obtains search results, and returns the search results to the terminal.

[0074] The electronic device for search engine query optimization provided in the embodiments of the present application can be various types of terminal devices or servers, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected through wired or wireless communication, and the embodiments of the present application do not limit this.

[0075] Taking servers as an example, it can be a server cluster deployed in the cloud, opening artificial intelligence cloud services (AIaaS, AI as a Service) to users. The AIaaS platform will split several common AI services and provide independent or packaged services in the cloud. This service model is similar to an AI theme mall. All users can access and use one or more artificial intelligence services provided by the AIaaS platform through the application programming interface.

[0076] For example, one AI cloud service may be a search engine query optimization service, where a cloud server encapsulates the search engine query optimization program provided by an embodiment of the present application. In response to a user's search operation on the search engine, the terminal invokes the search engine query optimization service in the cloud service, causing the server deployed in the cloud to invoke the encapsulated search engine query optimization program, determine the execution order of each word in the syntax tree, perform a search based on the syntax tree, obtain search results, and return the search results to the terminal.

[0077] The following description will be made by taking the server and the terminal cooperating to implement the search engine query optimization method provided in the embodiment of the present application as an example. Figure 1A , Figure 1AThe terminal 400 is connected to the server 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0078] In some embodiments, the search in terminal 400 responds to a search operation on a search engine, and sends a search request carrying a query statement to server 200. Server 200 searches the database based on the query statement in the search request, determines the query expression with the least consumption corresponding to the query statement, that is, determines the execution order of each word in the corresponding syntax tree. The search is performed based on the syntax tree (that is, based on the query expression corresponding to the syntax tree), and the corresponding search results are obtained, and the search results are returned to terminal 400 for display in terminal 400.

[0079] The present application embodiment can also be implemented through blockchain technology, see Figure 1B , Figure 1B Schematic diagram of the architecture of the search system 10 provided in an embodiment of the present application. After each query is completed, the terminal 400 sends the historical query record to the blockchain network 500 (exemplarily showing nodes 510-1, 510-2, and 510-3 included in the blockchain network 500). When the historical query record in the blockchain network 500 reaches a certain scale, the historical query record is sent to the server 200. The server 200 obtains the query statement in the historical query record, generates a syntax tree based on the words included in the query statement, and determines the execution order of each word in the syntax tree that minimizes consumption. The query expression corresponding to the syntax tree after the execution order is re-determined is saved in the database. When the server 200 receives a search request for the query statement, it searches based on the corresponding syntax tree, obtains search results, and returns the search results to the terminal 400.

[0080] The following will illustrate the search engine query optimization method provided by the embodiment of the present application with reference to the accompanying drawings. The execution entity of the following search engine query optimization method may be a server, and specifically, the server may be implemented by running the various computer programs described above. Of course, based on the understanding of the following, it is not difficult to see that the search engine query optimization method provided by the embodiment of the present application may also be implemented collaboratively by a terminal and a server.

[0081] See also Figure 2A , Figure 2A This is a flow chart of the search engine query optimization method provided by the embodiment of the present application, which will be combined with Figure 2A The steps shown are explained.

[0082] In step 101, historical query records of a search engine are obtained, where the historical query records include multiple query statements.

[0083] In some embodiments, when the terminal sends a search request to the server through a search engine, the server records relevant information of the search request in the historical query record. The relevant information of the search request includes the query statement, user identification number, search time, search results, etc. The query statement can be a paragraph, such as "Predict the weather for the next week", or some keywords, such as "Braised fish", "Cooking method", etc. The server obtains multiple query statements by acquiring the historical query records for a past period (such as the past week or the past month).

[0084] In step 102, a syntax tree is generated based on the words included in the query statement.

[0085] As Figure 2B shown Figure 2B is a schematic flowchart of the query optimization method of the search engine provided by an embodiment of the present application. In some embodiments, generating a syntax tree based on the words included in the query statement can be implemented through Figure 2B steps 1021 to 1024 therein.

[0086] In step 1021, the query statement is tokenized to obtain at least one token.

[0087] The server tokenizes the query statement through a tokenization component, including word segmentation, punctuation removal, and stop word removal. When the query statement is in Chinese, word segmentation can be performed through a dictionary or by segmenting characters individually. When the query statement is in English, word segmentation can be performed according to spaces. Stop words are adverbs without special meanings in a language, such as "this", "is" in English, and "is", "of" in Chinese, etc.

[0088] In some possible examples, when the query statement is "Tomorrow's weather", after tokenization, the obtained tokens are "Tomorrow" and "Weather".

[0089] In step 1022, at least one token is assimilated to obtain at least one word.

[0090] Taking English as an example, assimilating at least one token includes case conversion (for example, converting "STUDENT" to "student") and converting the token to its root form (for example, converting "cars" to "car"). When the query statement is in Chinese, after step 1021 is executed, step 1023 is executed.

[0091] In step 1023, at least one word is syntactically analyzed to obtain logical operators representing the logical relationships between the words.

[0092] For example, if at least one of the words is "tomorrow" and "weather," grammatical analysis shows that the logical relationship between these two words is parallel, and therefore the corresponding logical operator is "and." For another example, if the query is "Cat and mouse," the corresponding at least one of the words is "cat" and "mouse," and the logical operator between the two is "and."

[0093] In step 1024 , a syntax tree is generated based on the at least one term and the logical operator.

[0094] In some embodiments, the logical operator includes at least one of the following: an AND operator, an OR operator, and a NOT operator. Figure 3 As shown, Figure 3 is a schematic diagram of a syntax tree provided in an embodiment of the present application, Figure 3 The words in the syntax tree include a, b, c, and d, and a and b are connected by the or operator, (a or b) and c are connected by the and operator, and (a or b) and c and d are connected by the and operator. Through the syntax tree, you can clearly see the execution order of each word in the syntax tree.

[0095] In step 103, the execution order of the words in the syntax tree is adjusted based on the amount of information included in the inverted list corresponding to each word in the syntax tree.

[0096] In some embodiments, when a syntax tree includes at least two adjacent AND operators, the number of information corresponding to the words connected by the at least two adjacent AND operators is determined in the syntax tree; the execution order of the word with the least amount of corresponding information is adjusted to the first place. The number of information corresponding to the word connected by the AND operator is determined based on the search results in the historical query records. When the number of information corresponding to the word does not exist in the historical query results, a search is performed directly based on the word to determine the number of information corresponding to the word. Figure 3 As shown, Figure 3 The syntax tree shown includes two adjacent AND operators, connecting (a or b), c, and d respectively. The number of information contained in the inverted lists corresponding to a and b is 500,000, so the number of information corresponding to (a or b) is between 500,000 and 1,000,000, the number of information contained in the inverted list corresponding to c is 700,000, and the number of information contained in the inverted list corresponding to d is 800. Since the number of information corresponding to d is the least, we adjust Figure 3 The execution order of d is the first, and after adjustment, Figure 4 As shown, Figure 4 This is a schematic diagram of the syntax tree after the execution order is adjusted provided in an embodiment of the present application. Figure 4 The execution order in the syntax tree shown is d, (a or b), and c.

[0097] In some embodiments, when a syntax tree includes at least one not operator, the number of corresponding information items connected by the at least one not operator is determined in the syntax tree; and the execution order of the at least one not operator connected is adjusted to after the words connected by other logical operators, such as AND operators and OR operators. For example, if the query expression corresponding to the syntax tree is e not h and (f or g), then the not operator and the connected word h are adjusted to after (f or g), resulting in e and (f or g) not h.

[0098] When a syntax tree includes at least two non-operators, the execution order of the terms connected by the at least two non-operators is consistent with the order of the amount of information corresponding to the corresponding terms. For example, the query expression corresponding to the syntax tree is e not h not iand j. The amount of information included in the inverted list corresponding to h is 500,000, and the amount of information included in the inverted list corresponding to i is 700,000. Because the amount of information corresponding to h is smaller than that of i, the execution order of i is adjusted to before h. This way, more useless information can be filtered out earlier. The query expression corresponding to the syntax tree after the adjustment is e and j not i not h.

[0099] In step 104, a search is performed based on the syntax tree after the execution order is adjusted, and the execution order of each word in the syntax tree is determined according to the search results, and the execution order minimizes the overall search cost.

[0100] In some embodiments, step 103 only adjusts the execution order of some terms. In step 104, the execution order of all terms in the syntax tree, excluding those whose execution order has been adjusted, is adjusted to obtain multiple combinations consisting of all terms in the syntax tree. For example, if the query expression corresponding to the syntax tree is (a or b) and c and d, and after adjusting it to d and (a or b) and dc in step 103, further adjustments are made to the terms other than d, resulting in the combination "d and (a or b) and c" and the combination "d and c and (a or b)." After obtaining multiple combinations, these combinations are searched to obtain corresponding search results. In some possible examples, when the search cost is the search volume, the corresponding search volume is obtained along with the search results. For example, a search volume of 500,000 indicates that the server searched 500,000 data. Because the execution order of terms in different combinations varies, the corresponding search volume also varies. The execution order of the terms in the combination corresponding to the minimum search volume is the ideal execution order for each term. The query expression corresponding to the combination with the minimum search volume is stored in the database as the query expression corresponding to the query statement. It can be seen that this adjustment method can reduce the amount of data that the server needs to traverse, thereby alleviating the pressure on the server.

[0101] In some possible examples, when the search cost is the search time, the corresponding search time is also obtained when the search results are obtained. For example, when searching for a combination, the search time is 0.3 seconds. The execution order of each word in the combination corresponding to the minimum search time is determined, and the query expression corresponding to the combination with the minimum search time is stored in the database as the query expression corresponding to the query statement. In this way, when searching for the query statement, the search results can be obtained as quickly as possible and returned to the search engine, improving the user experience.

[0102] In some embodiments, when a server receives a search request carrying a query statement sent by a search engine, the server searches the database based on the query statement to determine the query expression corresponding to the query statement, and determines the syntax tree corresponding to the query expression, which matches the query statement. A search is performed based on the execution order of each word in the syntax tree to obtain at least one search result. The at least one search result is sorted in descending order of relevance to the query statement and returned to the search engine. For example, the search results are result 1, result 2, and result 3, the relevance of result 1 to the query statement is 70%, the relevance of result 2 to the query statement is 50%, and the relevance of result 3 to the query statement is 30%. The above three search results are returned to the search engine in the order of result 1, result 2, and result 3.

[0103] In some embodiments, a filter may be used to determine the search results for a query statement to address the issue of non-grammatical expressions in the search. The filter is composed of multiple filter conditions. When the filter is used to determine the search results for a query statement, the server determines the filter corresponding to the query statement through the following steps 1041 to 1044.

[0104] In step 1041, multiple filter conditions corresponding to the query statements in the historical query records are obtained.

[0105] For example, the query statement corresponds to multiple filter conditions: filter condition 1 "image", filter condition 2 "resolution higher than 480×800", and filter condition 3 "red".

[0106] In step 1042 , the plurality of filter conditions are reorganized, and a plurality of filters are constructed based on the reorganized plurality of filter conditions.

[0107] For example, by reorganizing the above three filter conditions, we can construct filter 1 which is sorted as filter condition 1, filter condition 2 and filter condition 3, filter 2 which is sorted as filter condition 1, filter condition 3 and filter condition 2, filter 3 which is sorted as filter condition 2, filter condition 1 and filter condition 3, filter 4 which is sorted as filter condition 2, filter condition 3 and filter condition 1, filter 5 which is sorted as filter condition 3, filter condition 2 and filter condition 1, and filter 6 which is sorted as filter condition 3, filter condition 1 and filter condition 2.

[0108] In step 1043 , the query statement is searched based on multiple filters respectively to obtain the search time corresponding to each filter.

[0109] In step 1044, the filter with the shortest search time is used as a filter for the query statement, and the filter is used to perform a search when a search request carrying the query statement is received.

[0110] For example, after searching the query using each of the six filters above, if the search time corresponding to each filter is the shortest, filter 2 will be used as the filter for the query, and both filter and query will be stored in the database. When the server receives a search request containing this query, it uses the filter to determine the search results and returns them to the search engine. This allows search results to be returned to the search engine as quickly as possible, improving the user experience.

[0111] In other embodiments, when determining the search results of a query statement through a filter, the server obtains multiple filter conditions corresponding to the query statement in the historical query record; determines the number of filtered information corresponding to each filter condition; sorts the multiple filter conditions in ascending order according to the number of filtered information, and constructs a filter based on the ascending order of the multiple filter conditions. For example, if there are two filter conditions, and the number of filtered information corresponding to filter condition A is 100,000, and the number of filtered information corresponding to filter condition B is 200,000, then the order of filter condition A is adjusted to be before filter condition B, resulting in a filter with the order of filter condition A and filter condition B.

[0112] In some possible examples, some filtering conditions, such as blacklist filtering conditions and city level restriction filtering conditions, will retain 99% of the data after filtering. Therefore, adjusting the sorting of these filtering conditions, which still have a large amount of data after filtering, in the filter can effectively improve search efficiency and thus speed up index recall performance.

[0113] It can be seen that the embodiment of the present application generates a syntax tree based on the query statements in the historical query records, and adjusts the execution order of the words in the syntax tree based on the amount of information included in the inverted list corresponding to each word in the syntax tree. Because the execution order of the words in the syntax tree is different, the corresponding search volume is also different. Therefore, adjusting the execution order of the words in the syntax tree can reduce the search volume; searching is performed based on the syntax tree after the execution order is adjusted, and the execution order of each word in the syntax tree is adjusted again according to the search results. In this way, the execution order of each word that minimizes the overall search cost can be determined based on the feedback of the actual search results, which is equivalent to introducing a cost-based optimizer. When searching based on the syntax tree obtained by the final adjustment, the search consumption is effectively reduced, the search efficiency is improved, and it has strong versatility.

[0114] In some embodiments, see Figure 5 , Figure 5 This is a schematic diagram of the interactive flow of the search engine query optimization method provided by the embodiment of the present application. Figure 5 Steps 201 to 209 in the figure illustrate the process of the terminal and the server collaboratively implementing the search engine query optimization method provided in the embodiment of the present application.

[0115] In step 201, the server obtains historical query records of the search engine, where the historical query records include multiple query statements.

[0116] In step 202, the server generates a syntax tree based on the terms included in the query statement.

[0117] In step 203, the server adjusts the execution order of the words in the syntax tree based on the amount of information included in the inverted list corresponding to each word in the syntax tree.

[0118] In step 204, the server performs a search based on the syntax tree after the execution order is adjusted, and determines the execution order of each word in the syntax tree according to the search results, and the execution order minimizes the overall search cost.

[0119] In step 205, the terminal sends a search request carrying a query statement to the server.

[0120] In step 206 , the server determines a syntax tree that matches the query statement in the search request.

[0121] In step 207, the server searches based on the execution order of each word in the matching syntax tree to obtain at least one search result.

[0122] In step 208, the server sorts the at least one search result in descending order of relevance to the query statement.

[0123] In step 209, the server sends at least one search result sorted in descending order to the terminal.

[0124] It should be noted that the above steps have been described in detail in the previous text and will not be repeated here.

[0125] Below, a specific example of the search engine query optimization method provided by the embodiment of the present application will be described.

[0126] First, let's introduce the process of traversing and finding the intersection of inverted lists in a search engine. For example, if we need to find the intersection of word 1 and word 2, the search results for word 1 are stored in inverted list 1, and the search results for word 2 are stored in inverted list 2. Generally, we first take out a document from inverted list 1 and check whether the document is in inverted list 2. Because inverted list 2 is ordered, we can use a fast search method to quickly locate the same document in inverted list 2 based on the documents in inverted list 1, thus avoiding traversing the entire inverted list 2. Repeat the above process for each document in inverted list 1 to determine the documents in inverted list 2 that are the same as in inverted list 1. The length of this process depends on the number of documents in inverted list 1. For example, if there is only one document in inverted list 1, the search can be ended after checking whether this document is in inverted list 2. If there are many documents in inverted list 1, it will bring a lot of extra overhead.

[0127] To reduce search overhead, the present application proposes the following two solutions: Solution 1: Predict the number of documents included in the inverted list corresponding to each term, and adjust the execution order of multiple terms connected to operators in the syntax tree, that is, prioritize the term with the smallest number of documents included in the corresponding inverted list. Solution 2: Optimize by counting the number of documents actually traversed.

[0128] The following describes the above two solutions in detail.

[0129] Query clause Data scale a or b Not exceeding 1 million (a: 500,000, b: 500,000) c 700,000 d 800

[0130] Table 1 Data size corresponding to query clauses

[0131] In solution 1, the server generates an inverted list offline based on the search engine's historical query records and determines the number of documents in the inverted list corresponding to each word. Figure 3 As shown in Table 1, assuming that the query expression corresponding to the syntax tree is (a or b) and c and d, the estimated data size corresponding to the object (query clause) of each query can be obtained.

[0132] According to the strategy of executing small query clauses first, the execution order of the words in the syntax tree is Figure 4 If the query expression corresponding to the syntax tree is adjusted, it will be adjusted to d and c and (a or b).

[0133] Among them, the embodiments of the present application provide different operators for implementing different functions. For example, the term query operator term is used to determine the number of documents included in the inverted list. When the number of documents corresponding to term a is 500,000, term a = 500,000. The union operator and is used to take the intersection of the documents corresponding to the two terms it connects. The or operator or is used to take the union of the documents corresponding to the two terms it connects. The not operator not is used to subtract the number of documents corresponding to the term it connects from the existing number of documents.

[0134] Considering that the two words connected by the OR operator have an intersection, the number of documents corresponding to the whole consisting of the OR operator and the two words it connects is difficult to determine and can only be roughly estimated. In addition, when using a filter to search, because the filter is composed of multiple filter conditions, and the filter conditions are not necessarily associated with a certain inverted list (for example, when range filtering is required based on a field in the document), the embodiment of the present application proposes Solution 2.

[0135] In the second solution, among the multiple filter conditions corresponding to the query statement, the number of documents traversed by each filter condition and the number of documents remaining after filtering are counted, and the multiple filter conditions are sorted in ascending order of the number of documents remaining after filtering to obtain a filter composed of multiple filter conditions and corresponding to the query statement. Alternatively, the consumption when executing the filter / syntax tree is counted, and the filter / syntax tree with the smallest consumption is used as the filter / syntax tree corresponding to the query statement. For example, the number of documents traversed in a single execution of the filter / syntax tree is counted, or the average time consumed when executing the filter / syntax tree over a period of time is counted. Among them, when counting the number of documents, it is only necessary to encode each operator or each filter condition, and the counted number can be integrated in the index kernel cache to complete the statistics.

[0136] As can be seen, the query operator in Solution 1 is used to traverse the length of the inverted list, while the filter in Solution 2 is used to count the number of documents remaining after filtering. The optimization method provided by Solution 2 places the filter conditions with a small number of remaining documents first, thereby removing unnecessary documents as early as possible, achieving the same effect as the optimization in Solution 1. Complex query optimization strategies often work well in certain targeted situations, but lack universality. Compared with complex query optimization strategies, Solution 2 is more efficient and easier to implement.

[0137] In some possible examples, Option 1 and Option 2 can also be combined, that is, according to the method provided by Option 1, the execution order of some words in the syntax tree is adjusted, and then the execution order of other words in the syntax tree is adjusted to obtain multiple combinations; the multiple combinations are searched separately to obtain corresponding multiple search results and the search volume corresponding to each search result; the expression of the combination corresponding to the minimum search volume is used as the query expression corresponding to the syntax tree. For example, the query expression corresponding to the syntax tree is a and b and c. According to Option 1, it is determined that the execution order of c is first, so the query expression is adjusted to cand a and b. The execution order of b and c is adjusted to obtain combination 1 "c and a and b" and combination 2 "c and b and a". Combination 1 and combination 2 are searched, and it is determined that the search volume corresponding to combination 2 is smaller, so the query expression corresponding to the syntax tree is determined to be combination 2 "c and b and a".

[0138] As can be seen, the embodiments of this application provide two simple and effective query optimization methods that can improve the performance of index query and filtering. In actual business, there are often filters that retain 99% of the data after filtering (such as blacklist filters, city level restriction filters, etc.); placing such filters afterward can effectively reduce the judgment time, thereby improving search recall performance.

[0139] Continuing to use the electronic device provided in the embodiment of the present application as an example to illustrate the exemplary structure, see Figure 6 , Figure 6 is a schematic diagram of the structure of the server provided in the embodiment of the present application, Figure 6 The server 200 shown includes: at least one processor 410, a memory 440, and at least one network interface 420. The various components in the server 200 are coupled together via a bus system 430. It is understood that the bus system 430 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 430 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 430 is not described in detail. Figure 2A Various buses are labeled as bus system 430 .

[0140] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0141] The memory 440 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 440 may optionally include one or more storage devices that are physically remote from the processor 410.

[0142] The memory 440 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 440 described in the embodiments of the present application is intended to include any suitable type of memory.

[0143] In some embodiments, the memory 440 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0144] Operating system 441, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, which are used to implement various basic services and process hardware-based tasks;

[0145] A network communication module 442 for reaching other computing devices via one or more (wired or wireless) network interfaces 420 , exemplary network interfaces 420 including Bluetooth, WiFi, and USB;

[0146] In some embodiments, the query optimization device of the search engine provided in the embodiments of the present application can be implemented in software. Figure 6 The search engine query optimization device 453 stored in the memory 440 is shown. This device can be software in the form of a program or plug-in, and includes the following software modules: a generation module 4531, an adjustment module 4532, a determination module 4533, and a search module 4534. These modules are logical and can be arbitrarily combined or further separated according to the functions they implement. The functions of each module will be described below.

[0147] A generation module 4531 is used to obtain historical query records of a search engine, the historical query records including multiple query statements, and to generate a syntax tree based on the words included in the query statements; an adjustment module 4532 is used to adjust the execution order of the words in the syntax tree based on the amount of information included in the inverted list corresponding to each word in the syntax tree; a determination module 4533 is used to search based on the syntax tree after the execution order is adjusted, and to determine the execution order of each word in the syntax tree based on the search results, and the execution order minimizes the overall search cost; wherein, the syntax tree is used for the search engine to search when it receives a query statement.

[0148] In some embodiments, the generation module 4531 is further used to: perform word segmentation on the query statement to obtain at least one word element; perform assimilation on at least one word element to obtain at least one term; perform grammatical analysis on at least one term to obtain a logical operator that represents the logical relationship between each term; and generate a syntax tree based on at least one term and the logical operator.

[0149] In some embodiments, when the syntax tree includes at least two adjacent AND operators, the adjustment module 4532 is further used to: determine the amount of information corresponding to the words connected by at least two adjacent AND operators in the syntax tree; and adjust the execution order of the word with the least amount of corresponding information to the first place.

[0150] In some embodiments, when the syntax tree includes at least one non-operator, the adjustment module 4532 is further used to: determine the number of corresponding information of words connected by at least one non-operator in the syntax tree; adjust the execution order of the words connected by at least one non-operator to after the words connected by other logical operators, and the other logical operators include the AND operator and the OR operator; wherein, when the syntax tree includes at least two non-operators, the execution order of the words connected by at least two non-operators is consistent with the order of the number of information corresponding to the corresponding words.

[0151] In some embodiments, the determination module 4533 is also used to: adjust the execution order of other words in the syntax tree to obtain multiple combinations consisting of all the words in the syntax tree, where the other words are the words in the syntax tree except for the words with the least amount of corresponding information; search the multiple combinations separately to obtain corresponding multiple search results, and the search volume corresponding to each search result; determine the execution order of each word in the combination corresponding to the minimum search volume.

[0152] In some embodiments, the determination module 4533 is also used to: adjust the execution order of other words in the syntax tree to obtain multiple combinations consisting of all the words in the syntax tree, where the other words are the words in the syntax tree except for the words with the least amount of corresponding information; search the multiple combinations separately to obtain corresponding multiple search results, and the search time corresponding to each search result; determine the execution order of each word in the combination corresponding to the minimum search time.

[0153] In some embodiments, the query optimization device of the search engine also includes a search module 4534, which is used to: receive a search request carrying a query statement sent by the search engine; determine a syntax tree that matches the query statement in the search request; search based on the execution order of each word in the matching syntax tree to obtain at least one search result; sort at least one search result in descending order of relevance to the query statement, and return it to the search engine.

[0154] In some embodiments, the search module 4534 is also used to: obtain multiple filter conditions corresponding to the query statement in the historical query record; reorganize the multiple filter conditions, and construct multiple filters based on the reorganized multiple filter conditions; search the query statement based on the multiple filters respectively to obtain the search time corresponding to each filter; use the filter with the shortest search time as the filter of the query statement, and the filter is used to search when a search request carrying the query statement is received.

[0155] In some embodiments, the search module 4534 is also used to: obtain multiple filter conditions corresponding to the query statement in the historical query record; determine the amount of filtered information corresponding to each filter condition; sort the multiple filter conditions in ascending order according to the amount of filtered information, and construct a filter based on the multiple filter conditions sorted in ascending order, and the filter is used to search when a search request carrying a query statement is received.

[0156] The embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the method provided by the embodiment of the present application, for example, Figure 2A A query optimization method for a search engine is shown.

[0157] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0158] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0159] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0160] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0161] To sum up, the embodiment of the present application generates a syntax tree based on the query statements in the historical query records, and adjusts the execution order of the words in the syntax tree based on the amount of information included in the inverted list corresponding to each word in the syntax tree. Because the execution order of the words in the syntax tree is different, the corresponding search volume is also different. Therefore, adjusting the execution order of the words in the syntax tree can reduce the search volume; thereafter, a search is performed based on the syntax tree after the execution order is adjusted, and the execution order of each word in the syntax tree is adjusted again according to the search results. In this way, the execution order of each word that minimizes the overall search cost can be determined according to the feedback of the actual search results. When a search is performed based on the syntax tree obtained by the final adjustment, the search consumption is effectively reduced and the search efficiency is improved.

[0162] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A search engine query optimization method, characterized in that: The method comprises: Obtaining historical query records of a search engine, the historical query records including multiple query statements, and generating a syntax tree based on the terms included in the query statements; When the syntax tree includes at least one non-operator, in the syntax tree, determining the number of information corresponding to words connected by the at least one non-operator; adjusting the execution order of the words connected by the at least one non-operator to be after words connected by other logical operators, wherein the other logical operators include an AND operator and an OR operator; wherein, when the syntax tree includes at least two non-operators, the execution order of the words connected by the at least two non-operators is consistent with the order of the number of information corresponding to the corresponding words; When the syntax tree includes at least two adjacent AND operators, in the syntax tree, determining the amount of information corresponding to the words connected by the at least two adjacent AND operators; adjusting the execution order of the word with the least amount of corresponding information to be first; Searching based on the syntax tree after adjusting the execution order, and determining the execution order of each word in the syntax tree according to the search results, wherein the execution order minimizes the overall search cost; The syntax tree is used by the search engine to search when receiving the query statement.

2. The method according to claim 1, characterized in that The generating a grammar tree based on the words included in the query statement includes: Performing word segmentation on the query statement to obtain at least one word element; Performing assimilation processing on the at least one word to obtain at least one word; Performing grammatical analysis on the at least one word to obtain a logical operator representing a logical relationship between the words; The syntax tree is generated based on the at least one term and the logical operator.

3. The method according to claim 1, characterized in that The searching based on the syntax tree after the execution order is adjusted, and determining the execution order of each word in the syntax tree according to the search results, includes: Adjusting the execution order of other words in the syntax tree to obtain multiple combinations consisting of all words in the syntax tree, wherein the other words are words in the syntax tree other than the word with the least amount of corresponding information; Searching the multiple combinations respectively to obtain corresponding multiple search results and a search volume corresponding to each search result; Determine the order in which each term in the combination that results in the smallest search volume should be executed.

4. The method according to claim 1, wherein The searching based on the syntax tree after the execution order is adjusted, and determining the execution order of each word in the syntax tree according to the search results, includes: Adjusting the execution order of other words in the syntax tree to obtain multiple combinations consisting of all words in the syntax tree, wherein the other words are words in the syntax tree other than the word with the least amount of corresponding information; Searching the multiple combinations respectively to obtain corresponding multiple search results and the search time corresponding to each search result; Determine the execution order of each word in the combination corresponding to the minimum search time.

5. The method according to claim 1, characterized in that After searching based on the syntax tree after adjusting the execution order and determining the execution order of each word in the syntax tree according to the search results, the method further includes: Receiving a search request carrying a query statement sent by the search engine; Determining a syntax tree that matches the query statement in the search request; Searching based on the execution order of each word in the matching syntax tree to obtain at least one search result; The at least one search result is sorted in descending order of relevance to the query statement and returned to the search engine.

6. The method according to claim 1, characterized in that The method further comprises: Obtaining multiple filter conditions corresponding to the query statements in the historical query records; reorganizing the plurality of filtering conditions, and constructing a plurality of filters based on the reorganized plurality of filtering conditions; Searching the query statement based on the multiple filters respectively to obtain the search time corresponding to each filter; The filter with the shortest search time is used as the filter of the query statement, and the filter is used to perform a search when a search request carrying the query statement is received.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining multiple filter conditions corresponding to the query statements in the historical query records; Determine the amount of filtered information corresponding to each filtering condition; The multiple filter conditions are sorted in ascending order according to the amount of filtered information, and a filter is constructed based on the multiple filter conditions sorted in ascending order. The filter is used to perform a search when a search request carrying the query statement is received.

8. A search engine query optimization device, characterized in that: include: A generation module is used to obtain historical query records of a search engine, wherein the historical query records include multiple query statements, and generate a syntax tree based on the terms included in the query statements; An adjustment module is configured to, when the syntax tree includes at least one non-operator, determine, in the syntax tree, the amount of information corresponding to words connected by the at least one non-operator; adjust the execution order of the words connected by the at least one non-operator to after words connected by other logical operators, wherein the other logical operators include an AND operator and an OR operator; wherein, when the syntax tree includes at least two non-operators, the execution order of the words connected by the at least two non-operators is consistent with the order of the amount of information corresponding to the corresponding words; when the syntax tree includes at least two adjacent AND operators, determine, in the syntax tree, the amount of information corresponding to the words connected by the at least two adjacent AND operators; and adjust the execution order of the word with the least amount of corresponding information to be first; A determination module is used to search based on the syntax tree after adjusting the execution order, and determine the execution order of each word in the syntax tree according to the search results, and the execution order minimizes the overall search cost; wherein, the syntax tree is used for the search engine to search when receiving the query statement.

9. An electronic device, characterized in that: include: a memory for storing computer-executable instructions or computer programs; The processor is configured to implement the search engine query optimization method according to any one of claims 1 to 7 when executing the computer executable instructions or computer program stored in the memory.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a processor, the search engine query optimization method according to any one of claims 1 to 7 is implemented.

Images