Method for parsing a search request

Abstract

The embodiment of the present application discloses a search request analysis method. The analysis method comprises: using multiple boundary detection methods to identify the search request of a user to obtain multiple entity sets corresponding to the search request; expanding the recall of the multiple entity sets to obtain multiple candidate entity sets corresponding to the multiple entity sets; and screening a target candidate entity combination matching the search request from the multiple candidate entity sets based on a preset screening condition. Thus, the present application can accurately identify the geographic location in the user search, thereby greatly improving the user experience, which can be applied to C-end search and assistant dialogue scenarios.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to a search request analysis method and analysis system. BACKGROUND

[0002] Natural language understanding (NLU) is one of the core problems of artificial intelligence, and is also the core of text and voice search. For example, the house search scene mainly includes C-end (for example, consumer, personal terminal user uses the client) search and voice house search. However, the difficulties of NLU implementation in house search are as follows:

[0003] 1. The search text input by the user is incomplete, and is generally short, such as more words, fewer words, wrong words, wrong words, alternative names, reverse order, etc., which brings great difficulty to the recognition of the user's search intent. For example, the user searches for "Anhui", which may refer to "Anhui Beili" or "Anhui Li".

[0004] 2. In the field of real estate, geographical location ambiguity is particularly serious: example 1, for example, the user searches for "Chaoyang", which may refer to Beijing Chaoyang District or Liaoning Chaoyang City; example 2, the user searches for "Wanda Plaza nearby" in Beijing, which refers to the Wanda Plaza in Chaoyang District or Tongzhou District; example 3, the user searches for "other houses in Fengtai" in Beijing, which has two ways of understanding: Fengtai / other houses, or Fengtai other / houses ("Fengtai other" is the name of a community in Fengtai District).

[0005] In the current house search scene, a single text matching method is usually used to identify the geographical location in the user's search, and the accuracy of the identification result is low, which brings poor user experience. SUMMARY

[0006] The purpose of the present application is to provide a search request analysis method and search system, which can accurately identify the geographical location in the user's search, thereby greatly improving the user's experience, and which can be applied to C-end search and assistant dialogue scene.

[0007] In order to achieve the above purpose, the first aspect of the present application provides a search request analysis method, which comprises: using multiple boundary detection methods to identify the user's search request to obtain multiple entity sets corresponding to the search request; expanding the recall of the multiple entity sets to obtain multiple candidate entity sets corresponding to the multiple entity sets; and screening a target candidate entity combination matching the search request from the multiple candidate entity sets based on a preset screening condition.

[0008] In at least one embodiment, the plurality of boundary detection methods include at least two of a preset dictionary-based detection method, a preset algorithm-based detection method, a preset rule-based detection method, and a preset template-based detection method.

[0009] In at least one embodiment, the expanding recall on the plurality of entity sets includes: recalling each entity in each entity set in the plurality of entity sets from a text dimension and / or a user preference dimension to obtain a candidate entity group corresponding to the each entity; and combining, for each entity set, the candidate entity group corresponding to the each entity in the each entity set to form a candidate entity set corresponding to the each entity set.

[0010] In at least one embodiment, the recalling each entity in each entity set in the plurality of entity sets from a text dimension includes: recalling a binary sequence of each entity.

[0011] In at least one embodiment, the recalling each entity in each entity set in the plurality of entity sets from a text dimension includes: recalling a binary sequence of each entity; and recalling based on at least one of a pinyin, a text, an intent, a heat, and a pairing rate of each entity.

[0012] In at least one embodiment, the screening, based on a preset screening condition, the target candidate entity combination matching the search request from the plurality of candidate entity sets includes: determining a matching degree between each candidate entity in each candidate entity set in the plurality of candidate entity sets and a corresponding entity in a corresponding entity set in the plurality of entity sets; and screening, according to the matching degree between the each candidate entity and the corresponding entity, the target candidate entity combination from the plurality of candidate entity sets.

[0013] In at least one embodiment, the determining a matching degree between each candidate entity in each candidate entity set in the plurality of candidate entity sets and a corresponding entity in a corresponding entity set in the plurality of entity sets includes: determining an edit distance, a jaccard distance, a proportion of a common substring in a length of each candidate entity, a pairing rate, and a proportion of a corresponding entity as a prefix or a suffix in each candidate entity in a length of each candidate entity between each candidate entity in each candidate entity set and a corresponding entity in a corresponding entity set; and performing a weighted summation on the edit distance, the jaccard distance, the proportion of the common substring in the length of each candidate entity, the pairing rate, and the proportion of the corresponding entity as the prefix or the suffix in the length of each candidate entity to determine the matching degree between the each candidate entity and the corresponding entity.

[0014] In at least one embodiment, the filtering the target candidate entity combination from the plurality of candidate entity sets comprises: determining a candidate entity matching each entity in each entity set according to a matching degree between the each candidate entity and the corresponding entity; determining the first candidate entity or the second candidate entity as a first target candidate entity in a case that a first entity in a first entity set is identical to a second entity in a second entity set, and a matching degree between the first candidate entity and the first entity and / or a matching degree between the second candidate entity and the second entity is greater than a first threshold, wherein the first candidate entity is a candidate entity matching the first entity, and the second candidate entity is a candidate entity matching the second entity; performing a weighted summation on a matching degree between a third candidate entity and a corresponding third entity in each candidate entity set to obtain a fusion matching degree about the each candidate entity set, wherein the third candidate entity is a candidate entity matching the third entity, and the third entity is different from the first entity and the second entity; determining the third candidate entity as a second target candidate entity in a case that the fusion matching degree about a specific candidate entity set in the plurality of candidate entity sets is highest and greater than a second threshold; and combining the first target candidate entity and the second target candidate entity as the target candidate entity combination.

[0015] In at least one embodiment, the performing the weighted summation on the matching degree between the third candidate entity and the corresponding third entity comprises: determining a length proportion of the third entity among entities in the search request except the first entity as a weight of the third candidate entity; and performing the weighted summation according to the matching degree between the third candidate entity and the third entity and the weight of the third candidate entity.

[0016] In at least one embodiment, the determining the candidate entity matching each entity in the each entity set comprises: for each entity in the each entity set, filtering a candidate entity with a highest matching degree from a corresponding candidate entity set, and determining the filtered candidate entity as the candidate entity matching the each entity.

[0017] In at least one embodiment, the parsing method further comprises: in a case that the target candidate entity combination is not filtered from the plurality of entity sets based on the preset filtering condition, filtering the target candidate entity combination from the plurality of candidate entity sets by using a conditional random field model.

[0018] By the technical scheme, the search request of the user is recognized by using multiple boundary detection methods to obtain multiple entity sets corresponding to the search request, recall of the multiple entity sets is expanded to obtain multiple candidate entity sets corresponding to the multiple entity sets, and a target candidate entity combination matching the search request is screened from the multiple candidate entity sets based on a preset screening condition. Thus, the geographic location in the search of the user can be accurately recognized, and the use experience of the user can be greatly improved, and the application can be applied to C-end search and assistant dialogue scenarios.

[0019] The second aspect of the application provides a search request analysis system, which comprises: an identification device for recognizing a search request of a user by using multiple boundary detection methods to obtain multiple entity sets corresponding to the search request; a recall device for expanding recall of the multiple entity sets to obtain multiple candidate entity sets corresponding to the multiple entity sets; and a first screening device for screening a target candidate entity combination matching the search request from the multiple candidate entity sets based on a preset screening condition.

[0020] In at least one embodiment, the multiple boundary detection methods comprise at least two of the following: a preset dictionary-based detection method, a preset algorithm-based detection method, a preset rule-based detection method, and a preset template-based detection method.

[0021] In at least one embodiment, the recall device comprises: a recall module for recalling each entity in each entity set in the multiple entity sets from a text dimension and / or a user preference dimension to obtain a candidate entity group corresponding to each entity; and a combination module for combining the candidate entity groups corresponding to each entity in each entity set to form a candidate entity set corresponding to each entity set.

[0022] In at least one embodiment, the recall module is configured to recall each entity in each entity set in the multiple entity sets from a text dimension, comprising: recalling a binary sequence of each entity.

[0023] In at least one embodiment, the recall module is configured to recall each entity in each entity set in the multiple entity sets from a text dimension, comprising: recalling a binary sequence of each entity; and recalling based on at least one of pinyin, text, intent, heat, and pairing rate of each entity.

[0024] In at least one embodiment, the first screening device comprises: a determining module configured to determine a matching degree between each candidate entity in each candidate entity set in the plurality of candidate entity sets and a corresponding entity in a corresponding entity set in the plurality of entity sets; and a screening module configured to screen the target candidate entity combination from the plurality of candidate entity sets according to the matching degree between the each candidate entity and the corresponding entity.

[0025] In at least one embodiment, the determining module comprises: a parameter determining unit configured to determine an edit distance, a jaccard distance, a proportion of a common substring in a length of the each candidate entity, a matching rate, and a proportion of the corresponding entity as a prefix or a suffix in a length of the each candidate entity between the each candidate entity in the each candidate entity set and the corresponding entity in the corresponding entity set; and a summing unit configured to perform a weighted sum on the edit distance, the jaccard distance, the proportion of the common substring in the length of the each candidate entity, the matching rate, and the proportion of the corresponding entity as the prefix or the suffix in the length of the each candidate entity to determine the matching degree between the each candidate entity and the corresponding entity.

[0026] In at least one embodiment, the screening module comprises: a matching candidate determining unit configured to determine a candidate entity matching each entity in the each entity set according to the matching degree between the each candidate entity and the corresponding entity; a first target candidate determining unit configured to determine the first candidate entity or the second candidate entity as a first target candidate entity in a case that a first entity in a first entity set and a second entity in a second entity set are the same, and a matching degree between the first candidate entity and the first entity and / or a matching degree between the second candidate entity and the second entity is greater than a first threshold value, wherein the first candidate entity is a candidate entity matching the first entity, and the second candidate entity is a candidate entity matching the second entity; a matching degree determining unit configured to perform a weighted sum on a matching degree between a third candidate entity and a corresponding third entity in each candidate entity group to obtain a fusion matching degree about the each candidate entity set, wherein the third candidate entity is a candidate entity matching the third entity, and the third entity is different from the first entity and the second entity; a second target candidate determining unit configured to determine the third candidate entity as a second target candidate entity in a case that the fusion matching degree about a specific candidate entity set in the plurality of candidate entity sets is the highest and greater than a second threshold value; and a combination unit configured to combine the first target candidate entity and the second target candidate entity as the target candidate entity combination.

[0027] In at least one embodiment, the matching degree determining unit comprises: a weight determiner configured to determine a length proportion of the third entity in entities in the search request other than the first entity as a weight of the third candidate entity; and a matching degree determiner configured to perform weighted summation according to matching degrees between the third candidate entity and the third entity and the weight of the third candidate entity.

[0028] In at least one embodiment, the matching candidate determining unit configured to determine candidate entities matching each entity in the each entity set comprises: a filter configured to filter a candidate entity with the highest matching degree from the corresponding candidate entity group for each entity in the each entity set, and determine the filtered candidate entity as a candidate entity matching the each entity.

[0029] In at least one embodiment, the resolution system further comprises: a second filtering device configured to filter the target candidate entity combination from the plurality of candidate entity sets by using a conditional random field model in a case where the target candidate entity combination is not filtered from the plurality of entity sets based on the preset filtering condition.

[0030] The specific details and benefits of the resolution system of the search request provided by the present application can be referred to the description of the resolution method of the search request above, and will not be repeated here.

[0031] The third aspect of the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the resolution method of the search request.

[0032] The fourth aspect of the present application further provides an electronic device, wherein the electronic device comprises: a processor; a memory configured to store the computer program; and the processor is configured to read the computer program from the memory and execute the computer program to implement the resolution method of the search request.

[0033] The fifth aspect of the present application further provides a computer program product, wherein the computer program product comprises a computer program, and the computer program is executed by a processor to implement the resolution method of the search request.

[0034] Other features and advantages of the present application will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0035] The accompanying drawings are included to provide a further understanding of embodiments of the application, and constitute a part of the specification, and are used to explain the embodiments of the application together with the following specific embodiments, but do not constitute a limitation on the embodiments of the application. In the drawings:

[0036] Figure 1 This is a flowchart of a search request parsing method provided in an embodiment of the present invention;

[0037] Figure 2 This is a flowchart of an embodiment of the present invention, which filters a combination of target candidate entities that match the search request from a plurality of candidate entity sets based on preset filtering conditions;

[0038] Figure 3 This is a flowchart of filtering the target candidate entity combination from the plurality of candidate entity sets according to an embodiment of the present invention;

[0039] Figure 4 This is a flowchart illustrating the process of parsing a search request according to an embodiment of the present invention; and

[0040] Figure 5 This is a structural diagram of a search request parsing system provided in an embodiment of the present invention. Detailed Implementation

[0041] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0042] Figure 1 This is a flowchart of a search request parsing method provided in an embodiment of the present invention. For example... Figure 1 As shown, the parsing method may include the following steps S101-S103.

[0043] Step S101: Multiple boundary detection methods are used to identify the user's search request in order to obtain multiple entity sets corresponding to the search request.

[0044] The various boundary detection methods may include at least two of the following: a detection method based on a preset dictionary, a detection method based on a preset algorithm, a detection method based on preset rules, and a detection method based on a preset template.

[0045] For detection methods based on a pre-defined dictionary, a geographic location dictionary (including search terms and the frequency of user searches for those terms) can be constructed based on user search requests (e.g., queries) and a property dictionary. This geographic location dictionary can be built according to city dimensions and updated weekly. Detecting "Shanghai Pudong Dongmingyuan" using a pre-defined dictionary-based detection method yields the entity: Shanghai / Pudong, as shown in Table 1. In other words, identifying "Shanghai Pudong Dongmingyuan" using a pre-defined dictionary-based detection method yields an entity set {"Shanghai", "Pudong"} (corresponding to the first row of Table 1). The property dictionary mentioned in this disclosure refers to a system that records property information data, including various information such as building details and unit information.

[0046] For detection methods based on preset algorithms, "Shanghai Pudong Dongmingyuan" can be identified using named entity recognition (NER) algorithms (consider using BiLSTM+CRF / CNN+BiLSTM+CRF / Lattice LSTM). For example, the entity result identified by BiLSTM+CRF is Pudong / Dongmingyuan, as shown in Table 1. In other words, by identifying "Shanghai Pudong Dongmingyuan" using detection methods based on preset algorithms, an entity set {"Pudong", "Dongmingyuan"} can be obtained (corresponding to the second row of Table 1).

[0047] For detection methods based on preset rules, when the length of the search request / search content is less than 6 characters, the entire search request / search content can be considered as a single entity. Therefore, the result of identifying "Shanghai Pudong Dongmingyuan" using preset rules can be Shanghai / Pudong Dongmingyuan, as shown in Table 1. In other words, identifying "Shanghai Pudong Dongmingyuan" using the detection method based on preset rules yields an entity set {"Shanghai", "Pudong Dongmingyuan"} (corresponding to the third row of Table 1).

[0048] For detection methods based on preset templates, data is analyzed to collect language expression templates, and corresponding entity results are extracted using regular expressions. For example, identifying "Shanghai Pudong Dongmingyuan" using a preset template could result in Shanghai / Pudong / Dongmingyuan (not shown).

[0049] Table 1. Identification results for the three detection methods.

[0050] Method Entity Entity Detection method based on preset dictionary Shanghai Pudong Detection method based on preset algorithm Pudong Dongming Garden Detection method based on preset rules Shanghai Pudong Dongming Garden

[0051] The various boundary detection methods performed in step S101 only identify multiple meaningful entities from the user's search, but cannot confirm the specific meaning of each entity.

[0052] Step S102: Expand the recall of the plurality of entity sets to obtain a plurality of candidate entity sets corresponding to the plurality of entity sets.

[0053] For step S102, expanding the recall of the plurality of entity sets may include: recalling each entity in each entity set of the plurality of entity sets from the text dimension and / or the user preference dimension to obtain a candidate entity group corresponding to each entity; and for each entity set, combining the candidate entity groups corresponding to each entity in each entity set to form a candidate entity set corresponding to each entity set.

[0054] The step of recalling each entity in each entity set of the plurality of entity sets from a textual dimension includes: recalling the binary sequence of each entity. In at least one embodiment, the step of recalling each entity in each entity set of the plurality of entity sets from a textual dimension includes: recalling the binary sequence of each entity; and recalling based on at least one of the following: pinyin, text, intent, popularity, and matching rate of each entity.

[0055] Specifically, for each entity set (i.e., each row of Table 1), search recall is performed based on the bigram of each entity. For example, recall is performed based on the bigram of the entity "Shanghai" to obtain multiple candidate entities such as "Shanghai City", "Shanghai Tower", and "Shanghai TV Station" (i.e., obtaining a candidate entity group ["Shanghai City", "Shanghai Tower", "Shanghai TV Station"] for the entity "Shanghai"); or recall is performed based on the bigram of the entity "Pudong" to obtain multiple candidate entities such as "Pudong District", "Pudong Road", and "Pudong Building" (i.e., obtaining a candidate entity group ["Pudong District", "Pudong Road", "Pudong Building"] for the entity "Pudong"), as shown in Table 2. Based on obtaining the candidate entity group for each entity, for each entity set, the candidate entity groups corresponding to each entity in each entity set are combined to form a candidate entity set corresponding to each entity set. For example, for the entity set {"Shanghai", "Pudong"}, the candidate entity group ["Shanghai City", "Shanghai Tower", "Shanghai TV Station"] corresponding to the entity "Shanghai" is combined with the candidate entity group ["Pudong District", "Pudong Road", "Pudong Building"] corresponding to the entity set {"Shanghai", "Pudong"} to form {["Shanghai City", "Shanghai Tower", "Shanghai TV Station"], ["Pudong District", "Pudong Road", "Pudong Building"]} corresponding to the entity set {"Shanghai", "Pudong"}.

[0056] Table 2 shows the recall results corresponding to the identification results of the three detection methods.

[0057]

[0058] In a preferred embodiment, in addition to the binary sequence, recall can also be performed simultaneously based on at least one of the entity's pinyin, text, intent, popularity, and matching rate to obtain more candidate entities. Specifically, when recalling candidate entities based on the entity's pinyin, text, intent, popularity, and matching rate, candidate entities with the same pinyin as the entity, candidate entities including the entity's text, candidate entities whose intent belongs to the same intent system as the entity, candidate entities retrieved based on a preset popularity, and candidate entities retrieved based on a preset matching rate (the correspondence between candidate entities and their pinyin, text, intent, popularity, and / or matching rate in the database can be stored in advance), etc.

[0059] Regarding user preferences, we can analyze each user's clicks on property listings within a week, including neighborhood names, administrative districts, business districts, nearby subway station names, nearby landmarks, and communities, and create user profiles corresponding to these user identifiers. Based on this, we can retrieve candidate entities related to the user's preferences according to the user identifier of the user who initiated the search request.

[0060] Of course, in different embodiments, any combination of text dimension and user preference dimension can be used to recall more candidate entities.

[0061] Step S103: Based on preset filtering conditions, select a combination of target candidate entities that match the search request from the multiple candidate entity sets.

[0062] For step S103, as Figure 2 As shown, the step of filtering target candidate entity combinations that match the search request from the multiple candidate entity sets based on preset filtering conditions may include the following steps S201-S202.

[0063] Step S201: Determine the matching degree between each candidate entity in each candidate entity set of the plurality of candidate entity sets and the corresponding entity in the corresponding entity set of the plurality of entity sets.

[0064] For step S201, determining the matching degree between each candidate entity in each candidate entity set of the plurality of candidate entity sets and the corresponding entity in the corresponding entity set of the plurality of entity sets may include: determining the edit distance, the proportion of the common substring of the Jaccard distance to the length of each candidate entity, the pairing rate, and the proportion of the length of the corresponding entity as a prefix or suffix in each candidate entity; and performing a weighted summation of the edit distance, the Jaccard distance, the proportion of the common substring to the length of each candidate entity, the pairing rate, and the proportion of the length of the corresponding entity as a prefix or suffix to the length of each candidate entity to determine the matching degree between each candidate entity and the corresponding entity.

[0065] Specifically, for each candidate entity set (i.e., the candidate entities in each row of Table 2), the edit distance, Jaccard distance, common substring proportion of the length of the corresponding candidate entity, pairing rate, and the proportion of the length of the corresponding entity used as a prefix or suffix in each candidate entity are calculated. For example, taking the candidate entity set shown in the first row of Table 2 as an example, the edit distance, Jaccard distance, common substring proportion of the length of each candidate entity, pairing rate, and the proportion of the length of the corresponding candidate entity used as a prefix or suffix "Shanghai" are calculated between "Shanghai", "Shanghai Tower", "Shanghai TV Station" and "Shanghai"; and the edit distance, Jaccard distance, common substring proportion of the length of each candidate entity, pairing rate, and the proportion of the length of the corresponding candidate entity used as a prefix or suffix "Shanghai" are calculated between "Pudong District", "Pudong Road", "Pudong Building" and "Pudong"; and the proportion of the length of the corresponding candidate entity used as a prefix or suffix "Pudong" is calculated between "Pudong District", "Pudong Road", "Pudong Building" and "Pudong". Then, the edit distance between each candidate entity and the corresponding entity, the Jaccard distance, the proportion of the common substring to the length of each candidate entity, the matching rate, and the proportion of "Pudong" as a prefix or suffix in each candidate entity ("Pudong District", "Pudong Road", "Pudong Building") to the length of the corresponding candidate entity are weighted and summed (the weights of each parameter can be reasonably set as needed). Thus, the matching degree (i.e., confidence) between each candidate entity and the corresponding entity can be determined, as shown in Table 3.

[0066] Table 3 Confidence levels of each candidate entity

[0067]

[0068] Step S202: Based on the matching degree between each candidate entity and the corresponding entity, filter the target candidate entity combination from the multiple candidate entity sets.

[0069] For step S202, as follows Figure 3 As shown, the process of filtering the target candidate entity combination from the plurality of candidate entity sets may include the following steps S301-S304.

[0070] Step S301: Based on the matching degree between each candidate entity and the corresponding entity, determine the candidate entity that matches each entity in each entity set.

[0071] For step S301, determining the candidate entity that matches each entity in each entity set may include: for each entity in each entity set, filtering the candidate entity with the highest matching degree from the corresponding candidate entity group, and determining the filtered candidate entity as the candidate entity that matches each entity.

[0072] Specifically, after obtaining the confidence level of each candidate entity in each candidate entity group (e.g., the candidate entity group corresponding to the entity "Shanghai" in Table 3 ["Shanghai Municipality", "Shanghai Mansion", "Shanghai TV Station"]) in each candidate entity set shown in Table 3, the candidate entity with the highest confidence level is selected from each candidate entity group. For example, the candidate entity "Shanghai Municipality" with a confidence level of 1 is selected from the candidate entity group corresponding to the entity "Shanghai" ["Shanghai Municipality", "Shanghai Mansion", "Shanghai TV Station"], and "Shanghai Municipality" is selected as the candidate entity matching the entity "Shanghai"; the candidate entity "Pudong District" with a confidence level of 1 is selected from the candidate entity group corresponding to the entity "Pudong" ["Pudong District", "Pudong Road", "Pudong Building"], and "Pudong City" is selected as the candidate entity matching the entity "Pudong", and so on.

[0073] Of course, each candidate entity can also be sorted according to the confidence level of each candidate entity in each candidate entity group (as shown in Table 3, sorted in descending order of confidence level), and then the candidate entity with the highest confidence level is taken as the candidate entity to match (corresponding to each candidate entity group).

[0074] Step S302: If the first entity in the first entity set is the same as the second entity in the second entity set, and the matching degree between the first candidate entity and the first entity and / or the matching degree between the second candidate entity and the second entity is greater than the first threshold, then the first candidate entity or the second candidate entity is determined as the first target candidate entity.

[0075] The first candidate entity is a candidate entity that matches the first entity, and the second candidate entity is a candidate entity that matches the second entity.

[0076] Specifically, as shown in Table 3, if the entity "Shanghai" identified by the detection method based on the preset dictionary is the same as the entity "Shanghai" identified by the detection method based on the preset rules (which means that the "Shanghai" identified by the two methods does not conflict) and the confidence level of "Shanghai" is greater than the first threshold (e.g., 0.5), then "Shanghai" is determined as the first target candidate entity.

[0077] Step S303: The matching degree between the third candidate entity and the corresponding third entity in each candidate entity group is weighted and summed to obtain the fusion matching degree for each candidate entity set.

[0078] The third candidate entity is a candidate entity that matches the third entity, and the third entity is different from the first entity and the second entity.

[0079] For step S303, the weighted summation of the matching degree between the third candidate entity and the corresponding third entity in each candidate entity group may include: determining the length ratio of the third entity to the entities other than the first entity in the search request as the weight of the third candidate entity; and performing a weighted summation based on the matching degree between the third candidate entity and the third entity and the weight of the third candidate entity.

[0080] Apart from entities that do not conflict, for other entities in each entity set (i.e., third entities, such as "Pudong" in the first row of Table 3; "Pudong" and "Dongmingyuan" in the second row and "Pudong Dongmingyuan" in the third row), calculate the fusion matching degree with respect to the corresponding candidate entity set.

[0081] Specifically, for the first row shown in Table 3, the weight of the third candidate entity "Pudong" is determined as 2 / 5 of the length ratio of the third entity "Pudong" to "Pudong Dongmingyuan" (the content other than "Shanghai" in the search request "Shanghai Pudong Dongmingyuan"). Then, a weighted sum is performed based on the matching degree 1 of the third candidate entity "Pudong District" and the determined weight 2 / 5, so that the fusion matching degree for the first row (i.e. the fusion matching degree for this candidate entity set) can be determined to be 0.4. For the second row shown in Table 3, the length ratios of the third entity "Pudong" and "Dongmingyuan" to "Pudong Dongmingyuan" (2 / 5 and 3 / 5 respectively) are determined as the weights of the third candidate entities "Pudong District" and "Hesha Hangcheng Dongmingyuan". Then, a weighted sum is calculated based on the matching degree of the third candidate entity "Pudong District" (1) and the determined weight 2 / 5, and the matching degree of the third candidate entity "Dongmingyuan" (0.9) and the determined weight 3 / 5. This results in a fusion matching degree (i.e., the fusion matching degree for this candidate entity set) of 0.94 for the second row. For the third row shown in Table 3, the length ratio of the third entity "Pudong Dongmingyuan" to "Pudong Dongmingyuan" (1) is determined as the weight of the third candidate entity "Pudong Garden". Then, a weighted sum is calculated based on the matching degree of the third candidate entity "Pudong Dongmingyuan" (0.69) and the determined weight 1. This results in a fusion matching degree (i.e., the fusion matching degree for this candidate entity set) of 0.69 for the third row.

[0082] Step S304: If the fusion matching degree of a specific candidate entity set in the plurality of candidate entity sets is the highest and greater than the second threshold, the third candidate entity is determined as the second target candidate entity.

[0083] Step S303 determines that the fusion matching degree of each candidate entity set is 0.4, 0.94, and 0.69, respectively. It is also determined that the fusion matching degree of the candidate entity set in the second row is the highest and is greater than the second threshold (e.g., 0.85). Thus, the corresponding third candidate entities "Pudong" and "Dongmingyuan" are further determined as the second target candidate entities.

[0084] In other words, both the detection method based on the preset dictionary and the detection method based on the preset algorithm identify "Pudong", but the detection method based on the preset rules identifies "Pudong Dongmingyuan". That is, the results of the two entities identified by different detection methods are conflicting (or ambiguous), and the above process can effectively eliminate the corresponding ambiguity.

[0085] Step S305: Combine the first target candidate entity and the second target candidate entity into the target candidate entity combination.

[0086] The first target candidate entity "Shanghai" and the second target candidate entity "Pudong District" determined through the above steps are combined with "Hesha Hangcheng Dongmingyuan" to form the target candidate entity combination.

[0087] The analysis method may further include: if the target candidate entity combination is not selected from the multiple entity sets based on the preset screening conditions, a conditional random field model is used to screen the target candidate entity combination from the multiple candidate entity sets.

[0088] When either of the following two conditions is met: the first entity is the same as the second entity but the matching degree between the first candidate entity and the first entity is less than or equal to the first threshold, or the matching degree between the second candidate entity and the second entity is the highest but less than or equal to the second threshold, it indicates that the target candidate entity combination has not been selected from the multiple entity sets based on the preset screening conditions.

[0089] The training process of the Conditional Random Field (CRF) model mainly consists of the following three steps: First, preparing training samples; second, automatically generating a set of feature functions by defining feature templates; and third, learning the weights of the feature functions using a predetermined algorithm (e.g., the L-BFGS algorithm). The following describes each of these three steps in detail.

[0090] First, prepare training samples.

[0091] Example (Organize all users' search requests for the query "houses in Dongmingyuan, Pudong, Shanghai" into the pattern below, and manually label them; B, I, and O are the results of the manual labeling).

[0092] Up B

[0093] HaiI

[0094] Pu B

[0095] East I

[0096] O

[0097] East B

[0098] Ming I

[0099] Court I

[0100] O

[0101] Room O

[0102] Zi O

[0103] Among them, B represents the beginning, I represents the middle, and O represents meaningless (these are the three main categories).

[0104] Second, by defining feature templates, a set of feature functions is automatically generated.

[0105] #Unigram (divided by single characters)

[0106] U01: %x[-1,0] (where -1 represents the previous line of the current line)

[0107] U02: %x[0,0] (where 0 represents the current line)

[0108] U03: %x[1,0] (where 1 represents the next line of the current line)

[0109] U04: %x[2,0] (where 2 represents two lines below the current line)

[0110] …

[0111] #Bigram (divided by two characters)

[0112] The following is an explanation of the feature templates.

[0113] Feature template format: %x[row,col]. x can take U (Unigram) or B (Bigram), corresponding to two types. The numbers in the square brackets are used to calibrate the feature source. row represents the line relative to the current position, and 0 is the current line; col corresponds to the column in the training file. Here, only the first column (number 0), that is, the text, is used.

[0114] Specifically, for the Unigram type, each line of the template generates a set of state feature functions with a quantity of L*N. L is the number of label states. N is the number of unique samples after expanding this line of the template on the training set (the above example is 11). In this example, the number of unique characters in the first column is 11, so there are L*N = 3*11 = 33.

[0115] For example: U01: %x[0,0], the following 15 functions are generated:

[0116] func1 = if(output = B and feature = U01: “上”) return 1 else return 0

[0117] func2 = if(output = I and feature = U01: “上”) return 1 else return 0

[0118] func3 = if (output = O and feature = U01: "up") return 1 else return 0

[0119] …

[0120] Func31 = if (output = B and feature = U01: "sub") return 1 else return 0

[0121] Func32 = if (output = I and feature = U01: "sub") return 1 else return 0

[0122] Func33 = if (output = O and feature = U01: "sub") return 1 else return 0

[0123] After these functions are trained, their weights represent the probability that the text within the function corresponds to the label (figuratively speaking, the sum of probabilities can be greater than 1).

[0124] Different from Unigram, the functions generated by Bigram - type templates have one more parameter: the label of the previous node. The first character is B, and each line %x[#,#] generates an edge function f(s’, s, o) in CRFs, where s’ is the label at time t - 1. That is to say, the Bigram type is roughly the same as Unigram, but also takes into account the label at time t - 1.

[0125] Each line of the template generates L * L * N feature functions. After training, the weights of these functions reflect the influence of the label of the previous node on the current node.

[0126] func1 = if (prev_output = B and output = B and feature = B01: "up") return 1 else return 0

[0127] func1 = if (prev_output = B and output = I and feature = B01: "up") return 1 else return 0

[0128] func1 = if (prev_output = B and output = O and feature = B01: "up") return 1 else return 0

[0129] func1 = if(prev_output = I and output = B and feature = B01: "up") return 1 else return 0

[0130] func1 = if(prev_output = I and output = I and feature = B01: "up") return 1 else return 0

[0131] func1 = if(prev_output = I and output = O and feature = B01: "up") return 1 else return 0

[0132] func1 = if(prev_output = O and output = B and feature = B01: "up") return 1 else return 0

[0133] func1 = if(prev_output = O and output = I and feature = B01: "up") return 1 else return 0

[0134] func1 = if(prev_output = O and output = O and feature = B01: "up") return 1 else return 0

[0135] …

[0136] B means that the previous output token and the current token will be combined into bigram features.

[0137] Third, learn the weights of the feature functions through a predetermined algorithm (such as the L-BFGS algorithm).

[0138]

[0139]

[0140] If we represent the weight vector as w, that is, w = (w1, w2, …, w K ) T .

[0141] The feature functions introduced above constitute f in the above formula kThen, the L-BFGS algorithm can be used to solve it. L-BFGS is a general-purpose algorithm library, and third-party software can be used to solve for the weight w.

[0142] The process of parsing a search request is explained and illustrated below, such as... Figure 4 As shown.

[0143] The process of parsing the search request may include the following steps S401-S404.

[0144] Step S401: Multiple boundary detection methods are used to identify the user's search request.

[0145] Specifically, various boundary detection methods (e.g., detection methods based on preset dictionaries, preset algorithms, preset rules, and preset templates) can be used to identify the search request in order to obtain the entity set obtained by each detection method.

[0146] Step S402: Expand the recall of the identification results.

[0147] Each identified entity can be expanded for recall based on multiple methods (e.g., from the text dimension and user preference dimension) to obtain changed candidate entities.

[0148] Step S403: Calculate the matching degree between each recall result and the corresponding identification result.

[0149] That is, calculate the matching degree between each candidate entity and the corresponding entity in each candidate entity group as shown in Table 3.

[0150] Step S404: Determine the target candidate entity combination that matches the search request based on the calculated matching degree.

[0151] Using preset filtering conditions or based on a conditional random field model, and based on the matching degree between each recall result and the corresponding identification result, a combination of target candidate entities matching the user's search request is selected from the recall results.

[0152] Through the above technical solution, this invention creatively first employs multiple boundary detection methods to identify the user's search request, thereby obtaining multiple entity sets corresponding to the search request; then, it expands and recalls these multiple entity sets to obtain multiple candidate entity sets corresponding to the multiple entity sets; finally, based on preset filtering conditions, it filters target candidate entity combinations that match the search request from these multiple candidate entity sets. Therefore, this invention can accurately identify the geographical location in a user's search, thus greatly improving the user experience, and is applicable to C-end search and assistant dialogue scenarios.

[0153] Figure 5This is a structural diagram of a search request parsing system provided in an embodiment of the present invention. For example... Figure 5 As shown, the parsing system may include: an identification device 10, used to identify a user's search request using multiple boundary detection methods to obtain multiple entity sets corresponding to the search request; a recall device 20, used to expand the recall of the multiple entity sets to obtain multiple candidate entity sets corresponding to the multiple entity sets; and a first filtering device 30, used to filter target candidate entity combinations that match the search request from the multiple candidate entity sets based on preset filtering conditions.

[0154] In at least one embodiment, the plurality of boundary detection methods includes at least two of the following: a detection method based on a preset dictionary, a detection method based on a preset algorithm, a detection method based on preset rules, and a detection method based on a preset template.

[0155] In at least one embodiment, the recall device 20 includes: a recall module for recalling each entity in each entity set of the plurality of entity sets from a text dimension and / or a user preference dimension to obtain a candidate entity group corresponding to each entity; and a combination module for combining the candidate entity groups corresponding to each entity in each entity set for each entity set to form a candidate entity set corresponding to each entity set.

[0156] In at least one embodiment, the recall module is used to recall each entity in each entity set of the plurality of entity sets from the text dimension, including: recalling the binary sequence of each entity.

[0157] In at least one embodiment, the recall module is used to recall each entity in each entity set of the plurality of entity sets from the text dimension, including: recalling the binary sequence of each entity; and recalling based on at least one of the following for each entity: pinyin, text, intent, popularity, and matching rate.

[0158] In at least one embodiment, the first screening device 30 includes: a determining module, configured to determine the matching degree between each candidate entity in each candidate entity set of the plurality of candidate entity sets and a corresponding entity in a corresponding entity set of the plurality of entity sets; and a screening module, configured to screen the target candidate entity combination from the plurality of candidate entity sets based on the matching degree between each candidate entity and the corresponding entity.

[0159] In at least one embodiment, the determining module includes: a parameter determining unit, configured to determine the edit distance, Jaccard distance, proportion of the length of the common substring in each candidate entity set to the length of the corresponding entity in the corresponding entity set, pairing rate, and proportion of the length of the corresponding entity that serves as a prefix or suffix in each candidate entity; and a summing unit, configured to perform a weighted summation of the edit distance, the Jaccard distance, the proportion of the length of the common substring in each candidate entity, the pairing rate, and the proportion of the length of the corresponding entity that serves as a prefix or suffix in the length of each candidate entity, to determine the matching degree between each candidate entity and the corresponding entity.

[0160] In at least one embodiment, the filtering module includes: a matching candidate determination unit, configured to determine a candidate entity matching each entity in each entity set based on the matching degree between each candidate entity and the corresponding entity; and a first target candidate determination unit, configured to determine the first candidate entity or the second candidate entity as a first target candidate entity when the first entity in the first entity set is the same as the second entity in the second entity set, and the matching degree between the first candidate entity and the first entity and / or the matching degree between the second candidate entity and the second entity is greater than a first threshold, wherein the first candidate entity is a candidate entity matching the first entity, and the second candidate entity is a candidate entity matching the first entity. The system includes: a candidate entity for the second entity; a matching degree determination unit, configured to perform a weighted summation of the matching degree between the third candidate entity and the corresponding third entity to obtain a fusion matching degree for each candidate entity set, wherein the third candidate entity is a candidate entity that matches the third entity, and the third entity is different from the first entity and the second entity; a second target candidate determination unit, configured to determine the third candidate entity as a second target candidate entity when the fusion matching degree for a specific candidate entity set in the plurality of candidate entity sets is the highest and greater than a second threshold; and a combination unit, configured to combine the first target candidate entity and the second target candidate entity into the target candidate entity combination.

[0161] In at least one embodiment, the matching degree determination unit includes: a weight determiner, configured to determine the length ratio of the third entity to the entities in the search request other than the first entity as the weight of the third candidate entity; and a matching degree determiner, configured to perform a weighted sum based on the matching degree between the third candidate entity and the third entity and the weight of the third candidate entity.

[0162] In at least one embodiment, the matching candidate determination unit is used to determine the candidate entity that matches each entity in each entity set by: for each entity in each entity set, filtering the candidate entity with the highest matching degree from the corresponding candidate entity group, and determining the filtered candidate entity as the candidate entity that matches each entity.

[0163] In at least one embodiment, the parsing system further includes: a second filtering device, used to filter the target candidate entity combination from the plurality of candidate entity sets using a conditional random field model when the target candidate entity combination is not filtered out from the plurality of candidate entity sets based on the preset filtering conditions.

[0164] For specific details and benefits of the search request parsing system provided by the present invention, please refer to the above description of the search request parsing method, which will not be repeated here.

[0165] An embodiment of the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for parsing search requests.

[0166] An embodiment of the present invention also provides an electronic device, the electronic device comprising: a processor; a memory for storing a computer program; the processor being configured to read the computer program from the memory and execute the computer program to implement the above-described method for parsing search requests.

[0167] An embodiment of the present invention also provides a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the above-described method for parsing search requests.

[0168] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention.

[0169] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not describe the various possible combinations separately.

[0170] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0171] Furthermore, various different implementations of the present invention can be combined arbitrarily, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed in the present invention.

Claims

1. A method for parsing search requests, characterized in that, The parsing method includes: Multiple boundary detection methods are used to identify the user's search request in order to obtain multiple entity sets corresponding to the search request, and at least one of the multiple entity sets includes the geographical location in the search request; Expand the recall of the plurality of entity sets to obtain a plurality of candidate entity sets corresponding to the plurality of entity sets; and Based on preset filtering criteria, the search process filters a combination of target candidate entities from the multiple candidate entity sets that match the search request, including the following steps: Determine the matching degree between each candidate entity in each candidate entity set and the corresponding entity in the corresponding entity set of the plurality of entity sets; Based on the matching degree between each candidate entity and the corresponding entity, determine the candidate entity that matches each entity in each entity set; If the first entity in the first entity set is the same as the second entity in the second entity set, and the matching degree between the first candidate entity and the first entity and / or the matching degree between the second candidate entity and the second entity is greater than a first threshold, the first candidate entity or the second candidate entity is determined as the first target candidate entity, wherein the first candidate entity is a candidate entity that matches the first entity, and the second candidate entity is a candidate entity that matches the second entity. The matching degree between the third candidate entity in each candidate entity group and the corresponding third entity is weighted and summed to obtain the fusion matching degree for each candidate entity set, wherein the third candidate entity is a candidate entity that matches the third entity, and the third entity is different from the first entity and the second entity; If the fusion matching degree of a specific candidate entity set within the plurality of candidate entity sets is the highest and greater than a second threshold, then the third candidate entity is determined as the second target candidate entity; and The first target candidate entity and the second target candidate entity are combined to form the target candidate entity combination.

2. The method for parsing search requests according to claim 1, characterized in that, The various boundary detection methods include at least two of the following: a detection method based on a preset dictionary, a detection method based on a preset algorithm, a detection method based on preset rules, and a detection method based on a preset template.

3. The method for parsing search requests according to claim 1, characterized in that, The expanded recall of the multiple entity sets includes: Recall each entity in each entity set from the plurality of entity sets from the text dimension and / or the user preference dimension to obtain a candidate entity group corresponding to each entity; and For each entity set, the candidate entity groups corresponding to each entity in each entity set are combined to form a candidate entity set corresponding to each entity set.

4. The method for parsing search requests according to claim 3, characterized in that, The step of recalling each entity in each entity set from the text dimension includes: recalling the binary sequence of each entity. Preferably, the recall of each entity in each of the multiple entity sets from the text dimension includes: Recall for each entity's binary sequence; and recall based on at least one of each entity's pinyin, text, intent, popularity, and match rate.

5. The method for parsing a search request according to claim 1, characterized in that, Determining the matching degree between each candidate entity in each of the plurality of candidate entity sets and the corresponding entity in the corresponding entity set of the plurality of entity sets includes: Determine the edit distance, Jaccard distance, common substring proportion of the length of each candidate entity in each candidate entity set, pairing rate, and proportion of the length of each candidate entity that is a prefix or suffix; and The matching degree between each candidate entity and the corresponding entity is determined by weighted summation of the edit distance, the Jaccard distance, the length ratio of the common substring, the pairing rate, and the length ratio of the corresponding entity as a prefix or suffix to the length of each candidate entity.

6. The method for parsing a search request according to claim 1, characterized in that, The weighted summation of the matching degree between the third candidate entity and the corresponding third entity in each candidate entity group includes: The proportion of the length of the third entity relative to the length of all entities in the search request other than the first entity is determined as the weight of the third candidate entity; and The results are weighted sums based on the matching degree between the third candidate entity and the third entity, and the weight of the third candidate entity.

7. The method for parsing a search request according to claim 1, characterized in that, The process of determining candidate entities that match each entity in each entity set includes: For each entity in each entity set, the candidate entity with the highest matching degree is selected from the corresponding candidate entity group, and the selected candidate entity is determined as the candidate entity that matches each entity.

8. The method for parsing a search request according to claim 1, characterized in that, The parsing method also includes: If the target candidate entity combination is not selected from the multiple entity sets based on the preset screening conditions, a conditional random field model is used to screen the target candidate entity combination from the multiple candidate entity sets.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method for parsing a search request as described in any one of claims 1-8.

10. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method for parsing the search request according to any one of claims 1-8.

11. A computer program product, the computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for parsing the search request as described in any one of claims 1-8.

Images