Dual session enhanced time-aware query completion system and method

Abstract

The invention provides a dual-session enhanced time-aware query completion model, which is particularly suitable for a real-time query completion scene of a search engine. By introducing time perception and personalized features, the method can effectively improve the accuracy and correlation of query completion. Specifically, the system comprises a query log screening module which is used for carrying out personalized and public log distinguishing and screening on query logs based on timestamps and user IDs; the dual-session enhanced time-aware session subset screening module is used for screening personalized and public time-aware session subsets by calculating semantic and time similarity; the popularity perception feature subset construction module is combined with popularity information of the query log; and the output module based on the NLG model is used for integrating the information and generating a query completion candidate list. According to the method, through a deep learning technology, the dynamic change of the query trend is captured, the timeliness and individuation of query completion are enhanced, and the user experience is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the fields of information retrieval and natural language processing, and in particular to a dual session enhanced time-aware query auto-completion model (DSETS-QAC) for improving the accuracy of query completion in search engines and user experience. Background Art

[0002] Query Auto-Completion (QAC) technology refers to the technology that provides query suggestions based on the user's historical query information or other users' historical query information when the user is typing a query. QAC technology plays a vital role in modern search engines, helping users complete queries quickly and accurately, improving search efficiency and user satisfaction.

[0003] In recent years, deep learning methods have been widely used in the field of natural language processing (NLP), and deep learning based query auto-completion (DQAC) technology has made significant progress. Traditional QAC methods, such as the most popular completion (MPC) method based on maximum likelihood estimation, mainly rely on query frequency to infer user needs, but this method is less effective when dealing with rare query prefixes and short query prefixes, and lacks in-depth mining of query context information.

[0004] With the emergence of pre-trained language models such as BERT, BART, and T5, Natural Language Generation (NLG) models are widely used in QAC technology. These NLG models can provide good completion effects at the semantic level, but often only model the query prefix, ignoring other important contextual information, such as time information and personalized feature information, and it is difficult to capture the dynamic changes of query trends.

[0005] In addition, existing QAC technologies also have certain deficiencies in dealing with real-time and personalized needs. Modern users' query behaviors are highly personalized and dynamic, and the query needs of different users in different time periods often vary significantly. At the same time, users' query interests and habits may change over time. Therefore, relying solely on static query log data for completion is not only difficult to meet users' personalized needs, but may also lead to insufficient timeliness of the completion results.

[0006] To address these challenges, researchers have proposed a variety of improvement schemes. For example, QAC methods based on contextual information attempt to enhance the completion effect by introducing users' historical query sessions, but these methods usually ignore the impact of time factors on user query behavior. On the other hand, QAC methods based on time series analysis attempt to capture the dynamic changes of query trends, but still have shortcomings in processing personalized features.

[0007] In this context, in order to solve the problems of time perception and personalized needs at the same time, the present invention proposes a dual-session enhanced time-aware query completion model (DSETS-QAC). Through the dual-session enhanced time-aware session subset screening module, the model can effectively combine time information and personalized feature information, thereby improving the accuracy and robustness of query completion. At the same time, combined with the powerful text generation capability of the pre-trained NLG model, the DSETS-QAC model can show higher efficiency and accuracy in generating query completion candidate lists. Summary of the invention

[0008] The present invention proposes a dual-session enhanced time-aware query completion model (DSETS-QAC) to solve the problem of insufficient time information and personalized requirements in query automatic completion technology.

[0009] In order to achieve the above object, the present invention adopts the following technical scheme.

[0010] A time-aware query completion model based on dual-session enhancement, including:

[0011] Step 1: Preprocess the query log data to be analyzed and extract the user's query history data, including query terms, timestamps, user IDs, etc. Clean and preprocess the raw data to remove noise and invalid data to ensure data quality.

[0012] Step 2: Through the dual-session enhanced time-aware session subset filtering module, a personalized time-aware session subset and a public time-aware session subset are obtained. The personalized time-aware session subset is based on the query history of a single user, filtering out sessions related to the current query prefix, reflecting the user's personalized query behavior; the public time-aware session subset is based on the query history of all users, filtering out sessions that frequently appear in the same time period, reflecting the public time trend of the query.

[0013] Step 3: The filtered conversation subset is used as context information and input into the pre-trained large language model for training. During the training process, the model generates a query completion candidate list by learning the relationship between the query prefix and the conversation subset.

[0014] Step 4: When the user enters a query prefix, the trained model is used to generate several query completion candidates. The candidates are sorted and displayed to the user based on their relevance and matching degree, helping the user to complete the query more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a structural schematic diagram of the DSETS-QAC model of the present invention;

[0016] Figure 2 This is a comparison chart of the test results of the present invention and other model methods on the test data set;

[0017] Figure 3 This is a comparison chart of the efficiency of the present invention and other model methods on the test data set. DETAILED DESCRIPTION

[0018] The method of the present invention is further described in detail below based on the implementation principle and specific data processing flow.

[0019] Step 1: Query the log filtering module

[0020] The first task of the DSETS-QAC model is to process massive query logs. Before filtering the time-aware session subset, the query logs need to be initially filtered. First, the query logs are divided into the user's personalized query logs and the public query logs without the user according to the user's ID. Then, according to the timestamp of the query prefix, the most recent queries are filtered from the query logs one month before the timestamp date. For example, if the timestamp is <2006.03.24 18:53:10>, then every day from 2006.02.25 to 2006.03.24, the queries in the time window around 18:53:10 are retrieved, and the retrieval rule is to compare the absolute value of the timestamp after sine transformation. Let x be the hour part of the timestamp, y be the minute part of the timestamp, and z be the second part of the timestamp, then the sine transformation formula of the timestamp is:

[0021]

[0022] This method of filtering query words by time anchor point can filter query words by day, and focus on extracting query words with time periodicity in modeling public time-aware session factors. Since the number of user query logs may be small, this module will not filter if the number of user logs is less than k; if the number of logs is greater than k, the filtering rules are the same as those for public query logs.

[0023] Step 2: Dual-session enhanced time-aware session subset filtering module

[0024] The DSETS-QAC model is divided into two parts in the feature representation layer: personalized time-aware session subset filtering and common time-aware session subset filtering.

[0025] (1) Personalized time-aware session subset filtering

[0026] In order to filter out the personalized time-aware conversation subset, the present invention establishes a calculation method that simultaneously considers the time similarity and semantic similarity between the query prefix and the query word. When filtering the personalized time-aware conversation subset, it is not rigorous to consider the time similarity or semantic similarity alone as a filtering indicator, and the two should be considered comprehensively. For example, if the semantic similarity between the query word and the query prefix is ​​high, but the time similarity between the two is low, it should not be an ideal filtering result; only when both the semantic and time similarities are high should they be filtered into the personalized time-aware conversation subset. The specific algorithm is as follows:

[0027] First, we calculate the semantic similarity. Let the query prefix be p, the single query term in the personalized time-aware candidate session be q, and M p and M q are the word similarity matrices obtained by p and q through the semantic space, f(p,q) is the function for calculating the similarity between the prefix and the most recent query, len(p) and len(q) represent the length of the query prefix and query word respectively. The semantic similarity calculation formula is:

[0028]

[0029] Next, the standard is calculated using the classification method and normalized by dividing it by the product of the query prefix and the query term length. The standard calculation formula is:

[0030]

[0031] Among them, depth(LCS) is the depth of the lowest common ancestor of two words, depth(M p ) and depth(M q ) represent M p and M q Position depth in WordNet.

[0032] Next is the minimum edit distance (MED) method, which is used when the query prefix cannot be recognized. The calculation formula is:

[0033] Sim(p,q)=1-MED(p,q) (4)

[0034] After calculating the semantic similarity, in order to calculate the temporal similarity between the query prefix and the query term, the present invention proposes a feature factor that integrates the temporal perception feature and the semantic similarity feature. The calculation formula is as follows:

[0035]

[0036] Among them, q * is the query with the highest score calculated by Sim among the most recent N queries in the personalized time-aware candidate session, q n Indicates the nth query word in the query log, t p To query the prefix time, Indicates q * Query time, Indicates q n Query time, all times have been sinusoidally transformed, and the sinusoidal transformation formula is the same as (1).

[0037] (2) Public time-aware session subset screening

[0038] The process of filtering the common time-aware session subset is similar to that of filtering the personalized time-aware session subset, but it focuses more on filtering the common time-aware candidate sessions from the query log. The filtering method has been described in the section on filtering the personalized time-aware session subset.

[0039] Step 3: Popularity-aware feature subset representation

[0040] The present invention follows the method of representing popularity perception factors in the Trie-NLG model, and uses the MPC method to model popularity perception factors. For query prefixes that have appeared in the query log, MPC can complete the query, but for rare prefixes, the MPC method cannot complete the query effectively. To solve this problem, the present invention uses the method in the Trie-NLG model to adopt the suffix MPC method for rare queries, and combines it with the traditional MPC method to obtain a hybrid MPC method. Suppose the user inputs a query prefix p. If p has appeared in the query log Q, it is calculated using the MPC method. The MPC method uses the prefix p as the index to search for all query records q that start with the prefix p in the search log, and records each probability as W(q), f Q (q) represents the frequency of query q in query log Q, and is the set of query words starting with prefix in query log. For query words starting with prefix, the calculation formula is as follows:

[0041]

[0042] Step 4: Output module based on NLG model

[0043] In this invention, the BART model is selected as the NLG model to complete the query completion task. The input of the BART model is all the session subsets connected in series, separated by a marker, which is used to mark the beginning of a new session subset. u , a subset of common time-aware sessions S p and the popularity perception factor conversation subset S mpc For splicing, the splicing formula is as follows:

[0044] S=[[SEP]S u [SEP]S p [SEP]S mpc [SEP]p] (7)

[0045] After obtaining the conversation subset, the text needs to be segmented into a series of tokens through Tokenizer. The segmentation formula is:

[0046] U=Tokenizer(S)={u1,…,u n} (8)

[0047] Among them, U is the Token set, u1,…,u n is a single token. It will be used as the input of the bidirectional encoder and autoregressive decoder. The bidirectional encoder uses the fine-tuned pre-trained encoder and uses the autoregressive decoder to obtain the hidden layer state of U. In the encoder stage, the input is for u n The state at the encoder layer 0. The state conversion formula for each layer in the encoder is as follows:

[0048]

[0049] in, is the attention mechanism from the encoder to the encoder at layer l. This formula describes the state transition of u from layer l-1 to layer l. e After the state conversion, the encoder finally gets the representation

[0050] In the autoregressive decoder, the input is different from the encoder. The output of the previous step is used as the input of the next step. Assuming w is the output of the autoregressive decoder, the output of the l-1 layer is expressed as The conversion formula from the l-1th layer to the lth layer autoregressive decoder is:

[0051]

[0052] in, is the attention mechanism from the l-th layer decoder to the encoder, is the attention mechanism from the decoder to the decoder at layer l, w (l) is the result of the decoder layer l. The training goal of the model of the present invention is to obtain the maximum likelihood estimation (Maximum Likelihood Estimation, MLE), and the loss function formula is as follows:

[0053]

[0054] Among them, |D| is the number of training data sets, is the input of the t-1th step in the lth layer, is the true value output of the t-th step in the l-th layer. Calculate the hidden state for the model for a given input sequence U probability.

[0055] The above are only preferred specific embodiments of the present invention, but the protection scope is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope should be based on the content of the claims.

Claims

1. A dual-session enhanced time-aware query autocompletion model (DSETS-QAC), characterized by: The model is implemented by the following modules: (1) Dual-session enhanced time-aware session subset screening module: used to screen personalized time-aware session subsets and public time-aware session subsets, and input the session subsets as context information into the next stage to enhance the model's understanding of time sensitivity and personalized needs; (2) Popularity factor perception module: It filters the popularity features in the query logs and inputs them into the model as context information, thereby improving the query completion capability of the model in different fields, especially in dealing with cold start problems and long-tail queries. (3) Pre-trained large language model module: Utilize the pre-trained large language model and combine it with the information of the above-mentioned conversation subset to generate a query completion candidate list.

2. The dual-session enhanced time-aware query auto-completion model according to claim 1, characterized in that: The dual-session enhanced time-aware session subset screening module further comprises: (1) Query log screening module: The query log is divided into the user's personalized query log and the public query log; according to the timestamp of the query prefix, the most recent n queries are screened from the query log one month before the timestamp date; if the number of user query logs is less than n, no screening is performed; if the number is greater than n, the screening rules are the same as those for the public query log. (2) Personalized time-aware session subset screening submodule: used to screen the session subset related to the user's personalized time, and the submodule is implemented by the following steps: S1: Collect and analyze the user's historical query logs, extract timestamp information, and build the user's time behavior model; S2: Using the user's ID and timestamp, obtain personalized time-aware candidate sessions arranged in chronological order; S3: For the query prefix and the query word in the candidate session, calculate the semantic similarity between them. If the query word and the prefix have a similarity matrix in the semantic space, use the classification method based on the depth of the lowest common ancestor of the two words to calculate the standard. If not, use the minimum edit distance method; S4: Calculate the temporal similarity between the query prefix and the query terms in the candidate session, which involves performing a sine transform on the timestamp and calculating a comprehensive evaluation of the time interval and semantic similarity. At the same time, a feature factor is proposed that combines the time-aware feature and the semantic similarity. Only when both are high will the query term be included in the personalized time-aware session subset, thereby screening out the best matching session. (3) Public time-aware session subset screening submodule: used to screen session subsets related to public time, the submodule is implemented by the following steps: S1: Extract common time features from the query logs of a large number of users, such as holidays, weekends, major events, etc. S2: Based on the extracted common time features, the screening method of the personalized time-aware session subset screening submodule is used to screen out query sessions related to these time features and generate a common time-aware session subset.

3. The dual-session enhanced time-aware query auto-completion model according to claim 1, characterized in that: The popularity factor perception module is implemented by the following steps: S1: If the query prefix has appeared in the query log, the MPC method is used to calculate the frequency of all queries starting with the prefix; for rare prefixes, the suffix MPC method is used, that is, the prefix is ​​divided into multiple substrings, starting from the last character, and the query term is searched from the query log using the suffix; S2: Finally, the query words with the highest scores are selected to form the popularity factor feature subset.

4. The dual-session enhanced time-aware query auto-completion model according to claim 1, characterized in that: The pre-trained large language model module combines information of the personalized time-aware session subset and the public time-aware session subset to generate a query completion candidate list, and the steps of the generation process include: S1: By inputting the contextual information of the personalized time-aware session subset and the common time-aware session subset into the pre-trained large language model, the model can consider both user personalized needs and common time features, thereby generating a more accurate and relevant query completion candidate list; S2: In the process of generating the query completion candidate list, the model further optimizes the ordering of the candidate list based on the conditional probability of the string generated by concatenating the query prefix, personalized time-aware session subset, public time-aware session subset, and popularity factor session subset, so that completion candidates with high relevance and high popularity are more likely to appear in the user's preferred position.

5. The dual-session enhanced time-aware query auto-completion model according to claim 1, characterized in that: The model implements query autocompletion through the following steps: S1: Extract query prefix, timestamp, user ID and other information from the query log, perform preliminary screening and cleaning of the query log to ensure the data quality of subsequent processing; S2: Filter personalized time-aware session subsets and public time-aware session subsets through the dual-session enhanced time-aware session subset filtering module; S3: Filter the popularity feature subset through the popularity factor perception module; S4: Input the screened personalized time-aware session subset, public time-aware session subset, and popularity feature subset as context information into the pre-trained large language model; S5: Generate a query completion candidate list through a pre-trained large language model. During the generation process, the model can quickly and accurately predict the most likely query completion options based on the input context information.

Images