Template search method and device based on multi-path recall

Abstract

The invention discloses a template search method and device based on multi-path recall, and the method comprises the steps: receiving a keyword and a user ID input by a user, combining a vector database and ElasticSearch, and adopting a multi-path recall strategy to obtain a recall template set, the multi-path recall strategy comprising three recall paths of keyword vectorization recall, historical behavior recall and classification label recall; template detailed data are retrieved according to the recall template set, a comprehensive sorting model of dynamic weight adjustment is introduced, a comprehensive sorting score is calculated in combination with a plurality of sorting factors, recall templates are sorted and then returned, and the sorting factors comprise correlation scores, template values, top values and new time; the vector representation of the hot template is pre-calculated and cached in a vector database, and the ElasticSearch synchronously updates the index; according to the multi-path recall mechanism, the recall coverage rate is remarkably increased; and a comprehensive sorting model for dynamic weight adjustment is introduced, so that the sorting accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of information retrieval technology, and more specifically to a template search method and apparatus based on multi-path recall. Background Technology

[0002] Currently, existing recall schemes used in template search or information retrieval have the following main drawbacks:

[0003] Low recall efficiency: Existing multi-path recall mechanisms usually rely on a single vector model or limited data sources, resulting in a limited recall range and failure to fully utilize multi-dimensional features. In particular, highly relevant templates may be missed in template search.

[0004] Insufficient sorting accuracy: In the current comprehensive ranking technology, the weight allocation of sorting factors is relatively fixed or lacks dynamic adjustment. It fails to fully consider dynamic factors such as template value, top placement requirements and new product launch time, which affects the accuracy of search results.

[0005] The separation of retrieval and ranking: In existing technologies, vector similarity calculation and ElasticSearch retrieval are performed in stages, lacking unified multi-way recall and ranking optimization, which increases computational complexity and latency.

[0006] Therefore, how to develop a template search method based on multi-path recall to improve recall coverage and ranking accuracy, reduce query latency, and optimize system response speed is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a template search method and apparatus based on multi-path recall to solve some of the technical problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A template search method based on multi-path recall includes the following steps:

[0010] S1. Receive the keywords and user ID input by the user, combine the vector database and ElasticSearch, and use a multi-path recall strategy to obtain the recall template set. The multi-path recall strategy includes three recall paths: keyword vectorization recall, historical behavior recall, and category tag recall.

[0011] S2. Retrieve detailed template data based on the recalled template set, introduce a comprehensive ranking model with dynamic weight adjustment, calculate a comprehensive ranking score by combining multiple ranking factors, and return the recalled templates after ranking. The ranking factors include relevance score, template value, top ranking value, and new release time.

[0012] S3. Precompute and cache vector representations of popular templates in a vector database, and update the index synchronously in Elasticsearch.

[0013] Preferably, the specific content of step S1 is as follows:

[0014] S11. Using Natural Language Processing (NLP) technology, the keywords input by the user are converted into vector representations;

[0015] S12. Using the cosine similarity algorithm, calculate the similarity between the query vector and the template vector in the vector space, filter out templates with a similarity greater than a preset threshold from the vector database, and use them as the first template set; recall relevant templates based on the user ID's historical search behavior, and use them as the second template set; use the template's category tags to recall templates through ElasticSearch, and use them as the third template set.

[0016] S13. Merge the first template set, the second template set, and the third template set through intersection and union operations to expand the recall scope and obtain the recall template set.

[0017] Preferably, step S2 includes the following:

[0018] S21. Based on the recalled template set, use ElasticSearch to retrieve detailed template data using inverted index technology;

[0019] S22. Based on multiple ranking factors such as relevance score, template value, top ranking value, and new product launch time, and with dynamically adjusted weights, input the comprehensive ranking model and output the comprehensive ranking score.

[0020] S23. The final recall template is obtained by sorting the recall template set according to the comprehensive ranking score.

[0021] The preferred comprehensive ranking model with dynamic weight adjustment is:

[0022]

[0023] Among them, score is the overall ranking score, w1, w2, w3, and w4 are dynamic weights that are adjusted according to user preferences and real-time data, similarity is the relevance score, template_value is the template value, stick_mark is the top value, and newness_factor is the freshness.

[0024] Preferably, the weights are trained using a machine learning model and updated in real time to adapt to different scenarios.

[0025] Preferably, in step S1, the multi-path recall strategy also includes using image feature-based visual recall, using ResNet50 to extract template thumbnail feature vectors, and building a visual index library, which is activated when the user uploads an image or selects an image search.

[0026] Preferably, in step S2, the ranking factors also include using user click-through rate (CTR) or template usage frequency as alternative ranking factors, and dynamically adjusting the weights.

[0027] A template search system based on multi-path recall, based on the aforementioned template search method based on multi-path recall, includes: a user data acquisition module, a vector database, a multi-path recall engine, a comprehensive ranking module, and a pre-computation module;

[0028] The user data acquisition module is used to acquire keywords input by the user and parse the user ID;

[0029] The multi-path recall engine is used to obtain a recall template set based on the keywords and user ID entered by the user, combined with a vector database and ElasticSearch, using a multi-path recall strategy. This includes three recall paths: keyword vectorization recall, historical behavior recall, and category tag recall. Detailed template data is retrieved based on the recall template set.

[0030] The comprehensive ranking module is used to introduce a comprehensive ranking model with dynamic weight adjustment. It combines multiple ranking factors to calculate a comprehensive ranking score, sorts the recalled templates, and returns them. The ranking factors include relevance score, template value, top ranking value, and new release time.

[0031] The end-to-end optimization module is used to precompute and cache vector representations of popular templates in the vector database, and ElasticSearch updates the index synchronously.

[0032] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the template search method based on multiple-way recall.

[0033] A processing terminal includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the template search method based on multiple-way recall.

[0034] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a template search method and device based on multi-path recall. It combines keyword vectorization recall, historical behavior recall and category tag recall in a multi-path recall mechanism, which significantly improves the recall coverage. It introduces a comprehensive ranking model with dynamic weight adjustment, which optimizes the weights of template value, top value and new release time in real time through machine learning, greatly improving the ranking accuracy. Through vector pre-computation and caching, it reduces query latency and ensures system efficiency. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0036] Figure 1 A schematic diagram of a template search method based on multi-path recall provided by the present invention;

[0037] Figure 2 This is a schematic diagram of the multi-path recall strategy provided by the present invention;

[0038] Figure 3 A schematic diagram of the sorting factor provided by this invention;

[0039] Figure 4 This is a schematic diagram illustrating the training of the comprehensive ranking model provided by the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] This invention discloses a template search method based on multi-path recall, such as... Figure 1 This includes the following steps:

[0042] S1. Receive the keywords and user ID input by the user, combine the vector database and ElasticSearch, and use a multi-path recall strategy to obtain the recall template set. The multi-path recall strategy includes three recall paths: keyword vectorization recall, historical behavior recall, and category tag recall.

[0043] S2. Retrieve detailed template data based on the recalled template set, introduce a comprehensive ranking model with dynamic weight adjustment, calculate a comprehensive ranking score by combining multiple ranking factors, and return the recalled templates after ranking. The ranking factors include relevance score, template value, top ranking value, and new release time.

[0044] S3. Precompute and cache vector representations of popular templates in a vector database, and update the index synchronously in Elasticsearch.

[0045] To further implement the above technical solutions, such as Figure 2 The specific content of step S1 is as follows:

[0046] S11. Using Natural Language Processing (NLP) technology, the keywords input by the user are converted into vector representations;

[0047] In this embodiment, a word embedding model, such as Word2Vec or BERT, is used to capture semantic information and convert the keywords input by the user into vector representations.

[0048] S12. Using the cosine similarity algorithm, calculate the similarity between the query vector and the template vector in the vector space, filter out templates with a similarity greater than a preset threshold from the vector database, and use them as the first template set; recall relevant templates based on the user ID's historical search behavior, and use them as the second template set; use the template's category tags to recall templates through ElasticSearch, and use them as the third template set.

[0049] S13. Merge the first template set, the second template set, and the third template set through intersection and union operations to expand the recall scope and obtain the recall template set.

[0050] In this embodiment, the specific method for recalling relevant templates based on the user ID's historical search behavior is as follows: establish a user behavior log table to record user behavior in real time to the log database; calculate the final interest score based on behavior weight and time decay factor to obtain a user profile calculated daily; create a user interest profile table to store the N templates with the highest interest for each user; query the user interest profile table based on the user ID to obtain the template ID list in its fields, sort them in descending order by interest score, and take the top K as the second template set.

[0051] To further implement the above technical solution, step S2 includes the following:

[0052] S21. Based on the recalled template set, use ElasticSearch to retrieve detailed template data using inverted index technology;

[0053] S22. Based on multiple ranking factors such as relevance score, template value, top ranking value, and new product launch time, and with dynamically adjusted weights, input the comprehensive ranking model and output the comprehensive ranking score.

[0054] S23. The final recall template is obtained by sorting the recall template set according to the comprehensive ranking score.

[0055] To further implement the above technical solution, the comprehensive ranking model with dynamic weight adjustment is as follows:

[0056]

[0057] Wherein, score is the overall ranking score, w1, w2, w3, and w4 are dynamic weights adjusted according to user preferences and real-time data, similarity is the relevance score, template_value is the template value, stick_mark is the top-ranked value, and newness_factor (also known as update_time) is the freshness factor. Figure 3 .

[0058] In this embodiment, the similarity score is calculated as follows:

[0059]

[0060] Where U is the user profile vector, which is generated daily by calculating the user's behavior history through the BERT model; T is the template feature vector; consine(U,T) is the cosine similarity between the user vector and the template vector; S is the original similarity set of all templates in the current recall set; and min(S) and max(S) are the minimum and maximum values ​​of the similarity set.

[0061] The template value score is calculated based on the number of clicks and favorites, including short-term value, medium-term value and long-term value;

[0062]

[0063]

[0064]

[0065]

[0066]

[0067]

[0068] Wherein, CTR and CVR represent short-term value, collection weight represents medium-term value, LV represents long-term value, µ represents the mean value of the platform template, σ represents the standard deviation of the platform template value, α is 0.4, β is 0.3, and γ is 0.1. This represents the number of clicks in the past 3 days. This refers to the number of times the content has been exposed in the past 3 days. This refers to the number of times the device has been used in the last 3 days. This refers to the number of times the item has been saved in the last 7 days. This refers to the number of times the device was used on day d.

[0069] The pinned value is:

[0070]

[0071] Where, Δt stick T is the difference between the current time and the time it was pinned, λ is the attenuation coefficient, and T is the time difference between the current time and the time it was pinned. max The pinned post is valid for a certain period of time;

[0072] The freshness factor is:

[0073]

[0074]

[0075]

[0076] Where k is the basic attenuation coefficient. Here, δ is the trend enhancement function, representing a trend enhancement term based on the template usage growth rate. δ is the platform comparison coefficient, used to preset the coefficient value based on the comparison between the template growth rate and the platform average to adjust the trend enhancement strength. current t represents the server's current timestamp. update G is the template's last update timestamp. 12h This refers to the usage of the template in the first 12 hours, G 24h This represents the usage amount of the template in the previous 24 hours.

[0077] To further implement the above technical solution, a machine learning model is used to train the weights, which are then updated in real time to adapt to different scenarios.

[0078] In this embodiment, the loss is calculated through forward propagation, the gradient is calculated through backpropagation, the optimizer updates the weights, and the process is iterated until convergence. Figure 4 ;

[0079] Dynamic weight generation is achieved using a lightweight neural network:

[0080]

[0081] Where x is the input feature vector, W1 is the first-layer weight matrix, b1 is the first-layer bias, W2 is the second-layer weight matrix, and b2 is the second-layer bias; the input features include user type encoding, current hour, day of the week, template category one-hot encoding, query keyword length, and system load coefficient (0-1).

[0082] The training loss function uses click-rate negative log-likelihood loss:

[0083]

[0084]

[0085] Where yi represents whether the user clicks the i-th template, pi is the predicted click probability, N is the number of templates, and score i Let be the overall score of the i-th template.

[0086] To further implement the above technical solution, in step S1, the multi-path recall strategy also includes using image feature-based visual recall, using ResNet50 to extract template thumbnail feature vectors, and establishing a visual index library, which is activated when the user uploads an image or selects an image search.

[0087] To further implement the above technical solution, in step S2, the ranking factors also include using user click-through rate (CTR) or template usage frequency as alternative ranking factors, and dynamically adjusting the weights.

[0088] A template search system based on multi-path recall, based on a template search method based on multi-path recall, includes: a user data acquisition module, a vector database, a multi-path recall engine, a comprehensive ranking module, and a pre-computation module;

[0089] The user data acquisition module is used to acquire keywords input by the user and parse the user ID;

[0090] The multi-path recall engine is used to obtain a recall template set based on the keywords and user ID entered by the user, combined with a vector database and ElasticSearch, using a multi-path recall strategy. This includes three recall paths: keyword vectorization recall, historical behavior recall, and category tag recall. Detailed template data is retrieved based on the recall template set.

[0091] The comprehensive ranking module is used to introduce a comprehensive ranking model with dynamic weight adjustment. It combines multiple ranking factors to calculate a comprehensive ranking score, sorts the recalled templates, and returns them. The ranking factors include relevance score, template value, top ranking value, and new release time.

[0092] The end-to-end optimization module is used to precompute and cache vector representations of popular templates in the vector database, and ElasticSearch updates the index synchronously.

[0093] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements a template search method based on multiple-way recall.

[0094] A processing terminal includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements a template search method based on multiple-way recall.

[0095] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0096] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A template search method based on multi-path recall, characterized in that, Includes the following steps: S1. Receive the keywords and user ID input by the user, combine the vector database and ElasticSearch, and use a multi-path recall strategy to obtain the recall template set. The multi-path recall strategy includes three recall paths: keyword vectorization recall, historical behavior recall, and category tag recall. S2. Retrieve detailed template data based on the recalled template set, introduce a comprehensive ranking model with dynamic weight adjustment, calculate a comprehensive ranking score by combining multiple ranking factors, and return the recalled templates after ranking. The ranking factors include relevance score, template value, top ranking value, and new release time. S3. Precompute and cache vector representations of popular templates in a vector database, and update the index synchronously in Elasticsearch.

2. The template search method based on multi-path recall according to claim 1, characterized in that, The specific content of step S1 is as follows: S11. Using Natural Language Processing (NLP) technology, the keywords input by the user are converted into vector representations; S12. Using the cosine similarity algorithm, calculate the similarity between the query vector and the template vector in the vector space, filter out templates with a similarity greater than a preset threshold from the vector database, and use them as the first template set; recall relevant templates based on the user ID's historical search behavior, and use them as the second template set; use the template's category tags to recall templates through ElasticSearch, and use them as the third template set. S13. Merge the first template set, the second template set, and the third template set through intersection and union operations to expand the recall scope and obtain the recall template set.

3. The template search method based on multi-path recall according to claim 1, characterized in that, The specific content of step S2 includes: S21. Based on the recalled template set, use ElasticSearch to retrieve detailed template data using inverted index technology; S22. Based on multiple ranking factors such as relevance score, template value, top ranking value, and new product launch time, and with dynamically adjusted weights, input the comprehensive ranking model and output the comprehensive ranking score. S23. The final recall template is obtained by sorting the recall template set according to the comprehensive ranking score.

4. The template search method based on multi-path recall according to claim 1, characterized in that, The comprehensive ranking model with dynamic weight adjustment is as follows: ; Among them, score is the overall ranking score, w1, w2, w3, and w4 are dynamic weights that are adjusted according to user preferences and real-time data, similarity is the relevance score, template_value is the template value, stick_mark is the top value, and newness_factor is the freshness.

5. The template search method based on multi-path recall according to claim 3, characterized in that, The weights are trained using a machine learning model and updated in real time to adapt to different scenarios.

6. The template search method based on multi-path recall according to claim 1, characterized in that, In step S1, the multi-path recall strategy also includes using image feature-based visual recall, using ResNet50 to extract template thumbnail feature vectors, and building a visual index library, which is activated when a user uploads an image or selects an image search.

7. The template search method based on multi-path recall according to claim 1, characterized in that, In step S2, the ranking factors also include using user click-through rate (CTR) or template usage frequency as alternative ranking factors, and dynamically adjusting the weights.

8. A template search system based on multi-path recall, characterized in that, A template search method based on multi-path recall according to any one of claims 1-7 includes: a user data acquisition module, a vector database, a multi-path recall engine, a comprehensive ranking module, and a pre-computation module; The user data acquisition module is used to acquire keywords input by the user and parse the user ID; The multi-path recall engine is used to obtain a recall template set based on the keywords and user ID entered by the user, combined with a vector database and ElasticSearch, using a multi-path recall strategy. This includes three recall paths: keyword vectorization recall, historical behavior recall, and category tag recall. Detailed template data is retrieved based on the recall template set. The comprehensive ranking module is used to introduce a comprehensive ranking model with dynamic weight adjustment. It combines multiple ranking factors to calculate a comprehensive ranking score, sorts the recalled templates, and returns them. The ranking factors include relevance score, template value, top ranking value, and new release time. The end-to-end optimization module is used to precompute and cache vector representations of popular templates in the vector database, and ElasticSearch updates the index synchronously.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements a template search method based on multiple-way recall as described in any one of claims 1-7.

10. A processing terminal, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes a computer program, it implements a template search method based on multiple-way recall as described in any one of claims 1-7.

Images