Advertisement-based training method, device and equipment for generative recall large model

Abstract

The invention provides a training method, device and equipment for an advertisement-based generative recall large model. The invention relates to the technical field of artificial intelligence, in particular to the technical fields of deep learning, large models, natural language processing, knowledge maps, data analysis and the like, and can be used for application scenes such as keyword recall, advertisement retrieval, advertisement putting and the like. According to the specific scheme, an existing training data set is used for supervising and training a model, and an initial large model capable of mapping keywords to corresponding identifier sequences is constructed; and on the basis of target object click behavior data in the advertisement recall scene, fine tuning is performed on the initial large model to obtain a generative recall large model, and the generative recall large model is used for generating an identifier sequence based on the keyword. According to the scheme of the invention, the keyword is mapped to the identifier sequence, and the initial large model is finely adjusted based on the click behavior data of the target object in the advertisement recall scene, so that the advertisement recall capability can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, especially to technical fields such as deep learning, large models, natural language processing, knowledge graphs, and data analysis. It can be used in application scenarios such as keyword recall, advertisement retrieval, and advertisement delivery, and specifically relates to a training method, device, and equipment for a generative recall large model based on advertisements. Background Art

[0002] In the current advertising system, each advertiser will input their own keywords and advertising creatives. Existing advertising recall technology based on large models generally recalls advertisements based on keywords generated by generative recall large models. However, the keyword set generated by generative recall large models based on query words is limited, resulting in low recall capability. Summary of the invention

[0003] The present disclosure provides a method, device and equipment for training a large generative recall model based on advertising.

[0004] According to the first aspect of the present disclosure, a training method for a generative recall big model based on advertising is provided, the method comprising: using an existing training data set to perform supervised training on the model to construct an initial big model that can map keywords to corresponding identifier sequences; based on the click behavior data of the target object in the advertising recall scenario, the initial big model is fine-tuned to obtain a generative recall big model, and the generative recall big model is used to generate an identifier sequence according to the query term.

[0005] According to a second aspect of the present disclosure, there is provided an advertisement display method, the method comprising: inputting a received query word into a generative recall large model to obtain an identifier sequence output by the generative recall large model; wherein the generative recall large model is trained by the method described in the first aspect; based on the identifier sequence, multiple advertisements are pulled from an advertisement library to obtain an advertisement set; using a double-tower model to filter out at least one target advertisement matching the query word from the advertisement set; and displaying at least one target advertisement.

[0006] According to a third aspect of the present disclosure, a training device for a generative recall big model based on advertising is provided, and the device includes: a training module, which is used to use an existing training data set to perform supervised training on the model, and construct an initial big model that can map keywords to corresponding identifier sequences; a fine-tuning module, which is used to fine-tune the initial big model based on the target object click behavior data in the advertising recall scenario, and obtain the generative recall big model, which is used to generate an identifier sequence according to the query word.

[0007] According to a fourth aspect of the present disclosure, an advertisement display device is provided, which includes: an acquisition module, used to input a received query word into a generative recall large model to obtain an identifier sequence output by the generative recall large model; wherein the generative recall large model is obtained by training the method described in the first aspect; an advertisement pulling module, used to pull multiple advertisements from an advertisement library based on the identifier sequence to obtain an advertisement set; an advertisement screening module, used to use a double-tower model to screen out at least one target advertisement matching the query word from the advertisement set; and an advertisement display module, used to display at least one target advertisement.

[0008] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any method in the embodiments of the present disclosure.

[0009] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute any method according to the embodiments of the present disclosure.

[0010] According to a seventh aspect of the present disclosure, a computer program product is provided, including a computer program stored on a storage medium, and when the computer program is executed by a processor, the method according to any embodiment of the present disclosure is implemented.

[0011] By adopting the solution of the present disclosure, the advertisement recall capability can be improved by mapping keywords to identifier sequences and fine-tuning the initial large model based on the target object click behavior data in the advertisement recall scenario.

[0012] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0014] Figure 1 is a flowchart of a method for training a large generative recall model based on advertising according to an embodiment of the present disclosure;

[0015] Figure 2 is a schematic diagram of hierarchical clustering of keywords according to an embodiment of the present disclosure;

[0016] Figure 3is a schematic diagram of implementing keyword identifier serialization based on a clustering algorithm according to an embodiment of the present disclosure;

[0017] Figure 4 is a flowchart of an advertisement display method according to an embodiment of the present disclosure;

[0018] Figure 5 is a flowchart of a generative recall large model reasoning process according to an embodiment of the present disclosure;

[0019] Figure 6 is a flowchart of a recall process of end-to-end verification according to an embodiment of the present disclosure;

[0020] Figure 7 is a structural schematic diagram of a training device for a generative recall large model based on advertising according to an embodiment of the present disclosure;

[0021] Figure 8 is a structural schematic diagram of an advertising display device according to an embodiment of the present disclosure;

[0022] Fig. 9 is a scenario diagram of a training method for a generative recall large model based on advertising according to an embodiment of the present disclosure;

[0023] Fig.10 is a schematic diagram of a scenario of an advertisement display method according to an embodiment of the present disclosure;

[0024] Fig.11 It is a structural schematic diagram of an electronic device used to implement the training method of the advertisement-based generative recall large model and / or the advertisement display method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0026] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0027] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0028] Before introducing the technical solutions of the embodiments of the present disclosure, the technical terms that may be used in the present disclosure are further explained:

[0029] Generative Recall Large Model: A pre-trained model with powerful analytical capabilities that can generate a sequence of identifiers based on keywords, where each identifier corresponds to multiple keywords.

[0030] In the related art, there are three roles in commercial search scenarios: users, advertisers, and commercial search engines. Among them, users are responsible for entering query words, advertisers are responsible for entering a combination of keywords and advertising creatives, and commercial search engines are responsible for matching keywords according to query words, and the advertising creatives corresponding to the keywords will be displayed in the user's search results page. The generative recall task takes the query text as input, uses the generative recall large model to generate multiple keyword candidates, and pulls advertisements through keywords to achieve targeted advertising. In the current advertising system, the generative recall large model is limited by resources and unique reasoning paradigms, and the generated keyword set is limited, resulting in low recall capabilities; at the same time, the large model occupies a large amount of server resources, the reasoning time is high, and it is easy to cause system performance to deteriorate; in addition, this recall method lacks end-to-end control from query words to advertisements.

[0031] In order to at least partially solve one or more of the above problems and other potential problems, the present disclosure proposes an advertisement display method, which enables the initial large model to map keywords to identifier sequences through self-learning and model evolution, and fine-tunes the initial large model based on the target object click behavior data in the advertisement recall scenario to obtain a generative recall large model, which solves the problems of low recall capability, high system resource consumption, and lack of end-to-end control in the advertisement system in the commercial search scenario, and helps to improve the advertisement recall capability of the commercial search system. At the same time, by improving the recall method, not only can the recall speed be accelerated and the retrieval time be reduced, but also the user's advertising experience can be improved.

[0032] The disclosed embodiment provides a method for training a large generative recall model based on advertisements. Figure 1 It is a flow chart of a method for training a generative recall large model based on advertising according to an embodiment of the present disclosure. The method for training a generative recall large model based on advertising can be applied to a training device for a generative recall large model based on advertising. The training device for a generative recall large model based on advertising is located in an electronic device. The electronic device includes but is not limited to fixed devices and / or mobile devices. For example, fixed devices include but are not limited to servers, and servers can be cloud servers or ordinary servers. For example, mobile devices include but are not limited to: mobile phones, tablet computers. In some possible implementations, the method for training a generative recall large model based on advertising can also be implemented by a processor calling computer-readable instructions stored in a memory. For example Figure 1 As shown, the training method of the advertising-based generative recall large model includes:

[0033] S101: Use the existing training data set to supervise the model and build an initial large model that can map keywords to corresponding identifier sequences;

[0034] S102: Based on the target object click behavior data in the advertisement recall scenario, the initial large model is fine-tuned to obtain a generative recall large model, where the generative recall large model is used to generate an identifier sequence according to the query term.

[0035] In the disclosed embodiment, supervised learning is a basic method in machine learning, which can train a model so that the model can learn how to predict or determine the output of unseen data, that is, predict an output identifier sequence based on given keywords.

[0036] In the disclosed embodiment, the known training data set is a pre-labeled data set used to train the data set of the model to be trained, including keywords and their corresponding identifier sequences. In the disclosed embodiment, for supervised learning tasks, a manual labeling method can be used to pre-match keywords with identifier sequences to ensure that each keyword has a correct identifier sequence, thereby obtaining a known training data set. The above is only an exemplary description and is not intended to limit all possible situations for obtaining a known training data set, but it is not exhaustive here.

[0037] In the embodiments of the present disclosure, keywords refer to specific words or phrases that can trigger the display of advertisements. Keywords are the core of the search engine advertising system, and keywords are provided by advertisers. The user's search intent can be matched with the advertiser's advertising content. Exemplarily, keywords are matched according to the query terms input by the user, and the generation method can select a generative recall method or a sequential reasoning method, etc., or other methods in the prior art can be used for generation. The above is only an exemplary description and is not intended to limit all possible situations for obtaining keywords, but it is not exhaustive here.

[0038] In the disclosed embodiment, the identifier sequence is a series of keyword identifiers with semantic hierarchical structure information. The keyword identifier is an identifier with semantic information obtained by mapping the acquired keyword plain text. The number of tokens of the keyword identifier is significantly smaller than the keyword plain text. Here, a token can be a word, a character, or a word or phrase after word segmentation.

[0039] In the embodiments of the present disclosure, the ad recall scenario refers to the process in which, when a user searches, a search engine or an advertising platform quickly selects advertisements related to the user's search intent from a massive advertising inventory and displays these advertisements to the user. This scenario involves the matching, sorting, and display of advertisements, with the goal of improving the relevance and effectiveness of advertisements while ensuring the user experience. In the embodiments of the present disclosure, the ad recall scenario mainly includes user intent identification, ad matching, relevance assessment, ad sorting, ad display, and effect tracking. The above is only an illustrative description and is not intended to limit all possible situations of the ad recall scenario, except that it is not exhaustive here.

[0040] In the disclosed embodiment, the target object click behavior data refers to the data generated by the click behavior of the user when facing the advertisement, which can be used to evaluate the relevance of the advertisement, the user's interest preference and the advertising effect. In the disclosed embodiment, the target object click behavior data may include query words, click-through rate, conversion rate and other data. The above is only an exemplary description and is not intended to limit all possible situations of the target object click behavior data. It is just not exhaustive here.

[0041] In some embodiments, a large number of keywords and their corresponding identifier sequences are collected as training data sets. These data can come from existing databases, user behavior logs, web crawlers, etc. The data is cleaned and preprocessed to remove noise and outliers to ensure the accuracy and consistency of the data.

[0042] In some embodiments, a model architecture suitable for supervised learning is selected, such as a decision tree, random forest, gradient boosting decision tree or deep learning model. These models can learn the mapping relationship from keywords to identifier sequences based on a training data set. The model is trained using the training data set, and the value of the loss function is minimized by continuously adjusting the model parameters to obtain an initial large model.

[0043] In some embodiments, fine-tuning the generative recall model based on the advertisement recall scenario includes: collecting click behavior data of the target object in the advertisement recall scenario, including the user's click, browse, favorite and other behaviors on the advertisement and related keyword information. The collected click behavior data is used to fine-tune the initial large model to make it more suitable for the needs of the advertisement recall scenario. In the fine-tuning process, some regularization methods, early stopping strategies, etc. can be introduced to prevent the model from overfitting, and at the same time, appropriate optimization algorithms, learning rates, etc. can be selected to improve the training efficiency and effect.

[0044] The technical solution of the disclosed embodiment uses an existing training data set to supervise the model training, builds an initial large model that can map keywords to corresponding identifier sequences; based on the target object click behavior data in the advertisement recall scenario, fine-tunes the initial large model to obtain a generative recall large model. In this way, by mapping keywords to identifier sequences and fine-tuning the initial large model based on the target object click behavior data in the advertisement recall scenario, it helps to improve the advertisement recall capability of the commercial search system, optimize system resource consumption, and improve the conversion rate of advertisements.

[0045] In some embodiments, based on the click behavior data of the target object in the advertising recall scenario, the initial large model is fine-tuned to obtain a generative recall large model, including: extracting query terms and keywords based on the click behavior data of the target object in the advertising recall scenario; mapping the keywords into identifier sequences; and fine-tuning the initial large model according to the query terms and identifier sequences to obtain a generative recall large model.

[0046] In the disclosed embodiment, the process of extracting query terms and keywords based on the click behavior data of the target object in the advertisement recall scenario can be divided into a query term extraction process and a keyword extraction process. Since the user's query terms are usually short, the core information of the term or entity can be extracted through information extraction technology to realize the query term extraction process. The keyword extraction process can generally use a model to automatically mine keywords, or use index mining. For the automatic mining method of the model, the mined keywords can also be sorted using the <query term, keyword> correlation model to generate more accurate keywords. For the index mining method, knowledge graphs and various relationship mining can be used to establish relationships such as entities and labels, mine knowledge, and generate keywords.

[0047] In the disclosed embodiment, in the process of mapping keywords to identifier sequences, a manual annotation method may be used to pre-match keywords with identifier sequences to ensure that each keyword has a correct identifier sequence, thereby achieving the mapping of keywords to identifier sequences.

[0048] In the disclosed embodiment, the initial large model is fine-tuned according to the query words and identifier sequence to obtain the generative recall large model. The query words and identifier sequence can be used as the validation set of the model, the query words and identifier sequence are input into the initial large model, and the hyperparameters of the initial large model are adjusted and optimized according to the output of the initial large model. At the same time, overfitting and underfitting can be prevented by regularization, increasing the amount of data, reducing the complexity of the model, and other methods.

[0049] In some embodiments, query words are extracted from the preprocessed data, and these query words can be search words entered by the user, advertisement titles, advertisement descriptions, etc. Keywords are extracted from the advertisement content, and these keywords can be product names, brands, functional descriptions, etc. in the advertisement. The extracted keywords are mapped to keyword identifier sequences by encoding the keywords, such as using hash functions or dictionary mapping. The processed query words and identifier sequence pairs are used as training data to fine-tune the initial large model. During the fine-tuning process, the performance of the model on the advertisement recall task is optimized by adjusting the model weights and parameters. Appropriate fine-tuning parameters, such as learning rate, batch size, training rounds, etc., are set to ensure the fine-tuning effect. The fine-tuned model is evaluated using a validation set to measure the performance of the model on the advertisement recall task. The parameters and structure of the model are adjusted according to the evaluation results to improve the generalization ability and accuracy of the model. The model is continuously optimized and iteratively updated according to user feedback and needs.

[0050] The technical solution of the disclosed embodiment extracts query words and keywords based on the click behavior data of the target object in the advertisement recall scenario; maps the keywords to an identifier sequence; and fine-tunes the initial large model according to the query words and the identifier sequence to obtain a generative recall large model. In this way, by fine-tuning the initial large model with the query words and the identifier sequence, it can help improve the advertisement recall capability of the commercial search system.

[0051] In some embodiments, each identifier corresponds to multiple semantically similar keywords.

[0052] In the disclosed embodiment, a plurality of semantically similar keywords are mapped to the same identifier. In this case, one identifier is a set of a plurality of semantically similar keywords.

[0053] In the technical solution of the embodiment of the present disclosure, each identifier corresponds to multiple keywords with similar semantics. Thus, through a one-to-many mapping method, each identifier can correspond to more relevant advertisements, thereby significantly improving the advertisement recall capability.

[0054] In some embodiments, the training method of the advertising-based generative recall large model also includes: hierarchical clustering of keywords to obtain clustering results; constructing a clustering tree based on the clustering results; constructing a mapping relationship from keywords to identifier sequences based on the clustering tree; wherein the mapping relationship is used for supervised training to obtain an initial large model.

[0055] In the embodiments of the present disclosure, hierarchical clustering refers to a cluster analysis method that demonstrates the similarity between data points by constructing a cluster tree composed of a hierarchical structure. Hierarchical clustering can intuitively demonstrate the hierarchical relationship between data points.

[0056] In the disclosed embodiment, the cluster tree is a tree-structured chart used to display the results of hierarchical clustering, which intuitively represents the similarity or distance between data points and how they are aggregated into clusters of different levels.

[0057] In some implementations, the encoder obtained by contrastive learning may be used to first obtain a dense vector of the advertisement, and then a hierarchical clustering method may be used to obtain a sequence of identifiers with a hierarchical relationship.

[0058] The technical solution of the disclosed embodiment utilizes a hierarchical clustering method and can be flexibly applied to data sets of different sizes. After building a clustering tree, new data points can be flexibly added to facilitate later maintenance and optimize system resource consumption.

[0059] In some embodiments, a mapping relationship from keyword mapping to identifier sequence is constructed based on a clustering tree, including: determining the number of layers of the clustering tree and the number of nodes in each layer, wherein, in the clustering tree, each leaf node corresponds to multiple semantically similar keywords; and determining the path from the root node to the leaf node as an identifier sequence.

[0060] In the disclosed embodiment, in the clustering tree constructed by hierarchical clustering, each leaf node corresponds to a group of semantically similar keywords, and the path from the root node to the leaf node is the identifier sequence.

[0061] In some embodiments, the structure of hierarchical clustering, including the number of layers and the number of nodes in each layer, can be customized according to needs, and a smaller number of layers (i.e., the number of tokens of keyword identifiers) can be customized to obtain resource consumption and reasoning time that is better than keyword plain text. At the same time, the overall structure is designed as a "wide at the front and narrow at the back" format, so that the speculative decoding strategy can be introduced in the reasoning stage to further improve performance.

[0062] Figure 2 A schematic diagram of hierarchical clustering of keywords is shown in FIG. Figure 2 As shown in FIG. 1 , keywords are extracted from the advertisement library, and a dense vector of the keyword is obtained using an encoder obtained by contrastive learning. The dense vector is discretized by hierarchical clustering to obtain an identifier sequence corresponding to the keyword.

[0063] Figure 3 A schematic diagram of implementing keyword identifier serialization based on a clustering algorithm is shown, Figure 3 As shown in the figure, first, the keyword plaintext is mapped into a dense vector V using an encoder; then, the vector V is clustered to obtain a codebook 0 consisting of n cluster center vectors. At this time, if V belongs to V01, the first-layer encoding result is L01; then, the residual vector of the cluster center is calculated, and the previous step is repeated to obtain the second-layer encoding L12; the above process of calculating the residual and clustering is repeated until the number of layers meets the requirements, and finally the keyword plaintext is quantized to [L01, L12, ..., L43].

[0064] The technical solution of the embodiment of the present disclosure can significantly reduce the time consumption of reasoning and the required server resources, and optimize the time consumption of system resources, because the length of the identifier sequence is obviously smaller than the keyword plaintext.

[0065] The present disclosure provides an advertisement display method. Figure 4: is a flowchart of an advertisement display method according to an embodiment of the present disclosure. The advertisement display method can be applied to electronic devices, including but not limited to fixed devices and / or mobile devices. For example, fixed devices include but are not limited to servers, and servers can be cloud servers or ordinary servers. For example, mobile devices include but are not limited to: mobile phones, tablet computers, vehicle-mounted devices, personal computers, etc. In some possible implementations, the advertisement display method can also be implemented by a processor calling computer-readable instructions stored in a memory. Figure 4 As shown, the advertisement display method includes:

[0066] S401: Input the received query word into the generative recall model to obtain an identifier sequence output by the generative recall model;

[0067] S402: Pull multiple advertisements from the advertisement library based on the identifier sequence to obtain an advertisement set;

[0068] S403: Filtering out at least one target advertisement matching the query term from the advertisement collection using the double tower model;

[0069] S404: Display at least one target advertisement.

[0070] In the disclosed embodiment, the advertisement library refers to a database storing advertisement information, including information such as the category, theme, and content of the advertisement.

[0071] In the disclosed embodiment, the twin tower model is a machine learning model widely used in the fields of recommendation systems and information retrieval. Its core idea is to encode the features of two different fields into a vector respectively, and then perform matching and recall by calculating the similarity between the two vectors.

[0072] In some embodiments, generating keywords based on received query words includes: performing word segmentation processing on the received query words and extracting core words therein as keywords. For example, if the query word is "new smartphones in 2024", the keywords obtained after word segmentation may be "2024", "new models", and "smartphones". In this way, key information in the query words can be extracted quickly and accurately, providing a basis for subsequent advertisement matching.

[0073] In some embodiments, generating keywords based on received query words includes: using a semantic understanding model in natural language processing (NLP) technology to perform semantic analysis on the query words and extract keywords related to the query intent. For example, if the query word is "I want to buy a laptop suitable for office use", the semantic understanding model can extract "buy", "office", and "laptop" as keywords. In this way, the user's query intent can be understood more deeply, thereby improving the accuracy of advertisement matching.

[0074] Figure 5 The figure shows a flow chart of the generative recall large model reasoning process. Figure 5 As shown, first, keywords are generated based on the query words; then the keywords are input into the generative recall large model, and the generative recall large model performs inference to obtain the identifier sequence corresponding to the keyword. Since the length of the identifier sequence (the number of tokens) is significantly smaller than the keyword plaintext, the inference time is reduced and the required resources are also significantly reduced. In addition, since the hierarchical structure of the identifier sequence is known and designed as a "wide at the front and narrow at the back" structure, combining speculative decoding can further reduce the inference time. As the decoding depth increases, the size of the candidate subtree shrinks rapidly. Based on the speculative decoding strategy, the entire tree can be directly "verified" when the tree size is small, and the remaining tokens of the entire length can be decoded by a decoder once, and performance benefits can be obtained by reducing the number of decoder calls.

[0075] In some embodiments, pulling multiple advertisements from the advertisement library based on the identifier sequence to obtain an advertisement set corresponding to the query term includes: performing a matching query in the advertisement library based on the generated identifier sequence to pull out advertisements corresponding to the identifier sequence. For example, if the identifier sequence includes "[tourism category ID _ 1, tourist attraction tag 1]", the pulled advertisements may include introductions and discount information about specific tourist attractions, and all advertisements belonging to this category are pulled from the advertisement library. In this way, it can be ensured that the pulled advertisements are highly relevant to the query terms and the generated identifier sequence.

[0076] In some implementations, multiple advertisements are pulled from the advertisement library based on the identifier sequence to obtain an advertisement set corresponding to the query term, including: using an index mechanism in the advertisement library to quickly locate relevant advertisements based on the identifier sequence. The index mechanism may be an index based on a hash table, a search based on an inverted index, etc. The index mechanism based on the hash table is used to locate the relevant advertisements based on the identifier "tourism category ID _ 1" to quickly locate all ads belonging to this category. This can significantly improve the efficiency of ad retrieval and reduce query time.

[0077] In the disclosed embodiment, the dual-tower model is a deep learning model architecture for text matching, which consists of two independent neural networks (i.e., a query tower and an advertising tower).

[0078] In the disclosed embodiment, all advertisements in the advertisement set are sorted based on the double tower model, and target advertisements can be screened out according to the sorting result.

[0079] In some embodiments, the twin tower model is used to filter out at least one target advertisement that matches the query term from the advertisement collection, including: using the query tower in the twin tower model to encode the query term, using the advertisement tower to encode the advertisement, and then filtering out the target advertisement by calculating the similarity between the encoded vectors of the two. Advertisements with high similarity are selected as target advertisements. In this way, the twin tower model can capture the semantic relationship between the query term and the advertisement, thereby improving the accuracy of the match. For example, the query term is "latest mobile phone" and the advertisement is "XXX new product on the market". The twin tower model calculates the similarity of the encoded vectors of the two. If the similarity is high, the advertisement is selected as the target advertisement.

[0080] In some embodiments, the twin tower model is used to filter out at least one target advertisement that matches the query term from the advertisement set, including: introducing an attention mechanism based on the twin tower model so that the model can pay more attention to key information in the query term and the advertisement. For example, a multi-head attention mechanism is used to capture the complex relationship between the two. In this way, the introduction of the attention mechanism can improve the twin tower model's ability to capture key information and further improve the accuracy of matching. For example, the query term is "home fitness equipment" and the advertisement is "YYY home treadmill". After introducing the attention mechanism, the twin tower model can more accurately capture the key information between the two, such as "home" and "fitness equipment", thereby improving the accuracy of matching.

[0081] In some embodiments, displaying at least one target advertisement includes: directly displaying the filtered target advertisement in the user interface, which may be a web page, a pop-up window, etc. The display method may be in the form of a list, a carousel, etc. In this way, directly displaying the target advertisement enables the user to quickly obtain the advertising information related to the query term. In some embodiments, displaying at least one target advertisement includes: while displaying the target advertisement, providing user interaction functions, such as jumping to the ad details page after clicking the advertisement, adding the advertisement to the shopping cart, etc. In the search results page of the e-commerce platform, the target advertisement related to the query term "new mobile phone" is displayed, and a function of jumping to the mobile phone details page after clicking is provided. In this way, providing the user interaction function can enhance the user experience and improve the conversion rate of the advertisement. The above is only an exemplary description and is not intended to limit all possible situations for displaying the target advertisement, but it is not exhaustive here.

[0082] In some implementations, advertisements may be pre-assigned to identifier sequences and an advertisement index may be constructed. When advertisements need to be pulled, advertisements associated with the identifier sequence may be retrieved from the advertisement index based on the identifier sequence, thereby pulling multiple advertisements from the advertisement library. Exemplarily, multiple advertisements may be pulled from the advertisement library based on the identifier sequence, and methods such as recall matching and vector indexing in the prior art may also be used. The above is only an exemplary description and is not intended to limit all possible situations for pulling multiple advertisements from the advertisement library, but it is not exhaustive here.

[0083] The technical solution of the disclosed embodiment, by converting query words into keywords and using a generative recall large model to generate an identifier sequence corresponding to the keywords, can efficiently pull a set of advertisements that are highly relevant to the query words from the advertisement library, thereby reducing the time complexity of advertisement matching and improving the real-time nature of advertisement recommendations. The query words and advertisements are encoded using a twin-tower model, and the target advertisements are screened out by calculating the vector similarity of the two after encoding. This can capture the semantic relationship between the query words and the advertisements, thereby ensuring that the advertisements pushed to the user are not only literally related to the query words, but also highly matched at the semantic level, effectively improving the advertisement recall capability. By introducing the twin-tower model, end-to-end control from query words to advertisements is achieved for the two-stage recall strategy, so that the recalled advertisements can more accurately meet the needs of users.

[0084] In some embodiments, the method of filtering out at least one target advertisement from an advertisement set based on the twin-tower model includes: inputting a query term and an advertisement landing page of each advertisement in the advertisement set into the twin-tower model; obtaining a relevance or an expected click-through rate between the query term output by the twin-tower model and each advertisement landing page; and selecting an advertisement having a relevance greater than a first threshold or an expected click-through rate greater than a second threshold as at least one target advertisement.

[0085] In the disclosed embodiment, the dual-tower model includes two main parts: a query word tower and an ad landing page tower. The query word tower receives features related to the query word, performs feature learning and representation learning through a multi-layer neural network, and finally outputs a low-dimensional semantic vector of the query word. In addition, the ad landing page tower processes features related to the ad landing page, including user behavior and advertising effects, such as page visits, dwell time, conversion rate, etc., and also processes them through a neural network to output a low-dimensional semantic vector of the landing page. Then, the low-dimensional semantic vectors of the query word and the low-dimensional semantic vectors of the landing page output by the two towers are calculated to obtain the relevance or expected click-through rate.

[0086] In the disclosed embodiment, each advertisement in the advertisement set can be sorted according to relevance or expected click-through rate and compared with a first threshold and / or a second threshold, so that advertisements having a relevance greater than the first threshold or an expected click-through rate greater than the second threshold are selected as target advertisements.

[0087] The technical solution of the disclosed embodiment, based on the screening mechanism of the double tower model, can ensure that only advertisements highly relevant to the query words are displayed to users, thereby reducing the interference of irrelevant advertisements and improving the user's browsing experience. Accurate matching and recommendation can increase users' interest in advertisements and their willingness to click, thereby improving the click-through rate and conversion rate of advertisements.

[0088] Figure 6 A flow chart of the recall process of end-to-end verification is shown, as Figure 6 As shown in the figure, the query word is input into the generative recall model to obtain the identifier sequence, and then the advertisement set is obtained from the advertisement library according to the identifier sequence. Finally, the query word and the landing page of each advertisement in the advertisement set are input into the double tower model, and at least one target advertisement can be selected from the advertisement set. After such processing, not only can relevant advertisements be recalled according to the query word, but also the recalled advertisements can be optimized in quality to ensure that the recalled advertisements can better meet the needs of users.

[0089] It should be understood that Figures 2 to 6 The schematic diagram shown is only exemplary and not restrictive, and it is scalable, and those skilled in the art can Figures 2 to 6 Various obvious changes and / or substitutions can be made to the examples, and the resulting technical solutions still fall within the scope of the disclosure of the embodiments of the present disclosure.

[0090] The present disclosure provides a training device 700 for a generative recall model based on advertisements, such as Figure 7 As shown, the device may include: a training module 701, which is used to use the existing training data set to supervise the model training and build an initial large model that can map keywords to corresponding identifier sequences; a fine-tuning module 702, which is used to fine-tune the initial large model based on the target object click behavior data in the advertising recall scenario to obtain a generative recall large model, and the generative recall large model is used to generate an identifier sequence according to the query word.

[0091] In some embodiments, the fine-tuning module 702 includes:

[0092] An extraction submodule, used to extract query words and keywords based on the target object click behavior data;

[0093] A mapping submodule, used for mapping the keyword to the identifier sequence;

[0094] A generation submodule is used to fine-tune the initial large model according to the query word and the identifier sequence to obtain the generative recall large model.

[0095] In some embodiments, in the identifier sequence, each identifier corresponds to a plurality of semantically similar keywords.

[0096] In some embodiments, the training device of the generative recall large model further includes:

[0097] A clustering module, used for performing hierarchical clustering on the keywords to obtain clustering results;

[0098] A first construction module, used to construct a clustering tree according to the clustering result;

[0099] The second construction module is used to construct a mapping relationship from keywords to identifier sequences based on the clustering tree; wherein the mapping relationship is used for supervised training to obtain an initial large model.

[0100] In some embodiments, the second building block is used to:

[0101] Determine the number of layers of the clustering tree and the number of nodes in each layer, wherein each leaf node in the clustering tree corresponds to a plurality of keywords with similar semantics;

[0102] The path from the root node to the leaf node is determined as the identifier sequence.

[0103] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0104] The training device of the generative recall large model based on advertising in the embodiment of the present disclosure helps to improve the advertising recall capability of the commercial search system, optimize system resource consumption, and enhance the user's advertising experience by mapping keywords to identifier sequences and fine-tuning the initial large model based on the target object click behavior data in the advertising recall scenario.

[0105] The present disclosure provides an advertisement display device 800. Figure 8 As shown, the device may include: an acquisition module 801, used to input the received query word into the generative recall large model to obtain an identifier sequence output by the generative recall large model; an advertisement pulling module 802, used to pull multiple advertisements from the advertisement library based on the query word and the identifier sequence to obtain an advertisement set; an advertisement screening module 803, used to use the double tower model to screen out at least one target advertisement matching the query word from the advertisement set; an advertisement display module 804, used to display at least one target advertisement.

[0106] In some embodiments, the advertisement screening module 803 includes: an input submodule, used to input the query term and the advertisement landing page of each advertisement in the advertisement set into the dual-tower model; an acquisition submodule, used to acquire the relevance or expected click-through rate between the query term and each advertisement landing page output by the dual-tower model; and a screening submodule, used to select the advertisement whose relevance is greater than a first threshold or whose expected click-through rate is greater than a second threshold as the at least one target advertisement.

[0107] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0108] The advertising display device of the embodiment of the present disclosure can significantly reduce server resource overhead and optimize system resource consumption because the length of the identifier sequence is significantly smaller than the keyword plaintext. In addition, by introducing the dual-tower model, end-to-end control from query terms to advertisements is achieved for the two-stage recall strategy, so that the recalled advertisements can more accurately meet user needs, which helps to improve the conversion rate of advertisements.

[0109] The present disclosure provides a scenario diagram of a training method for a generative recall large model based on advertisements, such as Fig. 9 shown.

[0110] As mentioned above, the training method of the advertisement-based generative recall large model provided by the embodiment of the present disclosure is applied to electronic devices. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices.

[0111] Specifically, the electronic device can perform the following operations:

[0112] Use the existing training data set to supervise the model and build an initial large model that can map keywords to corresponding identifier sequences;

[0113] Based on the target object click behavior data in the advertisement recall scenario, the initial large model is fine-tuned to obtain a generative recall large model, which is used to generate an identifier sequence according to the query words.

[0114] It should be understood that Fig. 9 The scene diagram shown is only illustrative and not restrictive. Those skilled in the art can Fig. 9Various obvious changes and / or substitutions can be made to the examples, and the resulting technical solutions still fall within the scope of the disclosure of the embodiments of the present disclosure.

[0115] The present disclosure provides a scenario diagram of an advertisement display method, such as Fig.10 shown.

[0116] As mentioned above, the advertisement display method provided by the embodiment of the present disclosure is applied to electronic devices. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices.

[0117] Specifically, the electronic device can perform the following operations:

[0118] Input the query word into the generative recall large model to obtain the identifier sequence output by the generative recall large model;

[0119] Pull multiple advertisements from the advertisement library based on the identifier sequence to obtain an advertisement set corresponding to the query term;

[0120] Using the double tower model, at least one target advertisement matching the query term is selected from the advertisement collection;

[0121] Display at least one targeted ad.

[0122] It should be understood that Fig.10 The scene diagram shown is only illustrative and not restrictive. Those skilled in the art can Fig.10 Various obvious changes and / or substitutions can be made to the examples, and the resulting technical solutions still fall within the scope of the disclosure of the embodiments of the present disclosure.

[0123] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0124] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0125] Fig.11A schematic block diagram of an example electronic device 1100 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0126] like Fig.11 As shown, the device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 to a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0127] A number of components in the device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0128] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as a training method and / or an advertising display method for a generative recall large model based on advertising. For example, in some embodiments, a training method and / or an advertising display method for a generative recall large model based on advertising may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1100 via ROM 1102 and / or a communication unit 1109. When the computer program is loaded into RAM 1103 and executed by the computing unit 1101, one or more steps of the above-described advertising-based generative recall large model training method and / or advertising display method can be performed. Alternatively, in other embodiments, the computing unit 1101 can be configured to perform the advertising-based generative recall large model training method and / or advertising display method in any other appropriate manner (e.g., by means of firmware).

[0129] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0130] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0131] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0133] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

[0134] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0135] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0136] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A training method for a large generative recall model based on advertising, comprising: Use the existing training data set to supervise the model and build an initial large model that can map keywords to corresponding identifier sequences; Based on the target object click behavior data in the advertisement recall scenario, the initial large model is fine-tuned to obtain a generative recall large model, and the generative recall large model is used to generate an identifier sequence according to a query word.

2. The method according to claim 1, wherein: The initial large model is fine-tuned based on the target object click behavior data in the advertisement recall scenario to obtain a generative recall large model, including: Extracting query words and keywords based on the target object's click behavior data; Mapping the keyword to the identifier sequence; The initial large model is fine-tuned according to the query word and the identifier sequence to obtain the generative recall large model.

3. The method according to claim 1, wherein: In the identifier sequence, each identifier corresponds to a plurality of keywords with similar semantics.

4. The method according to claim 1, wherein: The method further comprises: Perform hierarchical clustering on keywords to obtain clustering results; Constructing a clustering tree according to the clustering results; A mapping relationship from the keyword to the identifier sequence is constructed based on the clustering tree; wherein the mapping relationship is used for supervised training to obtain the initial large model.

5. The method according to claim 4, wherein: The step of constructing a mapping relationship between the keyword and the identifier sequence based on the clustering tree includes: Determine the number of layers of the clustering tree and the number of nodes in each layer, wherein each leaf node in the clustering tree corresponds to a plurality of keywords with similar semantics; The path from the root node to the leaf node is determined as the identifier sequence.

6. An advertisement display method, comprising: Input the received query words into the generative recall large model to obtain an identifier sequence output by the generative recall large model; wherein the generative recall large model is trained by the method described in any one of claims 1 to 5; Pull multiple advertisements from the advertisement library based on the identifier sequence to obtain an advertisement set; Using the double tower model, at least one target advertisement matching the query term is selected from the advertisement collection; The at least one targeted advertisement is displayed.

7. The method according to claim 6, wherein: The step of using the double tower model to select at least one target advertisement matching the query term from the advertisement set includes: Inputting the query word and the advertisement landing page of each advertisement in the advertisement set into the double tower model; Obtaining the relevance or expected click-through rate between the query word output by the dual-tower model and each advertisement landing page; The advertisement whose relevance is greater than a first threshold or whose expected click rate is greater than a second threshold is used as the at least one target advertisement.

8. A training device for a large generative recall model based on advertising, comprising: The training module is used to supervise the model using the existing training data set and build an initial large model that can map keywords to corresponding identifier sequences; The fine-tuning module is used to fine-tune the initial large model based on the target object click behavior data in the advertisement recall scenario to obtain a generative recall large model, wherein the generative recall large model is used to generate an identifier sequence according to the query word.

9. The device according to claim 8, wherein: The fine-tuning module comprises: An extraction submodule, used to extract query words and keywords based on the target object click behavior data; A mapping submodule, used for mapping the keyword to the identifier sequence; A generation submodule is used to fine-tune the initial large model according to the query word and the identifier sequence to obtain the generative recall large model.

10. The device according to claim 8, wherein: In the identifier sequence, each identifier corresponds to a plurality of keywords with similar semantics.

11. The device according to claim 8, wherein: The device also includes: Clustering module, used to perform hierarchical clustering on keywords and obtain clustering results; A first construction module, used to construct a clustering tree according to the clustering result; The second construction module is used to construct a mapping relationship from the keyword to the identifier sequence based on the clustering tree; wherein the mapping relationship is used for supervised training to obtain the initial large model.

12. The device according to claim 11, wherein: The second building block is used to: Determine the number of layers of the clustering tree and the number of nodes in each layer, wherein each leaf node in the clustering tree corresponds to a plurality of keywords with similar semantics; The path from the root node to the leaf node is determined as the identifier sequence.

13. An advertising display device, comprising: An acquisition module, used for inputting the received query words into a generative recall large model to obtain an identifier sequence output by the generative recall large model; wherein the generative recall large model is obtained by training according to any one of claims 1 to 5; An advertisement pulling module, configured to pull multiple advertisements from an advertisement library based on the identifier sequence to obtain an advertisement set; An advertisement screening module, configured to screen out at least one target advertisement matching the query term from the advertisement set by using a double-tower model; The advertisement display module is used to display the at least one target advertisement.

14. The device according to claim 13, wherein: The advertisement screening module comprises: An input submodule, used to input the query word and the advertisement landing page of each advertisement in the advertisement set into the double tower model; An acquisition submodule, used to acquire the relevance or expected click rate between the query word output by the dual-tower model and each advertisement landing page; The screening submodule is used to select the advertisement whose relevance is greater than a first threshold or whose expected click rate is greater than a second threshold as the at least one target advertisement.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are for causing a computer to execute a method according to any one of claims 1-7.

17. A computer program product comprising a computer program stored on a storage medium, the computer program implementing the method according to any one of claims 1 to 7 when executed by a processor.

Images