Method and device for determining click rate of media information and computer equipment

Abstract

The invention relates to a media information click rate determination method and device, computer equipment, a storage medium and a computer program product. The method can be applied to an application scene in which a vehicle-mounted terminal, a cloud server or other equipment interacts with an application program, and comprises the following steps: performing media information search based on query information in a query request to obtain candidate media information; determining a query embedding vector of the query information; the query embedded vector is obtained by performing mapping processing on context information of query information by the large language model; based on the query embedding vector and a media embedding vector of the candidate media information, determining an interaction feature vector of the candidate media information; the media embedding vector is obtained by performing mapping processing on context information of the candidate media information by a large language model; and determining the click rate of the candidate media information based on the interaction feature vector of the candidate media information. By adopting the method, the accuracy of the predicted click rate of the media information can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, storage medium, and computer program product for determining the click-through rate of media information. Background Technology

[0002] With the development of computer and internet technologies and the arrival of the 5G era, the internet has brought immense convenience to modern life. More and more users can share information, exchange emotions, and process business through various multimedia applications, bringing them greater ease of use. For example, with the increasing prevalence of mobile devices and faster network access speeds, users can search for topics of interest on different domestic and international social media platforms (or applications). For instance, users can enter their queries into pages with search engine functionality, and the platform will recommend relevant interest groups or documents based on the user's interests and historical behavior to meet their personalized needs.

[0003] However, current methods for determining the click-through rate (CTR) of media information typically employ logistic regression to predict the CTR of media information related to the query. In practice, using logistic regression usually requires manually designing and selecting features, which is not only time-consuming and labor-intensive, but also results in potentially inaccurate CTR predictions based on manually designed and selected features, thus affecting the accuracy of the prediction. Therefore, how to effectively improve the accuracy of predicted CTR of media information has become an urgent problem to be solved. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer device, computer-readable storage medium, and computer program product for determining the click-through rate of media information, which can effectively improve the accuracy of predicting the click-through rate of media information, thereby also improving the accuracy of query result display and bringing convenience to users.

[0005] In a first aspect, this application provides a method for determining the click-through rate (CTR) of media information. The method includes: searching for media information based on query information in a query request to obtain candidate media information; determining a query embedding vector for the query information; the query embedding vector is obtained by mapping the context information of the query information using a large language model; determining an interaction feature vector for the candidate media information based on the query embedding vector and the media embedding vector of the candidate media information; the media embedding vector is obtained by mapping the context information of the candidate media information using the large language model; and determining the CTR of the candidate media information based on the interaction feature vector.

[0006] Secondly, this application also provides a device for determining the click-through rate of media information. The device includes: a search module, used to search for media information based on query information in a query request to obtain candidate media information; a determination module, used to determine the query embedding vector of the query information; the query embedding vector is obtained by mapping the context information of the query information using a large language model; based on the query embedding vector and the media embedding vector of the candidate media information, an interaction feature vector of the candidate media information is determined; the media embedding vector is obtained by mapping the context information of the candidate media information using the large language model; and based on the interaction feature vector of the candidate media information, the click-through rate of the candidate media information is determined.

[0007] In one embodiment, the apparatus further includes: an acquisition module, configured to acquire hot query information with a query count greater than a threshold; the hot query information includes query information in the query request; a processing module, configured to perform mapping processing on the hot query information through the large language model to obtain a query embedding vector of the hot query information; an storage module, configured to store the query embedding vector of the hot query information into a first database; and a query module, configured to query the query embedding vector of the query information from the first database.

[0008] In one embodiment, the processing module is further configured to perform real-time mapping processing on the query information through the large language model to obtain the query embedding vector of the query information when the query embedding vector of the query information is not found.

[0009] In one embodiment, the large language model includes a multi-layer encoder; the processing module is further configured to perform real-time mapping processing on the query information through the multi-layer encoder to obtain the embedding vectors output by the multi-layer encoder respectively; the device further includes: a determination module, configured to determine a target embedding vector from the embedding vectors output by the multi-layer encoder respectively, and use the target embedding vector as the query embedding vector of the query information.

[0010] In one embodiment, the acquisition module is further configured to acquire the embedding vector output by the last coding layer from the embedding vectors output by the multi-layer encoders respectively, as the target embedding vector; the search module is further configured to search for the embedding vector with an end marker from the embedding vectors output by the multi-layer encoders respectively, as the target embedding vector.

[0011] In one embodiment, the apparatus further includes: a processing module, configured to serialize the plain text document to obtain a serialized document when the document in the second database is a plain text document; and to map the serialized document using the large language model to obtain a document embedding vector of the plain text document; a storage module, configured to store the document embedding vector of the plain text document into a first database; and a search module, further configured to search for media information in the second database based on the query information in the query request.

[0012] In one embodiment, the processing module is further configured to, when the document in the second database is a non-plain text document, process the non-text elements in the non-plain text document using a multimodal model to obtain non-text features; and process the text elements in the non-plain text document using the large language model to obtain text features; the apparatus further includes: a generation module, configured to generate a document embedding vector of the document based on the non-text features and the text features; and a storage module, configured to store the document embedding vector of the document in the first database.

[0013] In one embodiment, the candidate media information includes candidate documents; the media embedding vector includes a document embedding vector; the apparatus further includes: an acquisition module, configured to acquire the document embedding vector of the candidate document from the first database; and a determination module, further configured to determine the interaction feature vector of the candidate document based on the query embedding vector and the document embedding vector of the candidate document.

[0014] In one embodiment, the determining module is further configured to determine the difference between the query embedding vector and the document embedding vector of the candidate document; determine the interaction feature vector of the candidate document based on the query embedding vector, the document embedding vector of the candidate document and the difference; and the processing module is further configured to perform feature processing on the query embedding vector and the document embedding vector of the candidate document through a neural network model to obtain the interaction feature vector of the candidate document.

[0015] In one embodiment, the apparatus further includes a processing module, configured to classify the interaction feature vector of the candidate media information using a classification model to obtain the click-through rate of the candidate media information.

[0016] In one embodiment, the processing module is further configured to perform feature processing on the interaction feature vector of the candidate media information through the hidden layer of the classification model to obtain the click vector of the candidate media information; the determining module is further configured to determine the click-through rate of the candidate media information based on the click vector, weight coefficients and bias terms.

[0017] In one embodiment, the click-through rate is obtained by processing the interaction feature vector through a classification model. The device further includes: an acquisition module for acquiring a training dataset; wherein the training dataset includes training pairs and labels for each training pair; the training pairs include historical query information and historical candidate media information; the labels are used to identify the click status of the training pairs; and a training module for jointly training an initial large language model and an initial classification model using the training pairs and their respective labels, until training stops when the loss function meets a preset condition, thereby obtaining the large language model and the classification model.

[0018] In one embodiment, the acquisition module is further configured to acquire search history logs within a preset time range; wherein the search history logs include the historical query information, the historical candidate media information, and the click status; the device further includes: a sampling module, configured to randomly sample the historical query information and the historical candidate media information as the training pair to obtain the training dataset; wherein, during the random sampling process, the sampling frequency of the historical query information is limited by a preset sampling frequency threshold.

[0019] In one embodiment, the apparatus further includes: a processing module, configured to process the click-through rate using a ranking model to obtain a ranking result; a determining module, configured to determine the ranking result based on the click-through rate, historical click-through rate, and relevance score; and a display module, configured to display the candidate media information based on the ranking result.

[0020] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and the processor, when executing the computer program, performs the following steps: searching for media information based on query information in a query request to obtain candidate media information; determining a query embedding vector for the query information; the query embedding vector is obtained by mapping the context information of the query information using a large language model; determining an interaction feature vector for the candidate media information based on the query embedding vector and the media embedding vector of the candidate media information; the media embedding vector is obtained by mapping the context information of the candidate media information using the large language model; and determining the click-through rate of the candidate media information based on the interaction feature vector of the candidate media information.

[0021] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps: searching for media information based on query information in a query request to obtain candidate media information; determining a query embedding vector for the query information; the query embedding vector is obtained by mapping the context information of the query information using a large language model; determining an interaction feature vector for the candidate media information based on the query embedding vector and the media embedding vector of the candidate media information; the media embedding vector is obtained by mapping the context information of the candidate media information using the large language model; and determining the click-through rate of the candidate media information based on the interaction feature vector of the candidate media information.

[0022] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps: searching for media information based on query information in a query request to obtain candidate media information; determining a query embedding vector for the query information; the query embedding vector is obtained by mapping the context information of the query information using a large language model; determining an interaction feature vector for the candidate media information based on the query embedding vector and the media embedding vector of the candidate media information; the media embedding vector is obtained by mapping the context information of the candidate media information using the large language model; and determining the click-through rate of the candidate media information based on the interaction feature vector of the candidate media information.

[0023] The aforementioned method, apparatus, computer equipment, storage medium, and computer program product for determining the click-through rate of media information involves searching for media information based on query information in a query request to obtain candidate media information and determining the query embedding vector of the query information. The query embedding vector is obtained by mapping the context information of the query information using a large language model. Based on the query embedding vector and the media embedding vector of the candidate media information, the interaction feature vector of the candidate media information is determined. The media embedding vector is obtained by mapping the context information of the candidate media information using a large language model. Based on the interaction feature vector of the candidate media information, the click-through rate of the candidate media information is determined. Since both the query embedding vector of the query information and the media embedding vector of the candidate media information are obtained by mapping contextual information through a large language model, the query embedding vector and media embedding vector obtained in this application can more accurately reflect the user's query intent and the content of the candidate media information. This makes the interaction feature vector generated based on the query embedding vector and media embedding vector more accurate, and the click-through rate of the candidate media information determined based on the interaction feature vector will also be more accurate. In practical applications, this means that it can more effectively predict the user's click behavior for different candidate media information, thereby providing more accurate query results. That is, it effectively improves the accuracy of the predicted click-through rate of media information, and thus also effectively improves the accuracy of the query result display. It can better predict the user's satisfaction with the displayed results of the queried media information, thereby improving the user experience. Attached Figure Description

[0024] Figure 1 This is a diagram illustrating the application environment of a method for determining the click-through rate of media information in one embodiment.

[0025] Figure 2 This is a flowchart illustrating a method for determining the click-through rate of media information in one embodiment;

[0026] Figure 3 This is a schematic diagram of a page that triggers a specific query request in a social application, as shown in one embodiment.

[0027] Figure 4 This is a schematic diagram of the algorithm flow for determining the predicted click-through rate of each candidate media information in one embodiment;

[0028] Figure 5 This is a flowchart illustrating the steps of real-time mapping of query information using a large language model to obtain the query embedding vector of the query information in one embodiment.

[0029] Figure 6 This is a schematic diagram of a structure in a large language model that includes a multi-layer encoder in one embodiment.

[0030] Figure 7This is a schematic diagram illustrating the generation of interactive feature vectors in one embodiment;

[0031] Figure 8 This is a schematic diagram of the Transformer layer of an LLM in one embodiment;

[0032] Figure 9 A structural block diagram of a media information click-through rate determination device in one embodiment;

[0033] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0035] It should be noted that in the following description, the terms "first, second, and third" are used only to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, and third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0036] The click-through rate determination method for media information provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Specifically, terminal 102 can interact with a multimedia information platform with search capabilities, i.e., server 104. Server 104 searches for media information based on the query information in the query request triggered by the user through terminal 102, obtaining candidate media information. Server 104 determines the query embedding vector of the query information; the query embedding vector is obtained by mapping the context information of the query information using a large language model. Based on the query embedding vector and the media embedding vector of the candidate media information, server 104 determines the interaction feature vector of the candidate media information; the media embedding vector is obtained by mapping the context information of the candidate media information using a large language model. Based on the interaction feature vector of the candidate media information, server 104 determines the click-through rate of the candidate media information. Furthermore, server 104 can display each candidate media information according to the determined click-through rate, so that the user can quickly view the displayed candidate media information through terminal 102.

[0037] Terminal 102 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smart TV, smartwatch, IoT device, or portable wearable device. IoT devices can include smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc.

[0038] Server 104 can be an independent physical server or a service node in a blockchain system. The service nodes in the blockchain system form a peer-to-peer (Peer To Peer) network. The Peer To Peer protocol is an application layer protocol that runs on top of the Transmission Control Protocol (TCP).

[0039] In addition, server 104 can also be a server cluster consisting of multiple physical servers, which can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0040] Terminal 102 and server 104 can be connected via Bluetooth, USB (Universal Serial Bus) or network, etc., and this application does not impose any restrictions.

[0041] In one embodiment, such as Figure 2 As shown, a method for determining the click-through rate of media information is provided. This method can be executed by the server or the terminal alone, or by both the server and the terminal. This method can be applied to... Figure 1 Taking the server in the example, the following steps are included:

[0042] Step 202: Based on the query information in the query request, search for media information to obtain candidate media information.

[0043] In this context, a query request refers to a query request initiated by different objects. For example, in this application, a query request may be a query request triggered by different objects (operators) by entering specific query information in a search engine or search page.

[0044] Query information refers to the information to be queried carried in a query request. For example, the query information in this application may include query terms or query statements. For instance, object A may enter specific query information such as "how to buy genuine honey plums" into the search page of application A.

[0045] Candidate media information refers to media information related to the query information in the query request that is retrieved from a database or retrieval library. For example, candidate media information in this application may include documents, images (icons), videos, public accounts, mini programs, social circle information (friends circle information), and documents containing images, videos or other non-text elements.

[0046] Specifically, the click-through rate determination method for media information provided in this application can be widely applied to various scenarios involving the display of personalized materials, such as social media, games, and film and television production. This means that different users (operation objects) can interact with the multimedia information platform (or application) using their devices. When a user (operation object) wants to query specific media information (such as searching for an article on "how to stir-fry beef"), the user can trigger an action to open a multimedia application (APP) on their terminal and enter the application's page through a selection action. In other words, the user can log in to a multimedia application (such as a short video application or a social application) through a trigger action. Furthermore, the user can initiate a specific query request on the page displayed in the multimedia application. For example, on the main page of a social application (video account application) displayed on the terminal, operation object A using the social application can trigger a query request by clicking icons such as "Search" or "Search Now." The terminal responds to the query request triggered by operation object A in the social application by sending the query request to the backend server of the social application. The backend server of the social application can then search for media information in the database based on the query information in the query request to obtain candidate media information.

[0047] For example, let's illustrate this with a processing scenario in a social application that has search functionality. For instance... Figure 3 The image shows a page illustration of triggering a specific query request within a social application. When user A wants to search for articles on "how to buy genuine honey plums," user A can trigger an action to open a specific social application on their device. That is, the user can log into the social application through a trigger action. Furthermore, on the page displayed in the social application, the user can initiate a query request for the information "how to buy genuine honey plums" through a trigger action. For example, as shown on the device's display... Figure 3 On the page of the social application shown, user A can view the specific content and related functional information on the page. Assume user A is in... Figure 3 If the "Search" icon is clicked on the page shown on the left, the terminal will respond to the "Search" request triggered by user A in this social application, displaying the following: Figure 3 The search page shown on the right allows user A to find search results such as... Figure 3In the search page shown on the right, if the user enters the query "how to buy genuine honey plums" and clicks the "search" icon, the terminal responds to this user's search request by sending it to the social application's backend server. The backend server can then use this query to search its database for media information related to "how to buy genuine honey plums." These articles represent candidate media information.

[0048] Step 204: Determine the query embedding vector of the query information; the query embedding vector is obtained by the large language model mapping the context information of the query information.

[0049] In this application, the query embedding vector refers to the embedding vector corresponding to each query. It can be understood that the query embedding vector in this application can be obtained by pre-processing the model offline or online in real-time. For example, in the offline stage, for high-frequency query information, the server can pre-process the high-frequency query information using a pre-trained Large Language Model (LLM) to obtain embedding vectors, and store these embedding vectors in a database (i.e., a vector retrieval database). This allows the server to directly retrieve the query embedding vector from the database (i.e., the vector retrieval database) when processing high-frequency query information in real-time, achieving fast retrieval. Alternatively, if the server does not obtain the query embedding vector from the database (i.e., the vector retrieval database) when processing high-frequency query information in real-time, the server can also process the query information in real-time using the Large Language Model (LLM) to obtain the corresponding query embedding vector. It can be understood that the embedding vector corresponding to each query in this application has a one-to-one correspondence.

[0050] Large language models refer to large-scale language models, which are intelligent models that can generate coherent and meaningful text.

[0051] The context information of a query refers to the specific content contained in the query information. For example, if the query information is "how to buy genuine honey plums", then the context information refers to the specific content contained in the sentence "how to buy genuine honey plums".

[0052] Mapping processing refers to the process of converting query information into a continuous vector. For example, if the query information is a word, then mapping processing refers to the process of converting that word into a vector.

[0053] Specifically, the server searches for media information in the database based on the query information in the query request. After obtaining candidate media information, the server can first query the query embedding vector of the query information from the database used to store vectors (i.e., the vector retrieval database). When the server does not find the query embedding vector of the query information in the vector retrieval database, the server can perform real-time mapping processing on the query information through a pre-trained large language model to obtain the query embedding vector of the query information.

[0054] For example, let's illustrate this with a scenario from a social application that has search functionality. Assume a terminal responds to a "search" query triggered by user A on the search page of the social application. The terminal sends this query to the application's backend server. Based on the query information "how to buy genuine honey plums," the backend server searches its database for media information related to "how to buy genuine honey plums." After obtaining multiple articles related to "how to buy genuine honey plums," the server can first retrieve the query embedding vector for "how to buy genuine honey plums" from a vector retrieval database. If the server does not find the query embedding vector in the vector retrieval database, it can use a pre-trained large language model to perform real-time mapping processing on "how to buy genuine honey plums" to obtain the query embedding vector A. This query embedding vector A can be a continuous vector.

[0055] Step 206: Based on the query embedding vector and the media embedding vector of the candidate media information, determine the interaction feature vector of the candidate media information; the media embedding vector is obtained by the large language model mapping the context information of the candidate media information.

[0056] Here, the media embedding vector refers to the embedding vector corresponding to each candidate media information. It can be understood that the media embedding vector in this application can be obtained by the model in advance during the offline stage, or it can be obtained by the model in real time online. For example, in the offline stage, the server can pre-process each document in the document library (a database used to store documents) using a pre-trained Large Language Model (LLM) to generate the document embedding vector (i.e., media embedding vector) of each document, and store the document embedding vector in the database (i.e., the vector retrieval database). This allows the server to directly obtain the document embedding vector (i.e., media embedding vector) of each candidate document (i.e., candidate media information) from the database (i.e., the vector retrieval database) when processing query information in real time, thus achieving fast retrieval.

[0057] An interaction feature vector is a vector generated by interacting with the query embedding vector and the media embedding vectors of different candidate media information. In other words, the interaction feature vector in this application is a vector used to reflect the relationship between query information and candidate media information. For example, the interaction feature vector in this application can be obtained by concatenating the query embedding vector, the media embedding vector of the candidate media information, and the difference vector between the query embedding vector and the media embedding vector of the candidate media information.

[0058] The contextual information of candidate media information refers to the specific content contained in the candidate media information. For example, if the candidate media information is "Article 1", then the contextual information refers to the specific text information contained in "Article 1".

[0059] Specifically, after determining the query embedding vector of the query information, the server can also preferentially retrieve the media embedding vectors of each candidate media information from the database used to store vectors (i.e., the vector retrieval database), and determine the interaction feature vectors between the query embedding vector and each candidate media information based on the query embedding vector and the media embedding vectors of each candidate media information. That is, there can be multiple interaction feature vectors in this application. For example, the server can first calculate the difference between the query embedding vector and the media embedding vector of candidate media information A (which is also a vector), and then determine the interaction feature vector between the query embedding vector and candidate media information A based on the query embedding vector, the media embedding vector of candidate media information A, and the difference (i.e., the difference vector).

[0060] For example, let's take candidate media information as candidate documents. Suppose that in response to a "search" query request triggered by user A on the search page of a social application, the terminal sends the "search" query request triggered by user A to the backend server of the social application. The backend server then searches the database for media information based on the query information in the "search" request, namely "how to buy genuine honey plums." After obtaining multiple candidate documents related to "how to buy genuine honey plums," the backend server can first query the query embedding vector A of the query information "how to buy genuine honey plums" from the vector retrieval database. When the server retrieves the query embedding vector A of the query information "how to buy genuine honey plums" and the document embedding vectors of the multiple candidate documents from the vector retrieval database, the server can calculate the interaction feature vector between the query embedding vector A and the document embedding vectors of the multiple candidate documents based on the query embedding vector A and the document embedding vectors of the multiple candidate documents. For example, the server can calculate the difference between the query embedding vector A and the document embedding vector A of the candidate document A, obtaining the difference vector B = |query embedding vector A - document embedding vector A|. The server can then concatenate the calculated difference vector B, the query embedding vector A, and the document embedding vector A, and use the concatenated vector as the interaction feature vector between the query embedding vector A and the document embedding vector A of the candidate document A.

[0061] Step 208: Determine the click-through rate of candidate media information based on the interaction feature vector of candidate media information.

[0062] The click-through rate (CTR) refers to the predicted CTR of each candidate media information. It can be understood that the CTR in this application can refer to the ratio of the number of times the candidate media information is displayed to the number of times the user (the object of operation) clicks on the candidate media information.

[0063] Specifically, after determining the interaction feature vector between the query embedding vector and the candidate media information based on the query embedding vector and the media embedding vector of the candidate media information, the server can classify the interaction feature vector between the query embedding vector and each candidate media information using a trained classification model to obtain the predicted click-through rate (CTR) of different candidate media information. Furthermore, the server can display each candidate media information based on the predicted CTR. For example, the server can process the predicted CTR using a ranking model to obtain a ranking result and display each candidate media information based on the ranking result; alternatively, the server can determine the ranking result based on the CTR (predicted CTR) of each candidate media information, the historical CTR of each candidate media information, and the relevance score between each candidate media information and the query information, and display each candidate media information based on the ranking result. It can be understood that the CTR in this embodiment is the predicted probability that each candidate media information may be clicked by a user.

[0064] For example, let's illustrate this using candidate media information as a candidate document. Figure 4 The diagram shows the algorithm flow for determining the predicted click-through rate of each candidate media information. It assumes the server is based on the following... Figure 4 The query embedding vector `query_emb` for the query information "how to buy genuine honey plums" and the document embedding vector `doc_emb` for candidate document A are shown below. The interaction feature vector between these two vectors is calculated as: vector f_vec = query_emb; doc_emb; |query_emb - doc_emb|. Furthermore, the server can use methods such as... Figure 4 The trained classification model (i.e., the MLP layer) shown classifies the interaction feature vector f_vec between the query embedding vector query_emb and the document embedding vector doc_emb of candidate document A, thus obtaining the predicted click-through rate P(C) of candidate document A. This allows the server to further display each candidate document based on its predicted click-through rate. For example, the server can use a ranking model to process the predicted click-through rates of each candidate document, obtain a ranking result, and then display each candidate document based on the ranking result. The display result can be as follows: Figure 3 As shown in the image.

[0065] In this embodiment, media information is searched based on the query information in the query request to obtain candidate media information, and the query embedding vector of the query information is determined. The query embedding vector is obtained by mapping the context information of the query information using a large language model. Based on the query embedding vector and the media embedding vector of the candidate media information, the interaction feature vector of the candidate media information is determined. The media embedding vector is obtained by mapping the context information of the candidate media information using a large language model. Based on the interaction feature vector of the candidate media information, the click-through rate of the candidate media information is determined. Since both the query embedding vector of the query information and the media embedding vector of the candidate media information are obtained by mapping contextual information through a large language model, the query embedding vector and media embedding vector obtained in this application can more accurately reflect the user's query intent and the content of the candidate media information. This makes the interaction feature vector generated based on the query embedding vector and media embedding vector more accurate, and the click-through rate of the candidate media information determined based on the interaction feature vector will also be more accurate. In practical applications, this means that it can more effectively predict the user's click behavior for different candidate media information, thereby providing more accurate query results. That is, it effectively improves the accuracy of the predicted click-through rate of media information, and thus also effectively improves the accuracy of the query result display. It can better predict the user's satisfaction with the displayed results of the queried media information, thereby improving the user experience.

[0066] In one embodiment, before performing media information search based on query information in the query request to obtain candidate media information, the method further includes:

[0067] Retrieve information on frequently queried queries that have exceeded a threshold; this information includes query details within the query requests.

[0068] By mapping the hot query information using a large language model, the query embedding vector of the hot query information is obtained;

[0069] The query embedding vector of hot query information is stored in the first database;

[0070] Determining the query embedding vector of the query information includes:

[0071] The query embedding vector for retrieving the query information from the first database.

[0072] Hot search terms refer to frequently occurring or popular queries. For example, if a recent hot topic is whether a household registration book is required for marriage, then the query "Does a household registration book need to be brought for marriage?" can be considered a hot search term. Alternatively, if a query A is searched more than a preset threshold, then query A can also be considered a hot search term.

[0073] The first database refers to a database used to store vectors. For example, the first database in this application could be a vector retrieval database.

[0074] Specifically, during the offline phase, the server can obtain information on hot queries with a query count exceeding a threshold, and then, through methods such as... Figure 4 The Large Language Model (LLM) shown maps each hot query to obtain its query embedding vector, which is then stored in a vector retrieval database. Further, when the server receives a query request A triggered by an operation, it can search for media information in the media data database based on query information A in the request A to obtain candidate media information. Furthermore, the server can prioritize checking the vector retrieval database for the existence of a query embedding vector for query information A. Since hot query information includes query information A, its query embedding vector is already stored in the vector retrieval database, allowing the server to directly retrieve it. This results in query embedding vectors and document embedding vectors generated using Large Language Models (LLMs) possessing extremely high semantic representation capabilities, providing richer and more accurate semantic information. This means they can more accurately reflect user intent and document content, leading to more accurate semantic information in the interaction features determined by high-quality embedding vectors generated by LLMs. The click-through rates of candidate media information determined based on interaction feature vectors will also be more accurate. In practical applications, this means that user click behavior for different candidate media information can be predicted more effectively, thus providing more accurate query results or search results and improving the user experience.

[0075] In one embodiment, the method further includes:

[0076] When the query embedding vector for the query information is not found, the query information is mapped in real time through the large language model to obtain the query embedding vector for the query information.

[0077] Specifically, when the server receives a query request A triggered by an operation object, the server can search for media information in the database used to store media data based on the query information A in the query request A to obtain candidate media information. Furthermore, the server can first check the vector retrieval database to see if there is a query embedding vector for the query information A. If the pre-processed offline hot query information does not include the query information A, then the query embedding vector for the query information A is not stored in the vector retrieval database. That is, when the server does not find the query embedding vector for the query information A in the vector retrieval database, the server can perform real-time mapping processing on the query information A through the large language model to obtain the query embedding vector for the query information A. This results in query embedding vectors and document embedding vectors generated using Large Language Models (LLMs) possessing extremely high semantic representation capabilities, providing richer and more accurate semantic information. This means they can more accurately reflect user intent and document content, leading to more accurate semantic information in the interaction features determined by high-quality embedding vectors generated by LLMs. The click-through rates of candidate media information determined based on interaction feature vectors will also be more accurate. In practical applications, this means that user click behavior for different candidate media information can be predicted more effectively, thus providing more accurate query results or search results and improving the user experience.

[0078] In one embodiment, such as Figure 5 As shown, the large language model includes a multi-layer encoder; the steps of performing real-time mapping processing on the query information through the large language model to obtain the query embedding vector of the query information include:

[0079] Step 502: The query information is mapped in real time through a multi-layer encoder to obtain the embedding vectors output by the multi-layer encoder respectively;

[0080] Step 504: Determine the target embedding vector from the embedding vectors output by the multi-layer encoders, and use the target embedding vector as the query embedding vector for the query information.

[0081] Here, the target embedding vector refers to one or more embedding vectors selected from the embedding vectors output by each encoder layer as the target embedding vector. For example, assuming that the large language model includes 10 encoder layers (i.e., 10 hidden layers), the server can select the embedding vector output by the last layer, i.e., the 10th layer, from the embedding vectors output by the 10 hidden layers as the target embedding vector in this embodiment of the application.

[0082] Specifically, when the server does not find the query embedding vector for query information A in the vector retrieval database, the server can perform real-time mapping processing on query information A through the large language model. That is, the server can perform real-time mapping processing on query information A using the multi-layer encoder (multiple hidden layers H) contained in the large language model, obtaining embedding vectors {H1, H2…Hn} output by each layer. The server can then determine the target embedding vector Hn from the embedding vectors {H1, H2…Hn} output by each layer and use the target embedding vector Hn as the query embedding vector for query information A. This allows for more effective prediction of user click behavior by using high-quality embedding vectors generated by LLM, thereby providing more accurate search results and improving the user experience.

[0083] In one embodiment, the step of determining the target embedding vector from the embedding vectors output by the multi-layer encoder includes:

[0084] From the embedding vectors output by the multi-layer encoders, the embedding vector output by the last coding layer is obtained as the target embedding vector; or

[0085] From the embedding vectors output by the multi-layer encoder, find the embedding vector with the end-of-line marker as the target embedding vector.

[0086] In this context, the last coding layer refers to the last layer in a multi-layer encoder. For example, if a large language model includes 10 encoder layers (i.e., 10 hidden layers or coding layers), the server can select the embedding vector H10 output by the last layer, i.e., the 10th last coding layer, from the embedding vectors output by the 10 hidden layers as the target embedding vector in this embodiment.

[0087] An embedding vector with an end-of-sentence marker (EOS) refers to an embedding vector that has an EOS marker. The EOS marker is typically used to indicate the end of a sentence. For example, for query information Q and candidate document D, the server can obtain their respective EOS embedding vectors as the target embedding vectors. , .

[0088] Specifically, such as Figure 6 The diagram shows a structural illustration of a large language model containing multiple encoders. The server can utilize methods such as... Figure 6The large language model shown contains multiple encoders (multiple hidden layers H) that perform real-time mapping processing on query information A, resulting in embedding vectors {H1, H2…Hn} output by the multiple encoders (multiple hidden layers H). The server can then obtain the embedding vector Hn output by the last coding layer, i.e., the Nth layer, from the embedding vectors {H1, H2…Hn} output by the multiple encoders (multiple hidden layers H) as the target embedding vector; or the server can also find the embedding vector Hn with the end-of-line marker EOS from the embedding vectors {H1, H2…Hn} output by the multiple encoders (multiple hidden layers H) as the target embedding vector.

[0089] In this embodiment, since the embedding vector output by the last layer in the multilayer encoder captures the contextual information of the entire text (query information or media information), the high-quality embedding vector output by the last layer can be directly used as the target embedding vector. This makes the interaction features determined by the high-quality embedding vector generated based on LLM contain more accurate semantic information, and the click-through rate of the candidate media information determined based on the interaction feature vector will also be more accurate. In practical applications, this means that it is possible to more effectively predict the user's click behavior for different candidate media information, thereby providing more accurate query results or search results and improving the user experience.

[0090] In one embodiment, before performing media information search based on query information in the query request to obtain candidate media information, the method further includes:

[0091] When the documents in the second database are plain text documents, the plain text documents are serialized to obtain serialized documents;

[0092] The serialized documents are mapped using a large language model to obtain the document embedding vector of the plain text document;

[0093] Store the document embedding vector of the plain text document in the first database;

[0094] The media information search based on the query information in the query request includes:

[0095] Based on the query information in the query request, media information is retrieved from the second database.

[0096] The second database refers to a database used to store media data (media information). For example, the second database in this application could be a document library. It is understood that the first and second databases in this application are merely used to distinguish between databases storing different types of data.

[0097] A plain text document is a document that contains only text elements.

[0098] Specifically, let's take candidate media information as an example of candidate documents. In the offline phase, the server can pre-process each document in the document library. Specifically, when the document in the second database (the document library) is a plain text document, the server can serialize it to obtain a serialized document. Then, it uses a trained large language model to map the serialized document, obtaining the document embedding vector of the plain text document. This document embedding vector is then stored in a vector retrieval database, allowing the server to directly query the stored document embedding vectors of plain text documents from the vector retrieval database. In other words, when the server receives a query request A triggered by an operation object, it can search for media information in the second database (the database used to store media data) based on the query information A in the query request A, obtaining multiple candidate documents. The server can then directly query the stored document embedding vectors of these multiple candidate documents (plain text documents) from the vector retrieval database. This allows for the rapid and efficient acquisition of high-quality embedding vectors, which in turn enables the interaction features determined based on these high-quality embedding vectors to contain more accurate semantic information. Consequently, the click-through rates of candidate media information determined based on the interaction feature vectors become more accurate. In practical applications, this means that user click behavior for different candidate media information can be predicted more effectively, thereby providing more accurate query results or search results and improving the user experience.

[0099] In one embodiment, the method further includes:

[0100] When the documents in the second database are not plain text documents, the non-text elements in the non-plain text documents are processed by a multimodal model to obtain non-text features;

[0101] Text features are obtained by processing text elements in non-plain text documents using a large language model;

[0102] Based on non-textual and textual features, document embedding vectors are generated for each document, and these vectors are stored in the first database.

[0103] Non-plain text documents refer to documents that contain both text and non-text elements. Non-text elements can include video, images, audio, and others.

[0104] A multimodal model is a model used to fuse multimodal features. For example, the multimodal model in this application can fuse non-textual features and textual features to generate richer embedding vectors.

[0105] Specifically, let's take candidate media information as an example of candidate documents. In the offline phase, the server can pre-process each document in the document library. Specifically, when the documents in the second database (the document library) are not plain text documents, the server can process the non-text elements in the non-plain text documents using a multimodal model or other feature extraction models to obtain non-text features. Furthermore, the server can process the text elements in the non-plain text documents using a large language model to obtain text features. Further, the server can generate document embedding vectors for the documents based on both non-text and text features and store these vectors in the first database. For example, when a document contains images, videos, or other non-text elements, the server can use multimodal learning methods (or multimodal models) to fuse text and non-text features to obtain richer embedding vectors. This effectively improves the accuracy of predicted click-through rates for each candidate media information. Search engines can more accurately identify and rank media content that is more likely to attract user clicks, which helps improve the overall efficiency and effectiveness of search engines. This enables search engines to better meet users' search needs, while providing more valuable display opportunities for content providers and advertisers.

[0106] In one embodiment, candidate media information includes candidate documents; the media embedding vector includes a document embedding vector; the method further includes:

[0107] Obtain the document embedding vectors of the candidate documents from the first database;

[0108] The step of determining the interaction feature vector of the candidate media information based on the query embedding vector and the media embedding vector of the candidate media information includes:

[0109] Based on the query embedding vector and the document embedding vector of the candidate document, the interaction feature vector of the candidate document is determined.

[0110] Specifically, let's take candidate media information as an example. In the offline phase, the server can pre-process each document in the document library and store the document embedding vectors of each document in a first database. When the server receives a query request A triggered by an operation object, it searches for media information in the second database (the database used to store media data) based on the query information A in query request A. After obtaining multiple candidate documents, the server can directly query the document embedding vectors of the stored candidate documents (plain text documents or non-plain text documents) from the first database (the vector retrieval database). Furthermore, the server can determine the interaction feature vectors between the query embedding vector of query information A and each candidate document based on the query embedding vector of query information A and the document embedding vectors of each candidate document. This effectively improves the accuracy of the predicted click-through rate of each candidate media information. The search engine can more accurately identify and rank media content that is more likely to attract user clicks, which helps improve the overall efficiency and effectiveness of the search engine, enabling it to better meet users' search needs and provide more valuable display opportunities for content providers and advertisers.

[0111] In one embodiment, the step of determining the interaction feature vector of a candidate document based on the query embedding vector and the document embedding vector of the candidate document includes:

[0112] Determine the difference between the query embedding vector and the document embedding vector of the candidate documents;

[0113] The interaction feature vector of the candidate document is determined based on the query embedding vector, the document embedding vector of the candidate document, and the difference; or,

[0114] By performing feature processing on the query embedding vector and the document embedding vector of the candidate document using a neural network model, the interaction feature vector of the candidate document is obtained.

[0115] The neural network model refers to a model used to determine the complex relationship between the query embedding vector and the document embedding vector of the candidate documents. In other words, the neural network model can quickly capture the complex relationship between the query embedding vector and the document embedding vector of each candidate document and generate the corresponding interaction feature vector.

[0116] Specifically, let's take candidate media information as an example to illustrate this. Figure 7 The diagram illustrates the generation of interaction feature vectors. Assume the server is based on... Figure 7The query information shown is the query embedding vector `query_emb` for "how to buy genuine honey plums" and the document embedding vector `doc_emb` for candidate document A. The difference between these two vectors is calculated as `query_emb - doc_emb`. The server can further calculate the absolute value of this difference, resulting in the difference vector `|query_emb - doc_emb|`. Based on this difference, the server can determine the interaction feature vector between `query_emb` and `doc_emb`. Figure 7 The vector shown is f_vec = query_emb; doc_emb; |query_emb - doc_emb|. Alternatively, the server can use a neural network model to perform feature processing on the query embedding vector query_emb and the document embedding vector doc_emb of candidate document A to obtain the interaction feature vector f_vec between them. This results in interaction features determined based on high-quality embedding vectors containing more accurate semantic information. The click-through rate of candidate media information determined based on the interaction feature vector will also be more accurate. In practical applications, this means that user click behavior for different candidate media information can be predicted more effectively, thus providing more accurate query results or search results and improving the user experience.

[0117] In one embodiment, the step of determining the interaction feature vector of a candidate document based on the query embedding vector, the document embedding vector of the candidate document, and the difference includes:

[0118] The query embedding vector, the document embedding vectors of the candidate documents, and the difference are concatenated to obtain the concatenated vector; or...

[0119] The query embedding vector, the document embedding vector of the candidate document, and the absolute value of the difference are concatenated to obtain the concatenated vector;

[0120] The concatenated vector is used as the interactive feature vector of the candidate document.

[0121] Specifically, let's take candidate media information as an example for illustration. Assume the server is based on... Figure 4The query information shown is the query embedding vector `query_emb` for "how to buy genuine honey plums" and the document embedding vector `doc_emb` for candidate document A. The difference between the query embedding vector `query_emb` and the document embedding vector `doc_emb` for candidate document A is calculated as: `query_emb - doc_emb`. When the difference `query_emb - doc_emb` is greater than 0, the server can concatenate the query embedding vector `query_emb`, the document embedding vector `doc_emb` for candidate document A, and the difference (vector) `query_emb - doc_emb` to obtain the concatenated vector `f_vec = query_emb; doc_emb; query_emb - doc_emb`. The concatenated vector `f_vec` is then used as the interaction feature vector between the query embedding vector `query_emb` and the document embedding vector `doc_emb` for candidate document A.

[0122] When the difference query_emb - doc_emb is less than 0, the server can further calculate the absolute value of the difference between the query embedding vector query_emb and the document embedding vector doc_emb of candidate document A, i.e., obtain the difference vector |query_emb - doc_emb|. The server can then concatenate the query embedding vector query_emb, the document embedding vector doc_emb of candidate document A, and the difference vector |query_emb - doc_emb| to obtain the concatenated vector f_vec = query_emb; doc_emb; |query_emb - doc_emb|, and use this concatenated vector f_vec as the interaction feature vector between the query embedding vector query_emb and the document embedding vector doc_emb of candidate document A.

[0123] It is understood that the methods for determining the interaction feature vector in this application embodiment include, but are not limited to: the server calculating the absolute value of the difference between the query embedding vector and the document embedding vector of the candidate document, and then determining the interaction feature vector based on the query embedding vector, the document embedding vector of the candidate document, and the absolute value of the difference. Other calculation methods are also possible. For example, the server can also calculate the sum between the query embedding vector and the document embedding vector of the candidate document, concatenate the query embedding vector, the document embedding vector of the candidate document, the difference (or the absolute value of the difference) between the query embedding vector and the document embedding vector of the candidate document, and the sum between the query embedding vector and the document embedding vector of the candidate document to obtain a concatenated vector, and use the concatenated vector as the interaction feature vector.

[0124] In this embodiment, the interaction features determined based on high-quality embedding vectors also contain more accurate semantic information, and the click-through rate of candidate media information determined based on interaction feature vectors will also be more accurate. In practical applications, this means that it is possible to more effectively predict users' click behavior for different candidate media information, thereby providing more accurate query results or search results, and thus improving the user experience.

[0125] In one embodiment, the step of determining the click-through rate of candidate media information based on the interaction feature vector of candidate media information includes:

[0126] The click-through rate of candidate media information is obtained by classifying the interactive feature vectors of candidate media information using a classification model.

[0127] The classification model in this application may be a multilayer perceptron (MLP).

[0128] Specifically, let's take candidate media information as an example to illustrate this. Figure 4 The diagram shows the algorithm flow for determining the predicted click-through rate of each candidate media information. It assumes the server is based on the following... Figure 4 The query embedding vector `query_emb` for the query information "how to buy genuine honey plums" and the document embedding vector `doc_emb` for candidate document A are shown below. The interaction feature vector between these two vectors is calculated as: vector f_vec = query_emb; doc_emb; |query_emb - doc_emb|. Furthermore, the server can use methods such as... Figure 4 The trained classification model (i.e., the MLP layer) shown classifies the interaction feature vector f_vec between the query embedding vector query_emb and the document embedding vector doc_emb of candidate document A, thus obtaining the predicted click-through rate P(C) of candidate document A. This allows the server to display each candidate document based on its predicted click-through rate. This effectively improves the accuracy of the predicted click-through rate for each candidate media information, enabling better prediction of user satisfaction with the displayed media information and thereby enhancing the user experience.

[0129] In one embodiment, the step of classifying the interaction feature vectors of candidate media information using a classification model to obtain the click-through rate of the candidate media information includes:

[0130] The interaction feature vectors of candidate media information are processed by the hidden layer of the classification model to obtain the click vectors of candidate media information.

[0131] The click-through rate of candidate media information is determined based on click vectors, weight coefficients, and bias terms.

[0132] The click vector can refer to a two-dimensional vector output by the hidden layer. In this embodiment, the first dimension value of the click vector can be taken as the predicted click rate.

[0133] Specifically, in the server via, such as Figure 4 In the process of classifying the interaction feature vector f_vec between the query embedding vector query_emb and the document embedding vector doc_emb of candidate document A, the trained classification model (i.e., the MLP layer) first processes the interaction feature vector f_vec through the hidden layer in the classification model to obtain the click vector z. Further, the server can calculate the click rate of the candidate media information based on the click vector z, the weight coefficient W, and the bias term b. That is, the server can input the click vector output by the hidden layer into the activation layer of the classification model, and process the click vector through the activation layer of the classification model to obtain the predicted probability P(C) output by the activation layer. The way the server determines the click rate of the candidate media information based on the click vector z, the weight coefficient W, and the bias term b can be shown in the following formula (1):

[0134] P(C)=σ(Wz+b ) (1)

[0135] Where P(C) represents the predicted click probability of qd (query information q and candidate document d), σ is the softmax function, and W and b are the weight values ​​(weight coefficients) and bias terms, respectively. Here, the binary classification softmax can be simplified to σ(z) = 1 / (1+e -z ).

[0136] In this embodiment, by using high-quality embedding vectors generated by LLM, the predicted click-through rate output by the classification model (i.e., the MLP layer) achieves a significant improvement in the AUC (Area Under the Receiver Operating Characteristic Curve) metric. As an important indicator for evaluating the performance of a classification model, the improvement in AUC indicates that the classification model in this application has higher accuracy and reliability in distinguishing between clicked and non-clicked behavior. A high AUC value means that the model performs well across all possible classification thresholds, which in practical applications means it can more effectively predict user click behavior for different candidate documents, thereby providing more accurate query results or search results.

[0137] In one embodiment, the click-through rate is obtained by processing the interaction feature vector using a classification model, and the method further includes:

[0138] Obtain the training dataset; the training dataset includes training pairs and their respective labels; the training pairs include historical query information and historical candidate media information; the labels are used to identify the click status of the training pairs;

[0139] Using training pairs and their respective labels, the initial large language model and the initial classification model are jointly trained until the loss function meets the preset conditions, at which point training stops, resulting in the large language model and the classification model.

[0140] The click status can include clicked and unclicked. Assume the sampled training dataset is: Data timerange ={(Q i D i ,Y i )|range∈[start date end date ]}, where (Q i D i ,Y i Y represents the i-th training pair, i.e., the query-doc pair, and its click state. i =1 indicates a click, Y i =0 indicates no click. Each element in the training dataset is a triple containing query information, document, and click label: Dataset={(Q,D,Y)|Q∈Queries,D∈Documents,Y∈{0,1}}, where Y=1 indicates a click and Y=0 indicates no click.

[0141] Specifically, during the offline phase, the server can acquire a training dataset, which includes training pairs and their respective labels. The training pairs include historical query information and historical candidate media information. Labels are used to identify the click status of the training pairs. The server can utilize the training pairs and their respective labels to perform actions such as... Figure 4 The initial large language model (i.e., the initial LLM model) and the initial classification model (i.e., the MLP) shown are jointly trained until the loss function meets a preset condition, at which point training stops, resulting in the large language model and the classification model, i.e., the joint training architecture of the LLM model's Transformer layer + MLP layer. The loss function in this embodiment can be defined as the binary cross-entropy loss function L. BCE To jointly train the model, the difference between the predicted click probability and the actual clicked tag is minimized. The specific formula for calculating the loss function can be shown in the following formula (2):

[0142] L BCE =-[ylog(P(C))+(1-y) log (1-P(C)) ] (2)

[0143] The optimization process for joint training includes:

[0144] Gradient backpropagation: by minimizing L BCE To jointly train the model, the backpropagation algorithm backpropagates the gradients, including the weights W of the MLP layer. MLP and b MLP The gradients are respectively: (∂L) BCE ) / (∂W MLP ) and (∂L BCE ) / (∂b MLP ); the parameter θ of each transformer layer in LLM LLM gradient: (∂L) BCE ) / (∂θ LLM These gradients will be passed to the parameters of each layer in the LLM through the chain rule.

[0145] Parameter Update: All parameters are updated using gradient descent or its variations, including the Transformer layers of MLP and LLM: W MLP ←W MLP -α (∂L BCE ) / (∂W MLP b MLP ←b MLP -α (∂L BCE ) / (∂b MLP ), θ LLM ←θ LLM - α(∂L BCE ) / (∂θ LLM That is, the server trains the model iteratively until the model's performance on the validation set no longer improves significantly, or the preset number of iterations is reached, or the loss function reaches its minimum value, at which point training stops.

[0146] In this embodiment, through this joint training method, the optimization objective of the model is to minimize the loss function L. BCE Simultaneously update the parameters of each layer in the MLP and LLM to generate higher-quality embedding vectors q. emb and d emb And achieve more accurate CTR prediction through a well-trained MLP layer.

[0147] In one embodiment, the method further includes:

[0148] Retrieve search history logs within a preset time range; the search history logs include historical query information, historical candidate media information, and click status.

[0149] The training dataset is obtained by randomly sampling historical query information and historical candidate media information as training pairs; wherein, during the random sampling process, the number of times the historical query information is sampled is limited by a preset sampling number threshold.

[0150] The search history log can be an online click search log, which includes the user's input query information, the corresponding document, and the document's historical click status.

[0151] Specifically, the server can select data from the logs within a specific time range for sampling, for example, selecting data from the most recent month. timerange ={(Q i D i ,Y i )|range∈[start date end date In essence, after the server obtains search history logs within a preset time range, it can use the historical query information and historical candidate media information included in the search history logs as training pairs, and the click status as a label for random sampling to obtain the training dataset. During the random sampling process, the server can limit the number of times historical query information is sampled by a preset sampling threshold. For example, the frequency of high-frequency queries can be limited, reducing their proportion in the sampled dataset. The server can predefine a threshold T to limit the maximum number of samplings for high-frequency queries: `Sampled_Dataset = Filter(Sampled_Dataset, T)` where the `Filter` function is used to limit the sampling of high-frequency queries in the dataset according to the threshold T. High-frequency queries refer to frequently accessed query information, such as high-frequency search terms. This effectively improves the accuracy of predicted click-through rates for each candidate media information, better predicts user satisfaction with the displayed media information, and thus enhances the user experience.

[0152] In one embodiment, after determining the click-through rate of candidate media information based on the interaction feature vector of the candidate media information, the method further includes:

[0153] The ranking results are obtained by processing the click-through rate using a ranking model; or

[0154] The ranking results are determined based on click-through rate, historical click-through rate, and relevance score.

[0155] Candidate media information is displayed based on the ranking results.

[0156] The ranking result refers to the ranking result determined based on the predicted click-through rate of each candidate media information. For example, if the predicted click-through rate P1 of candidate media information A is greater than the predicted click-through rate P2 of candidate media information B, then candidate media information A may be ranked ahead of candidate media information B in the ranking result.

[0157] Historical click-through rate (CTR) refers to the historical click-through rate of each candidate media information. For example, if the historical CTR P1 of candidate media information A is greater than the historical CTR P2 of candidate media information B, it means that in the case of querying that specific query information, candidate media information A was clicked more times by users than candidate media information B. The server may display candidate media information A before candidate media information B, thereby better predicting user satisfaction with the search results and improving user experience.

[0158] The relevance score is a value used to reflect the relevance between candidate media information and the query information in the query request. For example, if the relevance score P1 of candidate media information A is greater than the relevance score P2 of candidate media information B, it means that candidate media information A is more likely to be the result of the user's query when querying the specific query information. Therefore, the server may display candidate media information A before candidate media information B, thereby better predicting the user's satisfaction with the search results and improving the user experience.

[0159] Specifically, assuming the server uses, for example Figure 4 The trained classification model (i.e., the MLP layer) shown classifies the interaction feature vector f_vec between the query embedding vector query_emb and the document embedding vectors doc_emb of different candidate documents. After obtaining the predicted click-through rates P1 and P2 for candidate document A and candidate document B, respectively, the server can process the predicted click-through rates (P1, P2) of each candidate document using a ranking model. If the ranking result is P1 greater than P2, the server can then display each candidate document based on the ranking result. That is, the server can display the candidate documents as follows: Figure 3 The search page shown displays candidate document A before candidate document B.

[0160] Alternatively, the server can determine the overall ranking result based on the predicted click-through rate (P1, P2), historical click-through rate (P'1, P'2), and relevance score (B1, B2) of each candidate document, and then display the candidate media information based on the overall ranking result. This effectively improves the accuracy of the predicted click-through rate of each candidate media information, and can better predict user satisfaction with the displayed media information, thereby enhancing the user experience.

[0161] This application also provides an application scenario in which the above-mentioned method for determining the click-through rate of media information is applied. Specifically, the application of the method for determining the click-through rate of media information in this application scenario is as follows:

[0162] During user interaction with a multimedia information platform (or social application), the aforementioned method for determining media information click-through rates can be used. When a user (the target of the operation) wants to query specific information, the user can trigger an action to open a multimedia application on their terminal. That is, the user can log in to a multimedia application (such as a social application) through a trigger action. Furthermore, the user can initiate a query request for specific information on the page displayed in the multimedia application through a trigger action. When the backend server of the multimedia application receives the query request for specific information initiated by the user, the backend server can search for media information based on the specific query information in the query request to obtain candidate media information. The method involves determining the query embedding vector for a specific query; this query embedding vector is obtained by mapping the context information of the specific query information using a large language model. Further, the backend server can determine the interaction feature vector between the query embedding vector and each candidate media information based on the query embedding vector and the media embedding vector of each candidate media information; each media embedding vector is also obtained by mapping the context information of each candidate media information using a large language model. Based on the interaction feature vector between the query embedding vector and each candidate media information, the backend server determines the click-through rate of each candidate media information and displays each candidate media information in a specific display order according to its click-through rate. This allows the media information click-through rate determination method provided in this application to generate interaction feature vectors based on more accurate query embedding vectors for the query information and media embedding vectors for the candidate media information, and then uses a classification model to process the interaction feature vectors. This makes the model more accurate and reliable in distinguishing between clicked and non-clicked behaviors. In practical applications, this means that it can more effectively predict user click behavior, thereby providing more accurate search results and effectively improving the accuracy of media information display results for different query requests.

[0163] The method provided in this application can be applied to search engine and information retrieval tasks in various scenarios. The following example illustrates the method for determining the click-through rate of media information provided in this application.

[0164] Query: Query terms in the search engine.

[0165] Doc: Short for document, referring to the documents in the search library.

[0166] CTR (Click Through Rate) is short for the ratio of the number of times a document is clicked to the number of times the document is displayed.

[0167] NLP (Natural Language Processing): Natural Language Processing is used to enable computers to understand, parse, and generate human language.

[0168] Transformer: A deep learning model primarily used for processing sequential data.

[0169] LLM (Large Language Model): A large language model is an artificial intelligence model that can generate coherent and meaningful text.

[0170] Embedding: An embedding technique that transforms discrete variables (such as words) into continuous vectors.

[0171] ROC (Receiver Operating Characteristic): The ROC curve is a graphical tool for depicting the performance of a classifier. By using the true positive rate as the vertical axis and the false positive rate as the horizontal axis, it shows the trade-off between the sensitivity and specificity of the classifier at different thresholds.

[0172] AUC (Area Under the Curve): The area under the ROC curve (the curve between the true positive rate and the false positive rate), used to measure the overall performance of the classifier at different thresholds.

[0173] Traditional technical solutions include:

[0174] Traditional click-through rate (CTR) prediction techniques can be mainly categorized as follows:

[0175] 1. Logistic Regression (LR) Based Method: Logistic regression is a linear model widely used in CTR prediction. By inputting feature vectors into the logistic regression model, the probability of a user clicking is calculated.

[0176] 2. Factorization Machine (FM): Factorization machine is a method that extends linear models by capturing the interactions between features. FM improves the predictive power of the model by introducing latent vectors to represent feature interactions.

[0177] 3. Ensemble Learning Methods: Ensemble learning methods improve CTR prediction performance by combining multiple base models. Typical ensemble learning methods include Gradient Boosting Decision Tree (GBDT) and Random Forest (RF). These methods can effectively reduce the risk of overfitting and improve the model's generalization ability.

[0178] 4. Reinforcement Learning Methods: Reinforcement learning methods model the CTR prediction problem as a Markov decision process, enabling dynamic prediction of user click behavior. This method can learn optimal strategies through continuous interaction with the environment, adapting to changes in user behavior.

[0179] The disadvantages of traditional technologies include:

[0180] 1. Logistic Regression (LR) based methods:

[0181] a) Feature engineering dependency: The performance of logistic regression models depends heavily on feature engineering, which requires manual selection and combination of features. This is very time-consuming and difficult to scale when the feature dimension is high.

[0182] b) Sparse feature processing: For large-scale sparse data, logistic regression may not be the most effective model because it is not specifically optimized for sparsity.

[0183] c) Limitation of nonlinear relationships: Logistic regression is a linear model and cannot capture the nonlinear relationships and complex interaction effects between features very well.

[0184] 2. Factorization Machine (FM): FM has high computational complexity when dealing with large-scale data and high-dimensional features.

[0185] a) Higher-order interaction limitations: Although FM can automatically perform second-order feature crossing, its representational ability is limited for higher-order interaction effects.

[0186] b) Computational complexity: As the number of features increases, the number of parameters in FM grows quadratically, which may lead to increased computational complexity and memory requirements.

[0187] c) Overfitting risk: FM may overfit when the feature dimension is very high, especially when there are insufficient training samples.

[0188] 3. Ensemble learning method:

[0189] a) Model complexity: Ensemble learning methods, such as random forests or gradient boosting trees, can improve prediction accuracy, but the models are usually more complex and difficult to interpret and maintain.

[0190] b) Training time: Ensemble models typically require more training time because they need to build multiple base models and combine them.

[0191] c) Parameter tuning: Ensemble learning models usually have a large number of hyperparameters that need to be tuned, and finding the optimal combination of parameters may require a lot of experiments and time.

[0192] 4. Reinforcement learning method:

[0193] a) Data sparsity problem: Reinforcement learning may face the data sparsity problem in CTR prediction, especially in the exploration phase, which may lead to low learning efficiency.

[0194] b) Stability and convergence: Reinforcement learning algorithms may be unstable during training and have difficulty converging to the ideal policy.

[0195] c) Real-world application limitations: Reinforcement learning may be limited in practical CTR prediction applications due to the availability of online learning environments and real-time feedback.

[0196] In summary, traditional CTR prediction methods, such as logistic regression and factorization machines, often fail to fully capture the complex semantic relationships between user queries and documents, thus affecting the accuracy of click-through rate predictions.

[0197] The technical solution provided in this application aims to solve some key problems existing in traditional click-through rate (CTR) prediction technology, specifically including:

[0198] The technical solution provided in this application, by utilizing the Transformer structure of Large Language Models (LLM), can gain a deeper understanding of the semantics of queries and documents, thereby generating more accurate representation embeddings. Secondly, traditional methods typically require manual feature design and selection, which is not only time-consuming and labor-intensive but may also fail to capture some hard-to-discover key features. In contrast, the solution provided in this application automatically learns the embeddings of queries and documents, using only serialized text as input, thus avoiding this manual feature engineering process and discovering richer features.

[0199] Furthermore, this application trains the model using massive amounts of data by simulating real online click scenarios, enabling the model to better adapt to the actual online environment and thus improve the accuracy of CTR prediction. It also enhances the model's generalization ability, allowing it to adapt to various query and document types, thereby providing a more efficient search experience.

[0200] In summary, the method provided in this application combines the semantic understanding capabilities of large language models, automatic feature learning, massive data training, and strong generalization ability, offering an effective solution to the problems in traditional CTR prediction techniques and has already yielded benefits in practice.

[0201] The core of this application lies in proposing a click-through rate (CTR) prediction method based on Large Language Model (LLM). This method first utilizes LLM as an encoder to generate embeddings, transforming queries and documents from search engines into high-quality representation embeddings. Next, an interaction feature vector is generated by allowing the query embedding and the document embedding to interact. Then, the interaction feature vector is trained using historical click data to ultimately obtain a binary classification model used to predict the probability of a user clicking on a search result.

[0202] To further enhance the encoder's representation capabilities and click-through rate (CTR) prediction capabilities, and considering actual online prediction, this application employs an LLM architecture and a lightweight MLP layer for classification processing. This improves the accuracy of CTR prediction and effectively compensates for the shortcomings of traditional CTR prediction methods in predicting click-through rates, providing users with a more accurate and efficient search experience. The technical solution provided in this application has strong versatility and scalability, and can be widely applied to search engines and information retrieval tasks in various scenarios.

[0203] On the product side, the method provided in this application can be applied to, for example... Figure 3 The search products of various social applications shown, for example... Figure 3 Taking the interface shown as an example, when a user enters a query, the search engine retrieves the top N documents from the database as candidate results (N is set according to the search scenario; for example, N is set to 300 for article searches). Then, it uses features such as historical CTR scores, relevance scores, and quality scores for comprehensive ranking. Figure 3 For example, when a user opens the search entry, enters "how to buy real honey plums" to expand the search, and selects article vertical search, they can see the final sorted doc results. Among them, the CTR score is an important feature of the final sorting model, and the docs that rank highly are often those with high historical click rates.

[0204] On the technical side, 1.1 Overall Process

[0205] like Figure 4 The diagram shown is a schematic representation of the overall process of this application mechanism. Here, we first provide an overview of the usage flow of this method, while the algorithm details are detailed in the implementation section.

[0206] 1. Embedding Generation: In the offline phase, the server can use a pre-trained Large Language Model (LLM) to process each document in the document library, generating embedding vectors for each document. Simultaneously, for high-frequency queries, the server also pre-computes and stores their query embeddings.

[0207] 2. Model Training: The server uses historical click data to generate an LLM + MLP-based model through a training process. This model can calculate the click probability based on the embedding vectors of the query and document.

[0208] 3. Vector Retrieval Database: After processing by the LLM, the server stores the pre-calculated document embedding vectors and query embedding vectors in the vector retrieval database for fast retrieval.

[0209] 4. Real-time query processing: When a user initiates a query request, if the query embedding vector of the query information in the query request can be directly obtained from the vector library, the server will obtain it directly; otherwise, the system, i.e., the server, will process the query information in real time using the LLM model to generate the corresponding query embedding vector.

[0210] 5. MLP Inference: The server inputs the real-time generated query embedding vector and the retrieved document embedding vector into the trained MLP layer for inference calculation to obtain the click score of each document, i.e., the predicted click-through rate (click score).

[0211] 6. Ranking Model Fusion: The server incorporates the calculated click scores into the online ranking model, and performs the final ranking calculation together with other features to determine the display order of search results (the various documents retrieved).

[0212] 1.2 Algorithm Technical Details

[0213] Generally, vector generation uses the embedding corresponding to the token to represent the document.

[0214] In this method, the embodiments of this application use a large LLM, which can actually be Llama, to encode the query and document. LLM is based on the Transformer architecture and can capture long-distance dependencies in text. For a given input sequence X={x1,x2,…,x…} n The goal of LLM is to maximize the conditional probability P(x). i+1 |x1,x2,…,x i ), where (i = 1,2,…,n-1). In this process, LLM learns the contextual information of the text through a multi-layer Transformer encoder and maps each token to a high-dimensional vector space, such as... Figure 8 The diagram shown is a schematic of the Transformer layer in an LLM.

[0215] To represent the entire document, this method selects the last token generated by the LLM (here, the special terminator "[EOS]") as the embedding of the document from the output of the last Transformer layer. This embedding captures the context information of the entire document. The specific steps are as follows:

[0216] Step 1: Large-scale data sampling

[0217] 1. Data source definition: Define the data source as online click search logs, which include user queries, corresponding documents, and their historical click status.

[0218] 2. Time Range Selection: Select data from the logs within a specific time range for sampling. For example, select data from the most recent month: Data timerange ={(Q i D i ,Y i )|range∈[start date end date ]}, where (Q i D i ,Y i Y represents the i-th training pair, i.e., the query-doc pair, and its click state. i =1 indicates a click, Y i =0 indicates that it was not clicked.

[0219] 3. Random Sampling: Randomly sample query-doc pairs within a selected time range to form the training dataset: SampledDataset={(Q j D j ,Y j )∣(Q j D j ,Y j )∼Data timerange}

[0220] 4. Deduplication and frequency control:

[0221] a. Do not perform deduplication on the sampled query-doc pairs, expecting the model to automatically learn the probabilities of historical classics;

[0222] b. Limit the frequency of high-frequency queries to reduce their proportion in the sampled dataset. Define a threshold T to limit the maximum number of samples for high-frequency queries: Sampled_Dataset = Filter(Sampled_Dataset, T) where the Filter function is used to limit the sampling of high-frequency queries in the dataset according to the threshold T.

[0223] 5. Data Scale: Through the above sampling method, a large-scale training dataset of hundreds of millions of data points is generated, providing the model with enough samples to learn the patterns of click behavior.

[0224] 6. Dataset partitioning: The sampled dataset is further divided into training set, validation set and test set to facilitate model training and evaluation: Dataset = TrainSet ∪ ValidationSet ∪ TestSetDataset, where each set is randomly sampled from SampledDataset according to a certain proportion.

[0225] 7. Mathematical representation of the dataset: Ultimately, the dataset can be represented as a set where each element is a triple containing query Q, document D, and clicked label Y: Dataset={(Q,D,Y)|Q∈Queries,D∈Documents,Y∈{0,1}}.

[0226] Step 2: Model Embedding Acquisition

[0227] 1. Model Definition: Define a large language model (LLM), such as an Llama model, that can receive text input and generate corresponding embedding representations. An LLM can be represented as a function f. LLM This function maps the input text T to an embedding vector: e=f LLM (T).

[0228] 2. Hidden Layer Embedding: LLM models typically consist of multiple layers of transformers, each layer generating an embedding representation of the input. This method focuses on the output of the last hidden layer H, which can be represented as: Where L represents the number of layers in the model, Let represent the forward propagation function of the Lth layer of the model.

[0229] 3. Token-Specific Embeddings: In LLM, each token (e.g., a word or character) is mapped to an embedding vector. For each token t in Q and D, its embedding representation e on the hidden layer H can be obtained. t =H(t).

[0230] 4. EOS Embedding: In particular, the embedding representation of the end marker (EOS) can be represented using e. EOS In natural language processing, EOS (Embedded EOS) is typically used to indicate the end of a sentence. For query Q and document D, we obtain their respective EOS embeddings: q emb =e EOS (Q), d emb=e EOS (D)

[0231] Step 3: Feature Interaction and Training

[0232] 1. Feature Vector Construction: As mentioned earlier, the server can embed the query into q emb Document embedding d emb and the absolute value of the difference between them |q emb -d emb |, concatenate these three vectors to form the feature vector f. vec That is: f vec =[q emb ;d emb ;|q emb -d emb |].

[0233] 2. MLP (Classifier) ​​Fitting: The server uses a Multilayer Perceptron (MLP) to fit the feature vector f. vec After processing, we get z=MLP(f vec The server then obtains the click probability P(C) = σ(Wz + b), where σ is the softmax function, and W and b are the weight and bias terms, respectively. Here, the binary softmax can be simplified to σ(z) = 1 / (1 + e^(wz + b)). -z ).

[0234] 3. Loss Function Definition: Define the binary cross-entropy loss function L. BCE The model is trained to minimize the difference between the predicted click probability and the actual clicked tag 𝑦. The specific calculation method is shown in the aforementioned formula (2).

[0235] 4. Optimization process:

[0236] a. Gradient backpropagation: By minimizing L... BCE To jointly train the model, the backpropagation algorithm backpropagates the gradients, including the weights W of the MLP layer. MLP and b MLP The gradients are respectively: (∂L) BCE ) / (∂W MLP ) and (∂L BCE ) / (∂b MLP ); the parameter θ of each transformer layer in LLM LLM gradient: (∂L) BCE ) / (∂θ LLM These gradients will be passed to the parameters of each layer in the LLM through the chain rule.

[0237] b. Parameter Update: Use gradient descent or its variants to update all parameters, including the Transformer layers of MLP and LLM layers: W MLP ←W MLP -α (∂L BCE ) / (∂W MLP b MLP ←b MLP -α (∂L BCE ) / (∂b MLP ), θ LLM ←θ LLM - α(∂L BCE ) / (∂θ LLM ).

[0238] 5. Iterative training: Repeat the above steps until the model's performance on the validation set no longer improves significantly, or the preset number of iterations is reached.

[0239] 6. Optimization Objective: In this way, the optimization objective of the model is to minimize the loss function L. BCE Simultaneously update the parameters of the MLP layer and LLM to generate higher quality embeddings. emb and d emb And achieve more accurate CTR prediction through a well-trained MLP layer.

[0240] Step 4: Prediction

[0241] 1. Embedded reading of recalled documents: For the recalled document set {D} j The server uses pre-computed and stored document embeddings {d} embj}

[0242] 2. Online query embedding generation: For a given query Q, the server retrieves a pre-calculated q. emb If it is not pre-calculated, its embedding q is generated using an LLM model. emb .

[0243] 3. CTR Predicted Score Calculation: For each recalled document D j The server calculates its interaction feature vector f with query Q. vecj The predicted CTR score is obtained by processing the data through an MLP layer and a softmax layer. j In reality, there is a comprehensive ranking tree model online, CTR. j The scores are then fed into the final tree model for sorting and final display.

[0244] It is understood that, regarding the technical solution provided in this application, the following alternative solutions may also be considered to expand the innovative solution and the scope of protection:

[0245] 1. Multimodal feature fusion: If a document contains images, videos, or other non-text elements, multimodal learning methods can be used to combine textual and non-textual features to generate richer embedding vectors;

[0246] 2. Different interactive feature generation methods: Instead of simple concatenation and interpolation calculation, more complex interactive patterns, such as attention mechanisms or graph neural networks, can be used to capture the complex relationship between query and document;

[0247] 3. Different sampling strategies: In addition to limiting the sampling of high-frequency queries, weighted sampling methods based on object behavior or document popularity can be considered to more effectively reflect users' actual click tendencies;

[0248] 4. Context-aware CTR prediction: Incorporating contextual factors such as users' historical click data, time information, and location information into the model to achieve more accurate personalized CTR prediction;

[0249] 5. Different fitting models: In addition to MLP, after obtaining the query emb and document emb, other types of machine learning models can be considered, such as decision trees, random forests, gradient boosting machines (GBM) or support vector machines (SVM) for CTR prediction.

[0250] The beneficial effects of the technical solution in this application include:

[0251] The technical solution provided in this application significantly improves the accuracy of CTR prediction in search engines by combining the advanced features and optimization strategies of LLM.

[0252] 1. High semantic representation capability:

[0253] The technical solution provided in this application utilizes query embeddings and document embeddings generated by a Large Language Model (LLM), possessing extremely high semantic representation capabilities. Compared to traditional embedding methods, LLM can gain a deeper understanding of the complexity and diversity of language, thus providing richer and more accurate semantic information. Due to the deep learning capabilities of LLM, it can capture subtle differences and deep semantic relationships in queries and documents, enabling the generated embedding vectors to more accurately reflect user intent and document content.

[0254] 2. Significantly improves the AUC metric for CTR prediction:

[0255] By using high-quality embedding vectors generated by LLM, the CTR prediction method provided in this application achieves a significant improvement in the AUC (Area Under the Receiver Operating Characteristic Curve) metric. As an important indicator for evaluating the performance of classification models, an improvement in AUC indicates that the model has higher accuracy and reliability in distinguishing between clicks and non-clicks. A high AUC value means that the model performs well across all possible classification thresholds, which in practical applications means it can more effectively predict user click behavior, thereby providing more accurate search results.

[0256] 3. Improve user satisfaction and search engine efficiency:

[0257] User clicks are typically considered a direct reflection of search result satisfaction. This application improves the accuracy of CTR prediction, enabling better prediction of user satisfaction with search results and thus enhancing user experience. In search engine ranking mechanisms, CTR, as a crucial feature, plays a key role in the final search result ranking. The technical solution provided in this application, by optimizing CTR prediction, helps search engines more accurately evaluate and rank search results, thereby providing more relevant and higher-quality results.

[0258] 4. Promote the optimization of search engine ranking strategies:

[0259] Thanks to improvements in CTR prediction, search engines can more accurately identify and rank content that is more likely to attract user clicks, which helps improve the overall efficiency and display effectiveness of search engines. The technical solution provided in this application offers search engines a more scientific ranking basis, enabling them to better meet users' search needs while providing more valuable display opportunities for content providers and advertisers.

[0260] 5. Universality and scalability:

[0261] The technical solution provided in this application is not only applicable to specific search engines or information retrieval tasks, but also possesses excellent versatility and scalability. This means it can be applied to various search scenarios and fields, providing higher-quality search services to a wide range of users.

[0262] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0263] Based on the same inventive concept, this application also provides a media information click-through rate determination device for implementing the above-described media information click-through rate determination method. The solution provided by this device is similar to the solution described in the above-described method; therefore, the specific limitations of one or more media information click-through rate determination device embodiments provided below can be found in the limitations of the media information click-through rate determination method described above, and will not be repeated here.

[0264] In one embodiment, such as Figure 9 As shown, a device for determining the click-through rate of media information is provided, comprising: a search module 902 and a determination module 904, wherein:

[0265] The search module 902 is used to search for media information based on the query information in the query request and obtain candidate media information.

[0266] The determination module 904 is used to determine the query embedding vector of the query information; the query embedding vector is obtained by mapping the context information of the query information using a large language model; based on the query embedding vector and the media embedding vector of the candidate media information, the interaction feature vector of the candidate media information is determined; the media embedding vector is obtained by mapping the context information of the candidate media information using the large language model; based on the interaction feature vector of the candidate media information, the click-through rate of the candidate media information is determined.

[0267] In one embodiment, the apparatus further includes: an acquisition module, configured to acquire hot query information with a query count greater than a threshold; the hot query information includes query information in the query request; a processing module, configured to perform mapping processing on the hot query information through the large language model to obtain a query embedding vector of the hot query information; an storage module, configured to store the query embedding vector of the hot query information into a first database; and a query module, configured to query the query embedding vector of the query information from the first database.

[0268] In one embodiment, the processing module is further configured to perform real-time mapping processing on the query information through the large language model to obtain the query embedding vector of the query information when the query embedding vector of the query information is not found.

[0269] In one embodiment, the large language model includes a multi-layer encoder; the processing module is further configured to perform real-time mapping processing on the query information through the multi-layer encoder to obtain the embedding vectors output by the multi-layer encoder respectively; the device further includes: a determination module, configured to determine a target embedding vector from the embedding vectors output by the multi-layer encoder respectively, and use the target embedding vector as the query embedding vector of the query information.

[0270] In one embodiment, the acquisition module is further configured to acquire the embedding vector output by the last coding layer from the embedding vectors output by the multi-layer encoders respectively, as the target embedding vector; the search module is further configured to search for the embedding vector with an end marker from the embedding vectors output by the multi-layer encoders respectively, as the target embedding vector.

[0271] In one embodiment, the apparatus further includes: a processing module, configured to serialize the plain text document to obtain a serialized document when the document in the second database is a plain text document; and to map the serialized document using the large language model to obtain a document embedding vector of the plain text document; a storage module, configured to store the document embedding vector of the plain text document into a first database; and a search module, further configured to search for media information in the second database based on the query information in the query request.

[0272] In one embodiment, the processing module is further configured to, when the document in the second database is a non-plain text document, process the non-text elements in the non-plain text document using a multimodal model to obtain non-text features; and process the text elements in the non-plain text document using the large language model to obtain text features; the apparatus further includes: a generation module, configured to generate a document embedding vector of the document based on the non-text features and the text features; and a storage module, configured to store the document embedding vector of the document in the first database.

[0273] In one embodiment, the candidate media information includes candidate documents; the media embedding vector includes a document embedding vector; the apparatus further includes: an acquisition module, configured to acquire the document embedding vector of the candidate document from the first database; and a determination module, further configured to determine the interaction feature vector of the candidate document based on the query embedding vector and the document embedding vector of the candidate document.

[0274] In one embodiment, the determining module is further configured to determine the difference between the query embedding vector and the document embedding vector of the candidate document; determine the interaction feature vector of the candidate document based on the query embedding vector, the document embedding vector of the candidate document and the difference; and the processing module is further configured to perform feature processing on the query embedding vector and the document embedding vector of the candidate document through a neural network model to obtain the interaction feature vector of the candidate document.

[0275] In one embodiment, the apparatus further includes a processing module, configured to classify the interaction feature vector of the candidate media information using a classification model to obtain the click-through rate of the candidate media information.

[0276] In one embodiment, the processing module is further configured to perform feature processing on the interaction feature vector of the candidate media information through the hidden layer of the classification model to obtain the click vector of the candidate media information; the determining module is further configured to determine the click-through rate of the candidate media information based on the click vector, weight coefficients and bias terms.

[0277] In one embodiment, the click-through rate is obtained by processing the interaction feature vector through a classification model. The device further includes: an acquisition module for acquiring a training dataset; wherein the training dataset includes training pairs and labels for each training pair; the training pairs include historical query information and historical candidate media information; the labels are used to identify the click status of the training pairs; and a training module for jointly training an initial large language model and an initial classification model using the training pairs and their respective labels, until training stops when the loss function meets a preset condition, thereby obtaining the large language model and the classification model.

[0278] In one embodiment, the acquisition module is further configured to acquire search history logs within a preset time range; wherein the search history logs include the historical query information, the historical candidate media information, and the click status; the device further includes: a sampling module, configured to randomly sample the historical query information and the historical candidate media information as the training pair to obtain the training dataset; wherein, during the random sampling process, the sampling frequency of the historical query information is limited by a preset sampling frequency threshold.

[0279] In one embodiment, the apparatus further includes: a processing module, configured to process the click-through rate using a ranking model to obtain a ranking result; a determining module, configured to determine the ranking result based on the click-through rate, historical click-through rate, and relevance score; and a display module, configured to display the candidate media information based on the ranking result.

[0280] The various modules in the aforementioned media information click-through rate determination device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0281] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores click-through rate (CTR) data for media information. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for determining the CTR of media information.

[0282] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0283] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0284] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0285] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0286] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0287] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0288] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0289] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for determining the click-through rate of media information, characterized in that, The method includes: Based on the query information in the query request, media information is searched to obtain candidate media information; Determine the query embedding vector of the query information; the query embedding vector is obtained by mapping the context information of the query information using a large language model; Based on the query embedding vector and the media embedding vector of the candidate media information, the interaction feature vector of the candidate media information is determined; the media embedding vector is obtained by the large language model mapping the context information of the candidate media information. The click-through rate of the candidate media information is determined based on the interaction feature vector of the candidate media information.

2. The method according to claim 1, characterized in that, Before performing media information search based on the query information in the query request to obtain candidate media information, the method further includes: Retrieve hot query information that has been queried more than a threshold; the hot query information includes the query information in the query request. The hot query information is mapped using the large language model to obtain the query embedding vector of the hot query information; The query embedding vector of the hotspot query information is stored in the first database; Determining the query embedding vector of the query information includes: The query embedding vector from the first database is used to retrieve the query information.

3. The method according to claim 2, characterized in that, The method further includes: When the query embedding vector of the query information is not found, the query information is mapped in real time through the large language model to obtain the query embedding vector of the query information.

4. The method according to claim 3, characterized in that, The large language model includes a multi-layer encoder; The step of performing real-time mapping processing on the query information using the large language model to obtain the query embedding vector of the query information includes: The query information is mapped in real time by the multi-layer encoder to obtain the embedding vectors output by the multi-layer encoder respectively; The target embedding vector is determined from the embedding vectors output by the multi-layer encoders, and the target embedding vector is used as the query embedding vector of the query information.

5. The method according to claim 4, characterized in that, Determining the target embedding vector from the embedding vectors output by the multi-layer encoder includes: The target embedding vector is obtained from the embedding vectors output by the last coding layer from the embedding vectors output by the multi-layer encoders; or From the embedding vectors output by the multi-layer encoder, find the embedding vector with the end-of-line marker as the target embedding vector.

6. The method according to claim 1, characterized in that, Before performing media information search based on the query information in the query request to obtain candidate media information, the method further includes: When the document in the second database is a plain text document, the plain text document is serialized to obtain a serialized document; The serialized document is mapped using the large language model to obtain the document embedding vector of the plain text document; The document embedding vector of the plain text document is stored in the first database; The media information search based on the query information in the query request includes: Based on the query information in the query request, media information is searched from the second database.

7. The method according to claim 6, characterized in that, The method further includes: When the documents in the second database are not plain text documents, the non-text elements in the non-plain text documents are processed by a multimodal model to obtain non-text features; The text elements in the non-plain text document are processed using the large language model to obtain text features; Based on the non-text features and the text features, a document embedding vector of the document is generated, and the document embedding vector of the document is stored in the first database.

8. The method according to claim 6, characterized in that, The candidate media information includes candidate documents; the media embedding vector includes a document embedding vector; the method further includes: Obtain the document embedding vector of the candidate document from the first database; The step of determining the interaction feature vector of the candidate media information based on the query embedding vector and the media embedding vector of the candidate media information includes: Based on the query embedding vector and the document embedding vector of the candidate document, the interaction feature vector of the candidate document is determined.

9. The method according to claim 8, characterized in that, Determining the interaction feature vector of the candidate document based on the query embedding vector and the document embedding vector of the candidate document includes: Determine the difference between the query embedding vector and the document embedding vector of the candidate document; The interaction feature vector of the candidate document is determined based on the query embedding vector, the document embedding vector of the candidate document, and the difference; or, The query embedding vector and the document embedding vector of the candidate document are processed by a neural network model to obtain the interaction feature vector of the candidate document.

10. The method according to claim 1, characterized in that, Determining the click-through rate of the candidate media information based on the interaction feature vector of the candidate media information includes: The click-through rate of the candidate media information is obtained by classifying the interaction feature vectors of the candidate media information using a classification model.

11. The method according to claim 10, characterized in that, The step of classifying the interaction feature vectors of the candidate media information using a classification model to obtain the click-through rate of the candidate media information includes: The interaction feature vector of the candidate media information is processed by the hidden layer of the classification model to obtain the click vector of the candidate media information; The click-through rate of the candidate media information is determined based on the click vector, weight coefficient, and bias term.

12. The method according to claim 1, characterized in that, The click-through rate is obtained by processing the interaction feature vector using a classification model. The method further includes: Obtain a training dataset; wherein the training dataset includes training pairs and their respective labels; the training pairs include historical query information and historical candidate media information; the labels are used to identify the click status of the training pairs; Using the training pairs and their respective labels, the initial large language model and the initial classification model are jointly trained until the loss function meets a preset condition, at which point training stops, thus obtaining the large language model and the classification model.

13. The method according to claim 12, characterized in that, The method further includes: Obtain search history logs within a preset time range; wherein, the search history logs include the historical query information, the historical candidate media information, and the click status; The historical query information and the historical candidate media information are used as training pairs for random sampling to obtain the training dataset; wherein, during the random sampling process, the number of times the historical query information is sampled is limited by a preset sampling number threshold.

14. The method according to claim 1, characterized in that, After determining the click-through rate of the candidate media information based on the interaction feature vector of the candidate media information, the method further includes: The click-through rate is processed using a ranking model to obtain the ranking result; or The ranking results are determined based on the click-through rate, historical click-through rate, and relevance score. The candidate media information is displayed based on the ranking results.

15. A device for determining the click-through rate of media information, characterized in that, The device includes: The search module is used to search for media information based on the query information in the query request and obtain candidate media information. A determination module is used to determine the query embedding vector of the query information; the query embedding vector is obtained by mapping the context information of the query information using a large language model; based on the query embedding vector and the media embedding vector of the candidate media information, the interaction feature vector of the candidate media information is determined; the media embedding vector is obtained by mapping the context information of the candidate media information using the large language model; based on the interaction feature vector of the candidate media information, the click-through rate of the candidate media information is determined.

16. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 14.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 14.

18. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 14.

Images