Multimedia resource recommendation method and device, equipment and storage medium
Through the method of clustering and matching degree combined with click indicators, the efficiency and accuracy of video recommendation multimedia resources in the prior art are solved, and efficient multimedia resource recommendation is achieved.
Patent Information
- Application Number
- CN202310520861.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-09
- Publication Date
- 2024-11-12
AI Technical Summary
It is difficult for the prior art to effectively recommend multimedia resources related to videos, especially when considering the matching degree and click indicators of videos and multimedia resources while improving recommendation accuracy and efficiency.
By acquiring videos and multiple candidate multimedia resources, clustering and matching degree calculations are performed, target multimedia resources are determined in combination with click indicators, and data augmentation and recommendation model training are used to improve recommendation effect.
It improves the accuracy and efficiency of multimedia resource recommendations, ensures that the recommended results meet the video matching requirements and click indicator requirements, and reduces the calculation amount and storage space.
Smart Images

Figure CN118939815A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a method, device, equipment and storage medium for recommending multimedia resources. Background Art
[0002] With the continuous development of computer technology, people can browse videos on terminal devices such as mobile phones to enrich their entertainment life. When playing videos, terminal devices can also recommend related multimedia resources to users, such as advertisements. Therefore, how to recommend multimedia resources based on videos is a technical problem that needs to be solved. Summary of the invention
[0003] The embodiments of the present application provide a method, device, equipment and storage medium for recommending multimedia resources, which can be used to improve the recommendation effect of multimedia resources. The technical solution is as follows:
[0004] On the one hand, an embodiment of the present application provides a method for recommending multimedia resources, the method comprising:
[0005] Acquire a first video and multiple candidate multimedia resources;
[0006] Clustering the multiple candidate multimedia resources to obtain multiple multimedia resource sets and multiple cluster centers corresponding to the multiple multimedia resource sets, wherein one multimedia resource set includes at least one first candidate multimedia resource, and one multimedia resource set corresponds to one cluster center;
[0007] Performing first matching on the first video and multiple cluster centers corresponding to the multiple multimedia resource sets respectively to obtain multiple first matching degrees, where one first matching degree is used to indicate the similarity between the first video and a cluster center corresponding to a multimedia resource set;
[0008] Determine a target multimedia resource set corresponding to a maximum matching degree among the multiple first matching degrees;
[0009] Determine a target multimedia resource from among the first candidate multimedia resources in the target multimedia resource set based on a second matching degree between the first video and each first candidate multimedia resource in the target multimedia resource set and a click index of each first candidate multimedia resource in the target multimedia resource set, wherein a second matching degree is used to indicate a similarity between the first video and a first candidate multimedia resource in the target multimedia resource set;
[0010] The target multimedia resource is recommended.
[0011] In another aspect, a multimedia resource recommendation device is provided, the device comprising:
[0012] An acquisition module, used to acquire a first video and a plurality of candidate multimedia resources;
[0013] A clustering module, configured to cluster the plurality of candidate multimedia resources to obtain a plurality of multimedia resource sets and a plurality of cluster centers corresponding to the plurality of multimedia resource sets, wherein a multimedia resource set includes at least one first candidate multimedia resource, and a multimedia resource set corresponds to a cluster center;
[0014] A matching module, used to perform first matching on the first video and the multiple cluster centers corresponding to the multiple multimedia resource sets, respectively, to obtain multiple first matching degrees, wherein a first matching degree is used to indicate the similarity between the first video and the cluster center corresponding to a multimedia resource set;
[0015] A determination module, configured to determine a target multimedia resource set corresponding to a maximum matching degree among the plurality of first matching degrees;
[0016] The determination module is further used to determine a target multimedia resource from among the first candidate multimedia resources in the target multimedia resource set based on a second matching degree between the first video and each first candidate multimedia resource in the target multimedia resource set and a click index of each first candidate multimedia resource in the target multimedia resource set, wherein a second matching degree is used to indicate a similarity between the first video and a first candidate multimedia resource in the target multimedia resource set;
[0017] The recommendation module is used to recommend the target multimedia resource.
[0018] In a possible implementation manner, the device further includes:
[0019] A compression module, used for performing vector compression on the plurality of candidate multimedia resources respectively to obtain a plurality of compressed candidate multimedia resources;
[0020] The clustering module is used to cluster the compressed multiple candidate multimedia resources to obtain multiple multimedia resource sets and multiple cluster centers corresponding to the multiple multimedia resource sets.
[0021] In one possible implementation, the clustering module is used to perform a clustering process with a first number of clusters on the multiple candidate multimedia resources, divide the multiple candidate multimedia resources into the first number of multimedia resource sets, and obtain the first number of cluster centers; continue to iteratively divide the first number of multimedia resource sets to obtain a second number of multimedia resource sets and a second number of cluster centers corresponding to the second number of multimedia resource sets, until any multimedia resource set in the second number of multimedia resource sets contains a third number of candidate multimedia resources.
[0022] In a possible implementation, the determination module is used to determine, among the first candidate multimedia resources in the target multimedia resource set, at least one second candidate multimedia resource whose second matching degree satisfies the first matching condition and whose click index is greater than the first click threshold; and to use the second candidate multimedia resource whose click index satisfies the index requirement among the at least one second candidate multimedia resource as the target multimedia resource.
[0023] In a possible implementation, the acquisition module is used to acquire a target multimedia resource recommendation model, where the target multimedia resource recommendation model is trained based on click data of multimedia resources, where the click data is data for performing a data enhancement operation;
[0024] The determination module is used to determine the target multimedia resource from each first candidate multimedia resource in the target multimedia resource set by using the target multimedia resource recommendation model.
[0025] In a possible implementation, the acquisition module is used to acquire multiple sample pairs, historical click data of the multiple sample pairs, and an initial multimedia resource recommendation model, wherein any sample pair includes a video and a multimedia resource corresponding to the video;
[0026] The determination module is used to perform data enhancement on the multiple sample pairs to obtain multiple target sample pairs, wherein the data enhancement is an operation of recombining multiple videos and multiple multimedia resources in the multiple sample pairs; obtaining a target data set based on the historical click data of the multiple sample pairs and the multiple target sample pairs; and updating the initial multimedia resource recommendation model according to the target data set to obtain a recommendation model for the target multimedia resources.
[0027] In a possible implementation, the determination module is used to obtain a video set and a multimedia resource set based on the multiple sample pairs; combine the multiple videos in the video set and the multiple multimedia resources in the multimedia resource set in pairs to obtain multiple first sample pairs; and determine the target sample pair among the multiple first sample pairs.
[0028] In a possible implementation, the determination module is used to determine multiple third matching degrees of the multiple first sample pairs, one first sample pair corresponds to one third matching degree, and the third matching degree of any first sample pair is used to measure the similarity of the video and multimedia resources included in any first sample pair; based on the multiple third matching degrees of the multiple first sample pairs, a target sample pair is determined among the multiple first sample pairs.
[0029] In a possible implementation, the determination module is used to determine multiple second sample pairs corresponding to multiple third matching degrees that satisfy the second matching condition among the multiple third matching degrees, and the multiple second sample pairs belong to the multiple first sample pairs; and take sample pairs of multimedia resources whose click indicators are greater than the second click threshold among the multiple second sample pairs as target sample pairs.
[0030] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the computer device implements any of the multimedia resource recommendation methods described above.
[0031] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor so that a computer implements any of the above-mentioned methods for recommending multimedia resources.
[0032] On the other hand, a computer program product or a computer program is also provided, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes any of the above-mentioned methods for recommending multimedia resources.
[0033] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:
[0034] The method provided in the embodiment of the present application, in the process of considering the second matching degree between the first video and each first candidate multimedia resource in the target multimedia resource set, also considers the click index of each first candidate multimedia resource in the target multimedia resource set, and determines the target multimedia resource to be recommended based on these two factors. This method not only ensures the matching degree between the target multimedia resource and the first video, but also ensures the click index of the target multimedia resource, improves the accuracy of recommending the target multimedia resource, and better meets the recommendation requirements. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0036] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;
[0037] Figure 2 is a flowchart of a method for recommending multimedia resources provided in an embodiment of the present application;
[0038] Figure 3 is a flow chart of a method for determining a target multimedia resource provided by an embodiment of the present application;
[0039] Figure 4 It is a schematic diagram of a training method for a recommendation model of multimedia resources provided in an embodiment of the present application;
[0040] Figure 5 is a schematic diagram of an interface for recommending target multimedia resources provided by an embodiment of the present application;
[0041] Figure 6 It is a structural schematic diagram of a multimedia resource recommendation device provided in an embodiment of the present application;
[0042] Figure 7 It is a structural diagram of a server provided in an embodiment of the present application;
[0043] Figure 8 It is a structural diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION
[0044] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.
[0045] The following is an introduction to the terms involved in the embodiments of the present application.
[0046] Recall model: triggers as many correct results as possible from the candidate information set and feeds the results back to the downstream sorting model. In the multimedia resource recommendation method provided by the related art, taking the multimedia resource as advertisement as an example, if you want to recommend video-related advertisements based on the video, you can use the recall model to select multiple advertisements related to the video from the candidate advertisement set, and then feed these advertisements back to the downstream sorting model, and sort the advertisements through the sorting model.
[0047] Sorting model: sorts the results fed back by the recall model according to certain rules. In the multimedia resource recommendation method provided by the related art, the sorting model sorts the multiple advertisements fed back by the recall model according to specific indicators, and then selects the advertisements that meet the requirements for recommendation. For example, the indicator can be the click indicator of the advertisement.
[0048] CTR (Click-Through-Rate): refers to the click-through rate of multimedia resources, which is the quotient of the actual number of clicks on the multimedia resource and the display volume (Show Content) of the multimedia resource. Taking the multimedia resource as online advertising (picture advertising / text advertising / keyword advertising / ranking advertising / video advertising, etc.) as an example, the click-through rate of the advertisement is the quotient of the actual number of clicks on the advertisement and the display volume of the advertisement.
[0049] CPM (Cost Per Mille): refers to the cost of displaying multimedia resources one thousand times. Taking multimedia resources as advertisements as an example, CPM is the charging index of advertisements. It can be understood as the cost of displaying advertisements one thousand times, and can also be called the cost per thousand impressions. For example, if the price of a thousand impressions of an advertisement is 10 yuan, then CPM = 10.
[0050] ANNOY (Approximate Nearest Neighbors Oh Yeah): A search algorithm that aims to build a data structure that makes it easy to find the nearest neighbors of a point. For example, the data structure built by the algorithm can be a binary tree, which makes the search time complexity of each point O(logn).
[0051] The multimedia resource recommendation method provided in the embodiment of the present application can be applied to various scenarios, including but not limited to cloud technology, AI (Artificial Intelligence), ML (Machine Learning), etc.
[0052] AI is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, AI is a comprehensive technology in computer science. AI attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. AI is also about studying the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.
[0053] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.
[0054] ML is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.
[0055] Figure 1 The schematic diagram of the implementation environment of the method provided in the embodiment of the present application is shown. The implementation environment may include: a terminal 11 and a server 12.
[0056] The multimedia resource recommendation method provided in the embodiment of the present application can be executed by the terminal 11, can be executed by the server 12, or can be jointly executed by the terminal 11 and the server 12, and the embodiment of the present application does not limit this. In the case where the multimedia resource recommendation method provided in the embodiment of the present application is jointly executed by the terminal 11 and the server 12, the server 12 undertakes the main computing work and the terminal 11 undertakes the secondary computing work; or, the server 12 undertakes the secondary computing work and the terminal 11 undertakes the main computing work; or, the server 12 and the terminal 11 adopt a distributed computing architecture to perform collaborative computing.
[0057] Among them, the terminal 11 is installed with an application or webpage that can recommend multimedia resources. When the application or webpage needs to recommend multimedia resources, the method provided by the embodiment of the present application can be applied to recommend. The server 12 can store the multimedia resources that need to be recommended, and the terminal 11 can obtain the multimedia resources that need to be recommended from the server 12. Of course, the terminal 11 can also store the obtained multimedia resources.
[0058] Optionally, the terminal 11 can be any electronic product that can interact with the user through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device, such as PC (Personal Computer), mobile phone, smart phone, PDA (Personal Digital Assistant), wearable device, PPC (Pocket PC), tablet computer, smart car machine, smart TV, smart speaker, etc. The server 12 can be a server, a server cluster composed of multiple servers, or a cloud computing service center. The terminal 11 establishes a communication connection with the server 12 through a wired or wireless network.
[0059] Those skilled in the art should understand that the above-mentioned terminal 11 and server 12 are only examples, and other existing or future terminals or servers that are applicable to the present application should also be included in the protection scope of the present application and are included here by reference.
[0060] The present application embodiment provides a method for recommending multimedia resources, which can be applied to the above Figure 1 The method is executed by a computer device, which can be Figure 1 The terminal 11 shown may also be Figure 1 The server 12 shown in the figure is not limited in the present embodiment. Figure 2 As shown, the multimedia resource recommendation method provided in the embodiment of the present application includes the following steps 201 to 206.
[0061] In step 201, a first video and a plurality of candidate multimedia resources are obtained.
[0062] The embodiment of the present application does not limit the type of the first video. For example, the first video can be an entertainment video, a travel video, or a teaching video, etc. In addition, the duration of the first video can be set based on experience or flexibly according to the application scenario, and the embodiment of the present application does not limit this.
[0063] In the embodiment of the present application, the type of the candidate multimedia resource can be flexibly adjusted according to the application scenario. For example, the candidate multimedia resource can be a video, or a picture or text, etc. Taking the candidate multimedia resource as an advertisement as an example, the display type of the advertisement can be set based on experience, or can be flexibly adjusted according to the application scenario. For example, the advertisement can be a picture advertisement, a text advertisement, or a video advertisement, etc.
[0064] In addition, multiple candidate advertisements can be stored in a candidate advertisement set, that is, a candidate advertisement set includes multiple candidate advertisements. Among them, multiple candidate advertisements can belong to the same category or different categories, and this application does not limit this. Regardless of whether the multiple candidate advertisements belong to the same category or different categories, the embodiments of this application do not limit the category to which the candidate advertisements belong. Exemplarily, the category of the candidate advertisements can be entertainment, education, travel, or teaching, etc.
[0065] Exemplarily, the computer device may obtain the first video by receiving the first video uploaded by the user, or by obtaining the first video from the video platform, which is not limited in the embodiments of the present application. Similarly, taking the example of storing multiple candidate advertisements in a candidate advertisement set, the computer device may obtain the candidate advertisement set by receiving the candidate advertisement set uploaded by the user, or by obtaining the candidate advertisement set from a related website.
[0066] In a possible implementation, after the multiple candidate multimedia resources are acquired, the method further includes: performing vector compression on the multiple candidate multimedia resources respectively to obtain multiple compressed candidate multimedia resources.
[0067] In an embodiment of the present application, performing vector compression on multiple candidate multimedia resources may refer to performing vector compression on multimedia resource features of multiple candidate multimedia resources. Therefore, before performing vector compression on the features of each candidate multimedia resource among multiple candidate multimedia resources, it is necessary to first perform feature extraction on each candidate multimedia resource to obtain features corresponding to each candidate multimedia resource. The embodiment of the present application does not limit the method for extracting features of candidate multimedia resources. For example, a CNN (Convolutional Neural Networks) model may be used to extract features of multimedia resources, or an open source file containing multimedia resources and multimedia resource features corresponding to the multimedia resources may be queried from the Internet to obtain multimedia resource features corresponding to each candidate multimedia resource.
[0068] In addition, the embodiments of the present application do not limit the representation form of the multimedia resource features corresponding to each candidate multimedia resource. Exemplarily, the multimedia resource features corresponding to each candidate multimedia resource can be represented by a high-dimensional floating-point number. In this case, the multimedia resource features corresponding to each candidate multimedia resource can also be referred to as a high-dimensional floating-point feature vector corresponding to each candidate multimedia resource.
[0069] After the multimedia resource features corresponding to each candidate multimedia resource are obtained in the above manner, the multiple multimedia resource feature vectors may be compressed. Exemplarily, the multiple multimedia resource feature vectors may be compressed into random binary (or integer) hash codes or quantization codes.
[0070] The binary hash code is obtained by hashing technology, which encodes data such as text, images, and videos through a set of binary codes while maintaining the similarity of the original data. The above quantization code is obtained by quantization technology, which can reduce the length of the code and reduce unnecessary information in the feature without reducing the visual effect.
[0071] It should be noted that any of the above methods can be used to compress the feature vectors of multiple multimedia resources, and the present application embodiment does not limit this. Regardless of which of the above methods is used to compress the feature vectors of multiple multimedia resources, the storage space occupied by the feature vectors of the multimedia resources can be reduced, and the amount of calculation can also be reduced.
[0072] In step 202, a plurality of candidate multimedia resources are clustered to obtain a plurality of multimedia resource sets and a plurality of cluster centers corresponding to the plurality of multimedia resource sets.
[0073] Wherein, a multimedia resource set includes at least one first candidate multimedia resource, and a multimedia resource set corresponds to a cluster center. According to the content in step 201, if multiple candidate multimedia resources are respectively vector compressed to obtain multiple compressed candidate multimedia resources, then, in a possible implementation, the multiple candidate multimedia resources are clustered to obtain multiple multimedia resource sets and multiple cluster centers corresponding to the multiple multimedia resource sets, including: clustering the multiple compressed candidate multimedia resources to obtain multiple multimedia resource sets and multiple cluster centers corresponding to the multiple multimedia resource sets.
[0074] The embodiment of the present application does not limit the clustering method of multiple candidate multimedia resources. For example, ANNOY can be used to cluster multiple candidate multimedia resources. Among them, ANNOY is a fast similarity search algorithm suitable for practical applications. By establishing a binary tree, the complexity of the search time for each point is O (log n). According to the content in the previous step 201, the multiple multimedia resource feature vectors corresponding to the multiple candidate multimedia resources after compression are embodied in the form of multiple nodes, and one node corresponds to the feature vector of one multimedia resource. When using the ANNOY algorithm, two nodes can be randomly selected from multiple nodes, and these two nodes are used as initial center nodes. The K-Means (K-means clustering algorithm) process with a clustering number of 2 is executed, and finally two cluster center points after convergence are generated.
[0075] Subsequently, a vertical line is established between the lines connecting the two generated cluster center points, and the data space corresponding to the multiple nodes is divided into two parts (subspaces). After that, recursive iterations are continuously performed in the divided subspaces, and the subspaces are continued to be divided until there are only k nodes left in each subspace at most. By continuously performing iterative division in the above manner, the original data corresponding to the multiple candidate multimedia resources will form a binary tree structure. The bottom layer of the binary tree can be called a leaf node, which is used to record the original data nodes corresponding to the multiple candidate multimedia resources, and the other intermediate nodes of the binary tree structure can be used to express the information of the splitting hyperplane. In this binary tree structure, the positions of similar data nodes on the binary tree should be closer, and a splitting hyperplane will not divide similar data nodes into different branches of the binary tree.
[0076] Therefore, from the above content, it can be known that by clustering multiple candidate multimedia resources using the ANNOY algorithm, multiple multimedia resource sets and the cluster centers corresponding to each multimedia resource set can be obtained.
[0077] In step 203, first matching is performed on the first video and multiple cluster centers corresponding to the multiple multimedia resource sets to obtain multiple first matching degrees.
[0078] Among them, a first matching degree is used to indicate the similarity between the first video and the cluster center corresponding to a multimedia resource set. Exemplarily, any first matching degree can be positively correlated with the similarity between the first video and the cluster center corresponding to any multimedia resource set. For example, the similarity between the first video and the cluster center corresponding to any multimedia resource set is taken as the any first matching degree. The embodiment of the present application does not limit the method of calculating the similarity between the first video and the cluster center corresponding to any multimedia resource set, for example, the cosine similarity, Pearson similarity, etc. between the first video and the cluster center corresponding to any multimedia resource set is calculated.
[0079] Exemplarily, any first matching degree may also be negatively correlated with the similarity between the first video and the cluster center corresponding to any multimedia resource set. For example, taking the distance between the first video feature and the cluster center corresponding to the multimedia resource set as an example to represent the similarity, the distance between the first video feature and the feature of the cluster center corresponding to any multimedia resource set is taken as the first matching degree. The larger the distance, the lower the matching degree, that is, the distance and the matching degree are negatively correlated. The embodiment of the present application does not limit the method of calculating the distance between two features, for example, calculating the L2 distance (also called Euclidean distance) between two features, calculating the cosine distance between two text features, calculating the L1 distance (also called Manhattan distance) between two text features, etc.
[0080] In step 204, a target multimedia resource set corresponding to the maximum matching degree among the plurality of first matching degrees is determined.
[0081] In the embodiment of the present application, determining the target multimedia resource set corresponding to the maximum matching degree among multiple first matching degrees is to determine the target multimedia resource set that best matches the first video among multiple multimedia resource sets, that is, the target multimedia resource set with the maximum first matching degree. If there are multiple first matching degrees that are all maximum, then a maximum first matching degree can be randomly selected, and the multimedia resource set corresponding to the selected first matching degree is used as the target multimedia resource set.
[0082] For example, in step 202, multiple candidate multimedia resources are clustered by the ANNOY algorithm to obtain a binary tree, and after multiple first matching degrees are calculated in step 203, the process of searching for a set of multimedia resources that meet the first matching degree requirement of the first video, i.e., the maximum matching degree, can be implemented based on the binary tree structure, and the process is the process of traversing from the root node to the leaf node of the binary tree. For example, by calculating the matching degree of each intermediate node of the binary tree (the information of the splitting hyperplane) and the first video, it is determined whether the traversal process of the binary tree is to search for the left child node or the right child node of the intermediate node, thereby reducing the time complexity of the search to O(logn).
[0083] Therefore, from the above content, it can be seen that by clustering multiple candidate multimedia resources using the ANNOY algorithm, multiple multimedia resource sets and the cluster centers corresponding to each multimedia resource set can be obtained. By clustering multiple candidate multimedia resources and using the ANNOY algorithm for retrieval, the time required for recommending multimedia resources can be reduced and the recommendation efficiency can be improved.
[0084] In step 205, based on the second matching degree between the first video and each first candidate multimedia resource in the target multimedia resource set and the click index of each first candidate multimedia resource in the target multimedia resource set, the target multimedia resource is determined from each first candidate multimedia resource in the target multimedia resource set.
[0085] Among them, before determining the target multimedia resource among the first candidate multimedia resources in the target multimedia resource set, it is necessary to first determine the second matching degree between the first video and each first candidate multimedia resource in the target multimedia resource set, and a second matching degree is used to indicate the similarity between the first video and a first candidate multimedia resource in the target multimedia resource set. Exemplarily, the calculation method of the second matching degree is the same as the calculation method of the first matching degree in the previous step 203, so it is not repeated here.
[0086] In addition, it is also necessary to determine the click index of each first candidate multimedia resource in the target multimedia resource set. The click index of any first candidate multimedia resource is used to measure the click situation of the first candidate multimedia resource. In an exemplary embodiment, the click index can be the number of clicks, or it can be a click-through rate (such as click-through rate, CTR), etc. For example, the number of clicks can be the number of clicks, and it can also be the number of non-clicks, etc. The embodiment of the present application does not limit the click index.
[0087] In an exemplary embodiment, if Figure 3 As shown, the method for determining a target multimedia resource from each first candidate multimedia resource in a target multimedia resource set includes the following steps 2051 to 2052 .
[0088] Step 2051: Determine, from among the first candidate multimedia resources in the target multimedia resource set, at least one second candidate multimedia resource whose second matching degree satisfies the first matching condition and whose click index is greater than the first click threshold.
[0089] Exemplarily, the first candidate multimedia resource whose second matching degree satisfies the first matching condition refers to the first candidate multimedia resource with a higher similarity to the first video, and the first matching condition satisfied by the second matching degree can be flexibly adjusted according to the calculation method of the matching degree. Exemplarily, if any second matching degree refers to the distance between the characteristics of any first candidate multimedia resource and the characteristics of the first video, then the first candidate multimedia resource whose second matching degree satisfies the first matching condition can refer to the first candidate multimedia resource whose second matching degree is less than the distance threshold, or can refer to the first candidate multimedia resource whose second matching degree corresponds to the first n (n is a positive integer greater than 0) second matching degrees sorted from small to large among all the second matching degrees, where n is the number of first candidate multimedia resources to be obtained. Among them, the distance threshold can be set according to experience, or flexibly adjusted according to the application scenario.
[0090] Exemplarily, if any second matching degree refers to the similarity between any first candidate multimedia resource and the first video, the first candidate multimedia resource whose second matching degree satisfies the first matching condition may refer to the first candidate multimedia resource whose second matching degree is greater than the similarity threshold, or may refer to the first candidate multimedia resource whose second matching degree corresponds to the first n (n is a positive integer greater than 0) second matching degrees among all the second matching degrees sorted from large to small. The similarity threshold may be set based on experience or flexibly adjusted based on the application scenario.
[0091] In an exemplary embodiment, when calculating the click index of each first candidate multimedia resource included in the target multimedia resource set, taking the click index as the click rate, and the target multimedia resource set including K (K is an integer greater than 0) first candidate multimedia resources as an example, the click index of each first candidate multimedia resource included in the target multimedia resource set can be calculated to obtain K click rates. The embodiment of the present application does not limit the method for determining the click index of each first candidate multimedia resource. Exemplarily, the click index determination model can be used to calculate the click index of each first candidate multimedia resource.
[0092] Afterwards, at least one second candidate multimedia resource whose click index is greater than the first click threshold is determined among the K click indicators. The embodiment of the present application does not limit the size of the first click threshold. The first click threshold can be set based on experience and can also be flexibly adjusted according to the application scenario. In addition, the embodiment of the present application does not limit the form of expression of the first click threshold. The first click threshold can be an integer or a percentage, etc., as long as the form of expression of the click indicator is consistent with the form of expression of the first click threshold. Taking the click indicator as the click rate as an example, illustratively, the size of the first click threshold can be 70%, or 80%, etc.
[0093] Step 2052: taking at least one second candidate multimedia resource whose click index meets the index requirement as the target multimedia resource.
[0094] Still taking the click indicator as click rate as an example, after determining at least one second candidate multimedia resource based on the content in step 2051, taking the indicator requirement of maximizing the click indicator as an example, the second candidate multimedia resource with the largest click rate among the at least one second candidate multimedia resource is taken as the target multimedia resource.
[0095] In one possible implementation, a target multimedia resource is determined among each first candidate multimedia resource in a target multimedia resource set, including: obtaining a target multimedia resource recommendation model, the target multimedia resource recommendation model is trained based on click data of the multimedia resource, the click data is data for performing data enhancement operations; and determining the target multimedia resource among each first candidate multimedia resource in the target multimedia resource set through the target multimedia resource recommendation model.
[0096] Exemplarily, the target multimedia resource can be determined by a multimedia resource recommendation model. Figure 4 As shown, obtaining the target multimedia resource recommendation model includes the following steps 2053 to 2056.
[0097] Step 2053: Acquire multiple sample pairs, historical click data of the multiple sample pairs, and an initial multimedia resource recommendation model.
[0098] In an embodiment of the present application, any sample pair among the multiple sample pairs includes a video and a multimedia resource corresponding to the video. For any sample pair among the multiple sample pairs, taking the multimedia resource as an advertisement as an example, the historical click data of any sample pair includes but is not limited to at least one information such as the advertisement information of the advertisement in any sample pair, the click status of the advertisement, the video information of the video in any sample pair, and user information. Among them, the click status of the advertisement also indicates whether the advertisement in any sample pair is clicked (click) or not clicked (unclick). User information includes but is not limited to user name, user ID (Identity Document, identity identifier), video information of the user's historical browsing, and advertisement information of the advertisement clicked by the user in history.
[0099] In addition, obtaining multiple sample pairs means obtaining the videos and advertisements in the sample pairs and the one-to-one correspondence between the videos and advertisements. The method of obtaining the videos and advertisements can refer to the related method in the previous step 201, which will not be repeated here. In a possible implementation method, click history data can be obtained from the query log, thereby obtaining multiple sample pairs.
[0100] Step 2054: perform data enhancement on the multiple sample pairs to obtain multiple target sample pairs. Data enhancement is an operation of recombining multiple videos and multiple multimedia resources in the multiple sample pairs.
[0101] In a possible implementation, data enhancement is performed on multiple sample pairs to obtain multiple target sample pairs, including the following steps A to C (not shown in the figure).
[0102] Step A: Obtain a video set and a multimedia resource set based on multiple sample pairs.
[0103] In the embodiment of the present application, taking multimedia resources as advertisements as an example, a sample pair includes a video and an advertisement corresponding to the video, so multiple sample pairs include multiple videos and multiple advertisements. By extracting multiple videos from multiple sample pairs, a video set can be obtained. Similarly, by extracting multiple advertisements from multiple sample pairs, an advertisement set can be obtained.
[0104] For example, a batch of historical data of size n (i.e., pairs of watched videos and recommended ads) is obtained from the query log. The historical data is represented as follows:
[0105] data = {{user i , video i , (un)click i}, i = 1, 2, ..., n}
[0106] Where n is the number of each type of historical data obtained. user is user information, video is the video in the sample pair, click is the clicked situation, and unclick is the unclicked situation.
[0107] The above historical data are split into two sets: a video set videoSet and an advertisement set adSet. videoSet includes a video list list{video i}, list{video i} can also be understood as a video set. adSet includes an advertisement list list{ad i}, list{ad i} can also be understood as an advertising collection.
[0108] videoSet={list{video i}}
[0109] adSet = {list{ad i}}
[0110] Step B: Combine multiple videos in the video set and multiple multimedia resources in the multimedia resource set in pairs to obtain multiple first sample pairs.
[0111] The embodiments of the present application do not limit the number of videos in the video set and the number of advertisements in the advertisement set. For example, the number of videos and the number of advertisements can be the same or different. For example, the number of videos in the video set is b, and the number of advertisements in the advertisement set is c. b can be greater than c, b can be less than c, and b can also be equal to c. Regardless of whether b and c are the same, the number of first sample pairs obtained by combining multiple videos in the video set and multiple advertisements in the advertisement set is b×c.
[0112] For example, applying the cross join operation to the above two sets videoSet and adSet , generate data augData after data enhancement, that is, multiple first sample pairs.
[0113]
[0114] Step C: determining a target sample pair from a plurality of first sample pairs.
[0115] In a possible implementation, a method of determining a target sample pair from a plurality of first sample pairs includes the following steps 1 and 2 (not shown in the figure).
[0116] Step 1: Determine a plurality of third matching degrees of a plurality of first sample pairs, where one first sample pair corresponds to one third matching degree.
[0117] Based on the above step 5022, after obtaining b×c first sample pairs, the third matching degree of each first sample pair is determined to obtain b×c third matching degrees. Among them, one first sample pair corresponds to one third matching degree, and the third matching degree of any first sample pair is used to measure the similarity between the video and the advertisement included in any first sample pair. Exemplarily, the similarity between the video and the advertisement included in any first sample pair can be used as the third matching degree of any first sample pair, and the distance between the video features and the advertisement features included in any first sample pair can also be used as the third matching degree of any first sample pair. The above two methods of calculating the matching degree between videos and advertisements can be found in the description of the first matching degree in the previous text, and will not be repeated here.
[0118] Step 2: determining a target sample pair from among the plurality of first sample pairs based on the plurality of third matching degrees of the plurality of first sample pairs.
[0119] In a possible implementation, the implementation of step 2 may include the following steps S1 to S2 (not shown in the figure).
[0120] Step S1: determining a plurality of second sample pairs corresponding to a plurality of third matching degrees satisfying a second matching condition among a plurality of third matching degrees, wherein the plurality of second sample pairs belong to a plurality of first sample pairs.
[0121] Exemplarily, determining multiple second sample pairs corresponding to multiple third matching degrees that satisfy the second matching condition refers to determining multiple second sample pairs corresponding to third matching degrees with lower matching degrees, and the second matching condition can be flexibly adjusted according to the matching degree calculation method. Exemplarily, if any third matching degree refers to the distance between the video feature and the advertising feature included in any first sample pair, then the multiple second sample pairs that satisfy the second matching condition can refer to multiple second sample pairs whose third matching degrees are greater than the distance threshold, or can refer to multiple second sample pairs whose third matching degrees are the first n (n is a positive integer greater than 0) third matching degrees sorted from large to small among all the third matching degrees, where n is the number of second sample pairs that need to be obtained. Among them, the distance threshold can be set according to experience, or flexibly adjusted according to the application scenario.
[0122] For example, if any third matching degree refers to the similarity between the video and the advertisement included in any first sample pair, then the plurality of second sample pairs whose third matching degrees satisfy the second matching condition may refer to the plurality of second sample pairs whose third matching degrees are less than the similarity threshold, or may refer to the plurality of second sample pairs whose third matching degrees are the first n (n is a positive integer greater than 0) third matching degrees sorted from small to large among all the third matching degrees. The similarity threshold may be set based on experience or flexibly adjusted based on the application scenario.
[0123] Step S2: taking a sample pair of multimedia resources whose click index is greater than a second click threshold among the plurality of second sample pairs as a target sample pair.
[0124] Among them, the embodiment of the present application does not limit the size of the second click threshold. The second click threshold can be set according to experience, or it can be flexibly adjusted according to the application scenario. For example, taking m videos and n advertisements obtained from a batch of historical data as an example, data enhancement can help generate m x n synthetic video advertising pairs (ie, sample pairs). After listing all possible video advertising pairs, a similarity filter is used to grade the similarity according to whether it exceeds the similarity threshold, and obtain data with low similarity (or low correlation).
[0125] Step 2055: Based on the historical click data of multiple sample pairs and multiple target sample pairs, a target data set is obtained.
[0126] After obtaining the target sample pairs according to the above method, the target data set can be obtained by merging the historical click data of multiple sample pairs and multiple target sample pairs.
[0127] Step 2056: Update the initial multimedia resource recommendation model according to the target data set to obtain a recommendation model for the target multimedia resource.
[0128] After the target data set is input into the initial multimedia resource recommendation model, the initial multimedia resource recommendation model can output multimedia resources corresponding to the multiple videos respectively according to the multiple videos included in the target data set, and use the multimedia resources corresponding to the multiple videos respectively as the recommendation results. Then, the result loss is obtained based on the difference between the recommended result and the real result. Among them, the real result is a multimedia resource whose matching degree meets the matching condition and the click index also meets the index condition. The real result can be obtained by manual annotation.
[0129] In addition, the embodiment of the present application does not limit the method of obtaining the result loss. Exemplarily, the cross entropy or mean square error loss between the detection result and the true result can be used as the result loss. After obtaining the result loss, the parameters in the initial multimedia resource recommendation model are updated based on the result loss to obtain the trained initial multimedia resource recommendation model, and then it is determined whether the trained initial multimedia resource recommendation model meets the training termination condition. If the trained initial multimedia resource recommendation model meets the training termination condition, the trained initial multimedia resource recommendation model is used as the recommendation model of the target multimedia resource for recommending the target multimedia resource.
[0130] If the trained initial multimedia resource recommendation model does not meet the training termination conditions, continue to update the trained initial multimedia resource recommendation model in the manner described above, and so on, until an initial multimedia resource recommendation model that meets the training termination conditions is obtained, and the initial multimedia resource recommendation model that meets the training termination conditions is used as the recommendation model for the target multimedia resource.
[0131] Among them, the above-mentioned termination conditions for satisfying training are set according to experience, or flexibly adjusted according to the application scenario, and the embodiments of the present application do not limit this. Exemplarily, the termination conditions for satisfying training of the trained initial multimedia resource recommendation model include but are not limited to any one of the following: the number of model parameter updates executed when obtaining the trained initial multimedia resource recommendation model reaches a number threshold, the result loss is less than a loss threshold, or the result loss converges.
[0132] This model training method includes video-multimedia resource sample pairs with low matching degree but high click index in the training set for training the initial multimedia resource recommendation model. The initial multimedia resource recommendation model is trained according to the training set, so that the recommendation model of the target multimedia resource has the ability to recommend sample pairs with low matching degree but high click index, thereby improving the recommendation effect of the target multimedia resource.
[0133] In step 206, target multimedia resources are recommended.
[0134] After the target multimedia resource is determined from a plurality of first candidate multimedia resources in the target multimedia resource set, the target multimedia resource may be recommended. Figure 5 A schematic diagram of an interface for recommending target multimedia resources. Figure 5 In the figure, 501 is the first video, and the video is a food tutorial video; 502 is the target multimedia resource, and the target multimedia resource is an advertisement, and the advertisement is an ingredient recommendation advertisement.
[0135] In the related art, taking multimedia resources as advertisements as an example, advertisement recommendation based on videos is usually divided into two stages. The first stage can be called the recall stage, that is, some advertisements are selected from multiple candidate advertisements for recall. The factors considered in the recall stage are the matching degree between the video and multiple candidate advertisements, that is, the candidate advertisements whose matching degree meets the matching conditions are recalled. The second stage can be called the sorting stage, that is, the multiple recalled candidate advertisements are sorted according to the click index of the advertisements, and the target advertisement with the highest click index is recommended.
[0136] In the method of the related art, only the matching degree of the video and the candidate advertisement is considered in the recall stage, and only the click index of the candidate advertisement is considered in the sorting stage. Then, the recall stage may recall candidate advertisements with high matching degree but low click index. Such advertisements will reduce the profitability of advertisements and thus reduce the recommendation effect of target advertisements. This method can decouple the functions of the two stages when the amount of calculation is insufficient, but the decoupling between matching and sorting often leads to a relatively low CPM, which is one of the key business indicators of search engines. If only the relevance, that is, the matching degree, is considered in the recall stage, a large number of worthless recall candidate sets will be generated, which will cause a waste of resources. In addition, some advertisements that appear infrequently but may trigger high CPM may be filtered out in the recall stage because the recall does not consider the CPM indicator. However, the present application adds the judgment of business indicators (that is, click indicators) and recommendation relevance (that is, the matching of similarity, corresponding to the first and second matching degrees in the present application) in the recall stage, and adopts active learning and data compression methods to make large-scale retrieval recommendation possible, so that the advertisements screened by the algorithm can meet the relevance of the recall candidate set while meeting the business needs.
[0137] In summary, the method provided in the embodiment of the present application, in the process of considering the second matching degree between the first video and each first candidate multimedia resource in the target multimedia resource set, also considers the click index of each first candidate multimedia resource in the target multimedia resource set, and determines the target multimedia resource to be recommended based on these two factors. This method not only ensures the matching degree between the target multimedia resource and the first video, but also ensures the click index of the target multimedia resource, improves the accuracy of recommending the target multimedia resource, and better meets the recommendation requirements.
[0138] In addition, by compressing the feature vectors of multiple multimedia resources, the storage space occupied by the feature vectors of the multimedia resources can be reduced, and the subsequent calculation amount can also be reduced. Furthermore, by clustering multiple candidate multimedia resources and using the ANNOY algorithm for retrieval and search, the time required for recommending multimedia resources can be reduced and the recommendation efficiency can be improved.
[0139] See also Figure 6 , an embodiment of the present application provides a multimedia resource recommendation device, the device comprising:
[0140] A first acquisition module 601 is used to acquire a first video and a plurality of candidate multimedia resources;
[0141] A clustering module 602 is used to cluster multiple candidate multimedia resources to obtain multiple multimedia resource sets and multiple cluster centers corresponding to the multiple multimedia resource sets, where one multimedia resource set includes at least one first candidate multimedia resource and one multimedia resource set corresponds to one cluster center;
[0142] A matching module 603 is used to perform first matching on the first video and multiple cluster centers corresponding to the multiple multimedia resource sets, respectively, to obtain multiple first matching degrees, where a first matching degree is used to indicate the similarity between the first video and a cluster center corresponding to a multimedia resource set;
[0143] A determination module 604 is used to determine a target multimedia resource set corresponding to a maximum matching degree among a plurality of first matching degrees;
[0144] The determination module 604 is further used to determine the target multimedia resource from each first candidate multimedia resource in the target multimedia resource set based on the second matching degree between the first video and each first candidate multimedia resource in the target multimedia resource set and the click index of each first candidate multimedia resource in the target multimedia resource set, wherein a second matching degree is used to indicate the similarity between the first video and a first candidate multimedia resource in the target multimedia resource set;
[0145] The recommendation module 605 is used to recommend target multimedia resources.
[0146] In a possible implementation, the device further includes:
[0147] A compression module, used for performing vector compression on a plurality of candidate multimedia resources respectively to obtain a plurality of compressed candidate multimedia resources;
[0148] The clustering module 602 is used to cluster the compressed candidate multimedia resources to obtain multiple multimedia resource sets and multiple cluster centers corresponding to the multiple multimedia resource sets.
[0149] In one possible implementation, the clustering module 602 is used to perform a clustering process with a first number of clusters on multiple candidate multimedia resources, divide the multiple candidate multimedia resources into a first number of multimedia resource sets, and obtain a first number of cluster centers; continue to iteratively divide the first number of multimedia resource sets to obtain a second number of multimedia resource sets and a second number of cluster centers corresponding to the second number of multimedia resource sets, until any multimedia resource set in the second number of multimedia resource sets contains a third number of candidate multimedia resources.
[0150] In one possible implementation, the determination module 604 is used to determine, among the first candidate multimedia resources in the target multimedia resource set, at least one second candidate multimedia resource whose second matching degree satisfies the first matching condition and whose click index is greater than the first click threshold; and to use, among the at least one second candidate multimedia resource, a second candidate multimedia resource whose click index satisfies the index requirement as the target multimedia resource.
[0151] In a possible implementation, 601, an acquisition module is used to acquire a target multimedia resource recommendation model, where the target multimedia resource recommendation model is trained based on click data of multimedia resources, where the click data is data for performing data enhancement operations;
[0152] The determination module 604 is configured to determine a target multimedia resource from among the first candidate multimedia resources in the target multimedia resource set by using a multimedia resource recommendation model.
[0153] An acquisition module 601 is used to acquire multiple sample pairs, historical click data of the multiple sample pairs, and an initial multimedia resource recommendation model, wherein any sample pair includes a video and a multimedia resource corresponding to the video;
[0154] Determine module 604, which is used to perform data enhancement on multiple sample pairs to obtain multiple target sample pairs, where data enhancement is an operation of recombining multiple videos and multiple multimedia resources in the multiple sample pairs; obtain a target data set based on the historical click data of the multiple sample pairs and the multiple target sample pairs; and update the initial multimedia resource recommendation model according to the target data set to obtain a recommendation model for the target multimedia resources.
[0155] In one possible implementation, a determination module is used to obtain a video set and a multimedia resource set based on multiple sample pairs; combine multiple videos in the video set and multiple multimedia resources in the multimedia resource set in pairs to obtain multiple first sample pairs; and determine a target sample pair among the multiple first sample pairs.
[0156] In one possible implementation, a determination module is used to determine multiple third matching degrees of multiple first sample pairs, one first sample pair corresponds to one third matching degree, and the third matching degree of any first sample pair is used to measure the similarity of the video and multimedia resources included in any first sample pair; based on the multiple third matching degrees of the multiple first sample pairs, a target sample pair is determined from the multiple first sample pairs.
[0157] In one possible implementation, a determination module is used to determine multiple second sample pairs corresponding to multiple third matching degrees that satisfy a second matching condition among multiple third matching degrees, and the multiple second sample pairs belong to multiple first sample pairs; and to take sample pairs of multimedia resources whose click indicators are greater than a second click threshold among the multiple second sample pairs as target sample pairs.
[0158] The device provided in the embodiment of the present application, in the process of considering the second matching degree between the first video and each first candidate multimedia resource in the target multimedia resource set, also considers the click index of each first candidate multimedia resource in the target multimedia resource set, and determines the target multimedia resource to be recommended based on these two factors. The device not only ensures the matching degree between the target multimedia resource and the first video, but also ensures the click index of the target multimedia resource, improves the accuracy of recommending the target multimedia resource, and better meets the recommendation requirements.
[0159] In addition, by compressing the feature vectors of multiple multimedia resources, the storage space occupied by the feature vectors of the multimedia resources can be reduced, and the subsequent calculation amount can also be reduced. Furthermore, by clustering multiple candidate multimedia resources and using the ANNOY algorithm for retrieval and search, the time required for recommending multimedia resources can be reduced and the recommendation efficiency can be improved.
[0160] It should be noted that the device provided in the above embodiment only uses the division of the above functional modules as an example to implement its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.
[0161] Figure 7 It is a structural diagram of a server provided in an embodiment of the present application. The server may have relatively large differences due to different configurations or performances, and may include one or more processors 701 and one or more memories 702, wherein the processor 7801 may be a central processing unit (CPU), and the one or more memories 702 store at least one computer program, which is loaded and executed by the one or more processors 701, so that the server implements the multimedia resource recommendation method provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the server may also include other components for implementing device functions, which will not be repeated here.
[0162] Figure 8 1 is a schematic diagram of a terminal structure provided in an embodiment of the present application. The device may be, for example, a smart phone, a tablet computer, a laptop computer, or a desktop computer. The terminal may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal, or other names.
[0163] Typically, the terminal includes: a processor 1501 and a memory 1502 .
[0164] The processor 1501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1501 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1501 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1501 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
[0165] The memory 1502 may include one or more computer-readable storage media, which may be non-transitory. The memory 1502 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1502 is used to store at least one instruction, which is executed by the processor 1501 to enable the terminal to implement the multimedia resource recommendation method provided in the method embodiment of the present application.
[0166] In some embodiments, the terminal may further optionally include: a peripheral device interface 1503 and at least one peripheral device. The processor 1501, the memory 1502 and the peripheral device interface 1503 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1503 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 1504, a display screen 1505, a camera assembly 1506, an audio circuit 1507 and a power supply 1508.
[0167] The peripheral device interface 1503 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1501 and the memory 1502. In some embodiments, the processor 1501, the memory 1502, and the peripheral device interface 1503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1501, the memory 1502, and the peripheral device interface 1503 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.
[0168] The radio frequency circuit 1504 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1504 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1504 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1504 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 1504 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1504 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.
[0169] The display screen 1505 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1505 is a touch display screen, the display screen 1505 also has the ability to collect touch signals on the surface or above the surface of the display screen 1505. The touch signal can be input to the processor 1501 as a control signal for processing. At this time, the display screen 1505 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 1505 can be one, which is set on the front panel of the terminal; in other embodiments, the display screen 1505 can be at least two, which are respectively set on different surfaces of the terminal or are folded; in other embodiments, the display screen 1505 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal. Even, the display screen 1505 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1505 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).
[0170] The camera assembly 1506 is used to capture images or videos. Optionally, the camera assembly 1506 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1506 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.
[0171] The audio circuit 1507 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1501 for processing, or input them into the RF circuit 1504 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1501 or the RF circuit 1504 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1507 may also include a headphone jack.
[0172] The power supply 1508 is used to power various components in the terminal. The power supply 1508 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1508 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.
[0173] In some embodiments, the terminal further includes one or more sensors 1509 , including but not limited to: an acceleration sensor 1510 , a gyroscope sensor 1511 , a pressure sensor 1512 , an optical sensor 1513 , and a proximity sensor 1514 .
[0174] The acceleration sensor 1510 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal. For example, the acceleration sensor 1510 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 1501 can control the display screen 1505 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 1510. The acceleration sensor 1510 can also be used for collecting game or user motion data.
[0175] The gyro sensor 1511 can detect the body direction and rotation angle of the terminal, and the gyro sensor 1511 can cooperate with the acceleration sensor 1510 to collect the user's 3D actions on the terminal. The processor 1501 can implement the following functions based on the data collected by the gyro sensor 1511: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.
[0176] The pressure sensor 1512 can be set in the side frame of the terminal and / or the lower layer of the display screen 1505. When the pressure sensor 1512 is set in the side frame of the terminal, it can detect the user's holding signal of the terminal, and the processor 1501 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 1512. When the pressure sensor 1512 is set in the lower layer of the display screen 1505, the processor 1501 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 1505. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.
[0177] The optical sensor 1513 is used to collect the ambient light intensity. In one embodiment, the processor 1501 can control the display brightness of the display screen 1505 according to the ambient light intensity collected by the optical sensor 1513. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1505 is increased; when the ambient light intensity is low, the display brightness of the display screen 1505 is reduced. In another embodiment, the processor 1501 can also dynamically adjust the shooting parameters of the camera component 1506 according to the ambient light intensity collected by the optical sensor 1513.
[0178] The proximity sensor 1514, also known as a distance sensor, is usually arranged on the front panel of the terminal. The proximity sensor 1514 is used to collect the distance between the user and the front of the terminal. In one embodiment, when the proximity sensor 1514 detects that the distance between the user and the front of the terminal is gradually decreasing, the processor 1501 controls the display screen 1505 to switch from the screen-on state to the screen-off state; when the proximity sensor 1514 detects that the distance between the user and the front of the terminal is gradually increasing, the processor 1501 controls the display screen 1505 to switch from the screen-off state to the screen-on state.
[0179] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than those shown in the figure, or combine certain components, or adopt a different component arrangement.
[0180] In an exemplary embodiment, a computer device is also provided, the computer device comprising a processor and a memory, wherein at least one computer program is stored in the memory. The at least one computer program is loaded and executed by one or more processors, so that the computer device implements any of the above-mentioned multimedia resource recommendation methods.
[0181] In an exemplary embodiment, a computer-readable storage medium is further provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor of a computer device to enable the computer to implement any of the above-mentioned methods for recommending multimedia resources.
[0182] In one possible implementation, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.
[0183] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes any of the above-mentioned methods for recommending multimedia resources.
[0184] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the first video involved in this application was obtained with full authorization.
[0185] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.
[0186] It should be noted that the terms "first", "second", etc. (if any) in the specification and claims of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the above exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0187] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the protection scope of the present application.
Claims
1. A method for recommending multimedia resources, characterized in that: The method comprises: Acquire a first video and multiple candidate multimedia resources; Clustering the multiple candidate multimedia resources to obtain multiple multimedia resource sets and multiple cluster centers corresponding to the multiple multimedia resource sets, wherein one multimedia resource set includes at least one first candidate multimedia resource, and one multimedia resource set corresponds to one cluster center; Performing first matching on the first video and multiple cluster centers corresponding to the multiple multimedia resource sets respectively to obtain multiple first matching degrees, where one first matching degree is used to indicate the similarity between the first video and a cluster center corresponding to a multimedia resource set; Determine a target multimedia resource set corresponding to a maximum matching degree among the multiple first matching degrees; Determine a target multimedia resource from among the first candidate multimedia resources in the target multimedia resource set based on a second matching degree between the first video and each first candidate multimedia resource in the target multimedia resource set and a click index of each first candidate multimedia resource in the target multimedia resource set, wherein a second matching degree is used to indicate a similarity between the first video and a first candidate multimedia resource in the target multimedia resource set; The target multimedia resource is recommended.
2. The method according to claim 1, characterized in that Before clustering the multiple candidate multimedia resources to obtain multiple multimedia resource sets and multiple cluster centers corresponding to the multiple multimedia resource sets, the method further includes: Performing vector compression on the plurality of candidate multimedia resources respectively to obtain a plurality of compressed candidate multimedia resources; The clustering of the multiple candidate multimedia resources to obtain multiple multimedia resource sets and multiple cluster centers corresponding to the multiple multimedia resource sets includes: The compressed multiple candidate multimedia resources are clustered to obtain multiple multimedia resource sets and multiple cluster centers corresponding to the multiple multimedia resource sets.
3. The method according to claim 1, characterized in that The clustering of the multiple candidate multimedia resources to obtain multiple multimedia resource sets and multiple cluster centers corresponding to the multiple multimedia resource sets includes: Performing a clustering process with a first number of clusters on the multiple candidate multimedia resources, dividing the multiple candidate multimedia resources into the first number of multimedia resource sets, and obtaining the first number of cluster centers; Continue to iteratively divide the first number of multimedia resource sets to obtain a second number of multimedia resource sets and a second number of cluster centers corresponding to the second number of multimedia resource sets, until any multimedia resource set in the second number of multimedia resource sets contains a third number of candidate multimedia resources, and the second number is greater than the first number.
4. The method according to claim 1, characterized in that: The step of determining the target multimedia resource from among the first candidate multimedia resources in the target multimedia resource set comprises: Determine, from among the first candidate multimedia resources in the target multimedia resource set, at least one second candidate multimedia resource whose second matching degree satisfies the first matching condition and whose click index is greater than the first click threshold; The second candidate multimedia resource whose click index meets the index requirement among the at least one second candidate multimedia resource is used as the target multimedia resource.
5. The method according to claim 1, characterized in that The step of determining the target multimedia resource from among the first candidate multimedia resources in the target multimedia resource set comprises: Obtaining a target multimedia resource recommendation model, wherein the target multimedia resource recommendation model is trained based on click data of multimedia resources, wherein the click data is data for performing a data enhancement operation; The target multimedia resource recommendation model is used to determine the target multimedia resource from among the first candidate multimedia resources in the target multimedia resource set.
6. The method according to claim 5, characterized in that The step of obtaining a target multimedia resource recommendation model includes: Acquire multiple sample pairs, historical click data of the multiple sample pairs, and an initial multimedia resource recommendation model, wherein any sample pair includes a video and a multimedia resource corresponding to the video; Performing data enhancement on the multiple sample pairs to obtain multiple target sample pairs, wherein the data enhancement is an operation of recombining multiple videos and multiple multimedia resources in the multiple sample pairs; Based on the historical click data of the multiple sample pairs and the multiple target sample pairs, a target data set is obtained; The initial multimedia resource recommendation model is updated according to the target data set to obtain a recommendation model for the target multimedia resource.
7. The method according to claim 6, characterized in that Before obtaining the target data set based on the historical click data of the multiple sample pairs and the multiple target sample pairs obtained by data enhancement of the multiple sample pairs, the method further includes: Obtain a video set and a multimedia resource set based on the multiple sample pairs; Combine multiple videos in the video set and multiple multimedia resources in the multimedia resource set in pairs to obtain multiple first sample pairs; A target sample pair is determined from among the plurality of first sample pairs.
8. The method according to claim 7, characterized in that The step of determining a target sample pair from among the plurality of first sample pairs comprises: Determine a plurality of third matching degrees of the plurality of first sample pairs, one first sample pair corresponds to one third matching degree, and the third matching degree of any first sample pair is used to measure the similarity of the video and multimedia resources included in any first sample pair; A target sample pair is determined from among the plurality of first sample pairs based on a plurality of third matching degrees of the plurality of first sample pairs.
9. The method according to claim 8, characterized in that The step of determining a target sample pair from among the plurality of first sample pairs based on the plurality of third matching degrees of the plurality of first sample pairs comprises: Determine a plurality of second sample pairs corresponding to a plurality of third matching degrees satisfying a second matching condition among the plurality of third matching degrees, wherein the plurality of second sample pairs belong to the plurality of first sample pairs; The sample pairs including the multimedia resources whose click index is greater than the second click threshold among the multiple second sample pairs are taken as target sample pairs.
10. A multimedia resource recommendation device, characterized in that: The device comprises: A first acquisition module, used to acquire a first video and a plurality of candidate multimedia resources; A clustering module, configured to cluster the plurality of candidate multimedia resources to obtain a plurality of multimedia resource sets and a plurality of cluster centers corresponding to the plurality of multimedia resource sets, wherein a multimedia resource set includes at least one first candidate multimedia resource, and a multimedia resource set corresponds to a cluster center; A matching module, used to perform first matching on the first video and the multiple cluster centers corresponding to the multiple multimedia resource sets, respectively, to obtain multiple first matching degrees, wherein a first matching degree is used to indicate the similarity between the first video and the cluster center corresponding to a multimedia resource set; A first determination module, configured to determine a target multimedia resource set corresponding to a maximum matching degree among the plurality of first matching degrees; The determination module is further used to determine a target multimedia resource from among the first candidate multimedia resources in the target multimedia resource set based on a second matching degree between the first video and each first candidate multimedia resource in the target multimedia resource set and a click index of each first candidate multimedia resource in the target multimedia resource set, wherein a second matching degree is used to indicate a similarity between the first video and a first candidate multimedia resource in the target multimedia resource set; The recommendation module is used to recommend the target multimedia resource.
11. A computer device, characterized in that: The computer device includes a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor, so that the computer device implements the multimedia resource recommendation method according to any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by the processor, so that the computer device implements the multimedia resource recommendation method according to any one of claims 1 to 9.
13. A computer program product, characterized in that The computer program product includes computer instructions, which are loaded and executed by a processor of a computer device, so that the computer device implements the multimedia resource recommendation method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Multimedia resource recommending method and device
CN106227834A
Method and apparatus for recommending multimedia resources
CN109388739A
Multimedia resource recommendation method and device, electronic equipment and storage medium
CN113190757A
