Method and apparatus for recommending multimedia material, electronic device, and storage medium

Abstract

The application discloses a method and device for recommending multimedia materials, electronic equipment and storage medium, to solve the problem of inaccurate recommendation caused by low embedding representation efficiency of output multimedia materials. The method comprises: constructing a material relationship graph according to a plurality of first multimedia materials in a database; inputting the material relationship graph into a trained target model for feature aggregation to obtain a first embedding representation corresponding to each first multimedia material; determining a second embedding representation for representing interest characteristics of a target user according to at least one second multimedia material corresponding to the target user; determining an interest value corresponding to each third multimedia material according to the first embedding representation corresponding to each third multimedia material and the second embedding representation; and determining a target third multimedia material whose interest value meets a preset condition according to the interest value corresponding to each third multimedia material, to perform material recommendation.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and specifically to a method, apparatus, electronic device, and storage medium for recommending multimedia materials. Background Technology

[0002] With the ever-increasing volume of data on the internet, personalized recommendation systems are becoming increasingly important. These systems recommend data that users are likely to be interested in, thereby enhancing the user experience.

[0003] Personalized recommendation systems typically consist of a recall module and a ranking module. The recall module filters data from a database of millions of candidates, selecting a few thousand data points that might be of interest to users, thus forming the recall database. The ranking module scores the data in the recall database and selects the data that is most relevant to the user based on the scores, recommending it to the user.

[0004] Current personalized recommendation systems typically use Graph Convolutional Network (GCN) models in their recall modules. These models employ an inductive learning approach, requiring the capture of global information from a complete graph of user history behavior to learn the embedded representations of multimedia materials. When the recall module is applied to large-scale graphs, it consumes significant resources and has low output efficiency, impacting the accuracy of recommendations. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, and electronic device for recommending multimedia materials, so as to solve the problem of inaccurate recommendations caused by the low embedding representation efficiency of the generated multimedia materials.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows:

[0007] In a first aspect, embodiments of this application provide a method for recommending multimedia materials. The method includes: constructing a material relationship graph based on multiple first multimedia materials in a database, the material relationship graph including graph nodes and edges, wherein the graph nodes represent the first multimedia materials, and the edges represent the association relationship between two connected graph nodes; inputting the material relationship graph into a trained target model for feature aggregation to obtain a first embedding representation corresponding to each first multimedia material, the first embedding representation aggregating features of the corresponding first multimedia material and features of other first multimedia materials related to the first multimedia material; determining a second embedding representation representing the interest features of the target user based on at least one second multimedia material corresponding to the target user, the second multimedia material being the material from the multiple first multimedia materials for which the target user has historical interaction behavior; determining an interest value corresponding to each third multimedia material based on the first embedding representation and the second embedding representation, the third multimedia material being the material from the multiple first multimedia materials for which the target user has not historical interaction behavior; and determining target third multimedia materials whose interest values ​​meet preset conditions based on the interest values ​​corresponding to each third multimedia material, for material recommendation.

[0008] Secondly, embodiments of this application provide an apparatus for recommending multimedia materials, the apparatus comprising:

[0009] The first processing module is used to construct a material relationship diagram based on multiple first multimedia materials in the database. The material relationship diagram includes graph nodes and edges, wherein the graph nodes are used to represent the first multimedia materials, and the edges are used to indicate that two connected graph nodes have an association relationship.

[0010] The second processing module is used to input the material relationship diagram into the trained target model to perform feature aggregation and obtain a first embedding representation corresponding to each of the first multimedia materials. The first embedding representation aggregates the features of the corresponding first multimedia material and the features of other first multimedia materials that are related to the first multimedia material.

[0011] The first determining module is used to determine a second embedded representation for representing the interest characteristics of the target user based on at least one second multimedia material corresponding to the target user, wherein the second multimedia material is a material from a plurality of first multimedia materials in which the target user has generated historical interaction behavior;

[0012] The calculation module is used to determine the interest value corresponding to each third multimedia material based on the first embedding representation and the second embedding representation corresponding to each third multimedia material, wherein the third multimedia material is the material in which the target user has not generated historical interaction behavior among a plurality of first multimedia materials;

[0013] The second determining module is used to determine target third multimedia materials whose interest values ​​meet preset conditions based on the interest values ​​corresponding to each of the third multimedia materials, so as to recommend materials.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method for recommending multimedia materials as described in the first aspect.

[0015] Fourthly, embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the method for recommending multimedia materials as described in the first aspect.

[0016] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method of recommending multimedia materials as described in the first aspect.

[0017] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method of recommending multimedia materials as described in the first aspect.

[0018] In this embodiment, a material relationship graph is constructed from multiple first multimedia materials in the database. The graph nodes represent the first multimedia materials, and the edges represent the relationships between two graph nodes. The material relationship graph is input into a trained target model for feature aggregation to obtain a first embedding representation corresponding to each first multimedia material. This first embedding representation aggregates the features of the corresponding first multimedia material and the features of other first multimedia materials related to it. In other words, feature aggregation of graph nodes in the material relationship graph is used to create an embedding representation of a multimedia material. As the number of aggregation iterations increases, the aggregated features of each graph node approach the global features of the graph. This method increases the scalability of the material relationship graph and improves the efficiency of generating embedding representations of multimedia materials. User interest features are calculated based on the embedding representations of multimedia materials with historical interaction behavior of the target user. The user interest features are compared with multimedia materials in the database to determine the target user's interest value for the multimedia materials. This allows for the more accurate recommendation of multimedia materials that match the target user's historical interests. Attached Figure Description

[0019] Figure 1 This illustration shows a flowchart of a method for recommending multimedia materials according to an embodiment of this application;

[0020] Figure 2 This illustration shows another flowchart of the method for recommending multimedia materials provided in an embodiment of this application;

[0021] Figure 3a This is a schematic diagram showing the material relationship sub-diagram provided in an embodiment of this application;

[0022] Figure 3b This illustration shows a schematic diagram of the aggregation of material relationship diagrams provided in an embodiment of this application;

[0023] Figure 4 A flowchart illustrating the training method for the target model provided in an embodiment of this application is shown.

[0024] Figure 5 A schematic diagram of the structure of the apparatus for the recommended multimedia materials provided in the embodiments of this application is shown;

[0025] Figure 6 This diagram illustrates the hardware structure of the electronic device provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0027] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar users and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the users distinguished by "first," "second," etc., are generally of the same class, without limiting the number of users; for example, a first user may be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected users, and the character " / " generally indicates that the preceding and following users are in an "or" relationship.

[0028] The method for recommending multimedia materials provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0029] Figure 1 This illustration shows a flowchart of a method for recommending multimedia materials according to an embodiment of this application. This method can be executed by an electronic device, such as a terminal device or a server device. In other words, the method can be executed by software or hardware installed on the terminal device or server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. As shown, the method may include the following steps.

[0030] S101: Construct a material relationship diagram based on multiple first multimedia materials in the database.

[0031] The material relationship diagram includes graph nodes and edges. The graph nodes represent the first multimedia material, and the edges represent the association between two connected graph nodes.

[0032] Specifically, the first multimedia material is at least one type of data, or a combination of at least two types of data, such as photos, text, audio recordings, or videos. The graph nodes store data used to identify the features of the first multimedia material. Graph nodes with related relationships are connected by edges. These relationships can be that the corresponding first multimedia materials share at least one identical feature or have a high degree of correlation among multiple features. For example, multiple first multimedia materials uploaded or clicked by the same user are related, or text comments related to the same news article uploaded by different users are related.

[0033] S102: Input the material relationship diagram into the trained target model for feature aggregation to obtain a first embedded representation corresponding to each of the first multimedia materials.

[0034] Wherein, the first embedding means aggregating the features of the corresponding first multimedia material and the features of other first multimedia materials that are related to the first multimedia material.

[0035] Specifically, the feature aggregation generalizes the graph nodes in the material relationship diagram through an aggregator in the target model. The aggregator includes, but is not limited to, mean aggregators, inductive aggregators, LSTM (Long Short-Term Memory) aggregators, and pooling aggregators.

[0036] The first embedding representation is a low-dimensional vector representation, characterized by its ability to reduce the dimensionality of the high-dimensional adjacency information of a graph node in the material relationship graph, and to represent the features of multiple first-feature materials with related relationships in the material relationship graph in a vector space.

[0037] S103: Based on at least one second multimedia material corresponding to the target user, determine a second embedded representation for representing the target user's interest features. The second multimedia material is one of the multiple first multimedia materials from which the target user has generated historical interaction behavior.

[0038] Specifically, the historical interaction behavior can be actions performed by the user during application use, such as clicking, liking, commenting, or saving. For example, a user's historical interaction behavior on a payment platform may include, but is not limited to, user benefits that the user has clicked on, user benefits that the user has used, goods that the user has purchased through user benefits, and the transaction amount of those goods; a user's historical interaction behavior on a shopping platform may include, but is not limited to, goods for which the user has performed a specified action (such as clicking, browsing, saving, purchasing, or purchasing after clicking), and the amount spent on those goods; a user's historical interaction behavior on a video platform may include, but is not limited to, the user clicking to watch a video, sending bullet comments, commenting on, or saving a video.

[0039] Materials reflecting a user's historical interactions can reflect their preferences during those interactions. For example, in a shopping app, a user might add multiple shirts from the same clothing brand to their cart. The user's interests could be with that clothing brand or similar brands, or with shirts, business attire, etc.

[0040] Specifically, the second multimedia material is input into the target model for feature aggregation to obtain an embedding representation corresponding to each second multimedia material. This embedding representation is then used to determine the second embedding representation. Optionally, the second embedding representation can be obtained by averaging these embedding representations. For example, if the embedding representations corresponding to three second multimedia materials are (2, 4, 7, 0), (2, 0, 7, 1), and (0, 4, 2, 1), then the second embedding representation obtained by averaging these three embedding representations is (2, 4, 8, 1).

[0041] Step S104: Determine the interest value corresponding to each third multimedia material based on the first embedded representation and the second embedded representation corresponding to each third multimedia material.

[0042] The third multimedia material is a material among the multiple first multimedia materials for which the target user has not generated any historical interaction behavior. The first embedding representation has a one-to-one correspondence with the third multimedia material, and the second embedding representation corresponds to the features of the multimedia material that the target user is interested in. Specifically, the interest value of the third multimedia material can be determined using a similarity metric, that is, calculating the degree of similarity between the embedding representations of individuals. The smaller the similarity metric value, the smaller the similarity between individuals; conversely, the larger the similarity value, the greater the individual differences.

[0043] Optionally, the first embedding is represented as a first vector, and the second embedding is represented as a second vector. The similarity between the two vectors can be calculated by the vector dot product.

[0044] Step S105: Based on the interest value corresponding to each of the third multimedia materials, determine the target third multimedia materials whose interest values ​​meet the preset conditions, so as to recommend materials to the target users.

[0045] Specifically, the target third-party multimedia materials are used to recommend to target users, and the preset conditions can be preset quantity or preset score, etc. Optionally, additional conditions can be set based on the preset quantity or preset score. For example, the preset quantity can be determined to be 10 third-party multimedia materials, and the publishers of the 10 third-party multimedia materials can be at least 9 people, or the content of the 10 multimedia materials can include at least three types such as photos, text, audio, or video.

[0046] This application embodiment constructs a material relationship graph based on multiple first multimedia materials in a database. The material relationship graph includes graph nodes and edges, where graph nodes represent first multimedia materials and edges indicate that two connected graph nodes have an association relationship. The material relationship graph is input into a trained target model for feature aggregation to obtain a first embedding representation corresponding to each first multimedia material. The first embedding representation aggregates the features of the corresponding first multimedia material and the features of other first multimedia materials that are associated with the first multimedia material. Based on at least one second multimedia material corresponding to the target user, a second embedding representation is determined to represent the target user's interest features. The second multimedia material is the material among the multiple first multimedia materials for which the target user has historical interaction behavior. Based on the multiple first embedding representations and the second embedding representation, an interest value is determined for each third multimedia material. The third multimedia material is the material among the multiple first multimedia materials for which the target user has not historical interaction behavior. Based on the interest value corresponding to each third multimedia material, target third multimedia materials whose interest values ​​meet preset conditions are recommended to the target user. This can solve the problem of low efficiency in producing embedding representations of multimedia materials and improve the accuracy of material recommendations.

[0047] Figure 2 This is another schematic flowchart illustrating the method for recommending multimedia materials provided in the embodiments of this application, such as... Figure 2 As shown, the method may include the following steps.

[0048] S201: Construct a material relationship diagram based on multiple first multimedia materials in the database.

[0049] This step can be adopted. Figure 1 The description of step S101 in the embodiment will not be repeated here.

[0050] S202: Sample each node in the material relationship diagram and construct a material relationship sub-diagram corresponding to each sampled node.

[0051] The material relationship subgraph includes corresponding sampled graph nodes and associated graph nodes. The sampling includes at least one layer; the material relationship subgraph obtained from the k-th layer sampling includes the sampled graph node and associated graph nodes from the (k-1)-th layer; the material relationship subgraph obtained from the (k-1)-th layer sampling includes the sampled graph node, associated graph nodes from the (k-1)-th layer, and associated graph nodes from the (k-2)-th layer. For example... Figure 3aAs shown, when sampling two layers of graph nodes, i.e., k=2, the material relationship subgraph has a first-layer association graph node connected to vertex v by a solid line and a 0th-layer association graph node connected to the first-layer association graph node by a dashed line.

[0052] S203: Input the material relationship subgraph into the trained target model for feature aggregation to obtain the first embedded representation of the sampled graph node.

[0053] The first embedding representation aggregates the features of the sampled graph nodes and each of the associated graph nodes. Specifically, the material relationship subgraph is input into the trained target model, and the features of the sampled graph nodes and their associated graph nodes in the material relationship subgraph are aggregated to obtain the first embedding representation of the sampled graph nodes. For example, in Figure 3b In the (k-1)th layer, the sampling graph node v k-1 The information of two association graph nodes in the (k-2)th layer is aggregated, and the sampling graph node v is sampled. k-1 The association graph node u at level k-1 k-1 It also aggregates information from two related graph nodes in layer (k-2). For layer k, the sampled graph node v k Through the association graph node v at level k-1 k-1 This extends to the association graph nodes at level k-2. Therefore, during aggregation, aggregating k times is sufficient to extend to the association graph nodes at level k.

[0054] S204: Obtain at least one second multimedia material from the plurality of first multimedia materials that indicates the target user has generated historical interaction behavior.

[0055] Optionally, when the target user's historical interaction behavior is sparse (i.e., the target user has few historical interaction behaviors and / or the interaction time interval is long), the second multimedia material with the most recent interaction can be obtained. When the target user's historical interaction behavior is dense, multiple second multimedia materials can be filtered to save computing resources.

[0056] S205: Determine a second embedding representation for representing the target user's interest features based on the first embedding representation of each of the second multimedia materials.

[0057] Specifically, the second embedding representation representing the target user's interest features is obtained by averaging the first embedding representations of each of the second multimedia materials. This method ensures that the second embedding representation has the same dimension as the first embedding representation.

[0058] S206: Determine the interest value of each third multimedia material based on the first embedded representation and the second embedded representation corresponding to each third multimedia material.

[0059] This step can be adopted. Figure 1 The description of step S104 in the embodiment will not be repeated here.

[0060] S207: Sort each of the third multimedia materials according to the interest value corresponding to each of the third multimedia materials; based on the sorting result, select target third multimedia materials that meet the preset conditions from each of the third multimedia materials to recommend materials to the target user.

[0061] Specifically, the third multimedia materials are sorted from high to low or from low to high according to the interest value, and the third multimedia materials with the highest interest value that meet the quantity threshold in the preset conditions are selected according to the sorting results.

[0062] For example, the interest values ​​for sports news, movie recommendations, and beauty bloggers in third-party multimedia materials are 0.642, 0.553, and 0.784, respectively. Therefore, the result of sorting the above three third-party multimedia materials according to their interest values ​​is: beauty bloggers, sports news, and movie recommendations.

[0063] In one implementation, the step of inputting the material relationship subgraph into the trained target model for feature aggregation to obtain the first embedded representation of the sampled graph node includes: obtaining the first initial embedded representation of the sampled graph node and each of the associated graph nodes in the material relationship subgraph; aggregating the first initial embedded representations of each of the associated graph nodes to generate a first intermediate aggregated embedded representation of the associated graph node; and concatenating the first intermediate aggregated embedded representation and the first initial embedded representation of the sampled graph node to obtain the first embedded representation of the sampled graph node.

[0064] Specifically, feature extraction is performed on the first multimedia material corresponding to the sampling graph node to obtain the first initial embedding representation of the sampling graph node and its associated graph nodes. The material relationship subgraph is as follows: Figure 3a As shown, three of the five association graph nodes in layer 1 are associated with association graph nodes in layer 0. These three association graph nodes are aggregated with the first initial embedding representations of the association graph nodes in layer 0 to obtain the first intermediate aggregated embedding representation. The first embedding representation of the sampled graph node v is obtained by concatenating the first initial embedding representation of the sampled graph node v with the aforementioned first intermediate aggregated embedding representation. This method not only preserves the features of the first multimedia material corresponding to the sampled graph node v, but also learns the features of other first multimedia materials associated with this first multimedia material.

[0065] Optionally, the first embedding representation can be improved by learning features of multiple dimensions using a multilayer perceptron (MLP).

[0066] In one implementation, the first embedded representation of the sampling graph node Determined by the following formula:

[0067]

[0068] Where v represents the sampling graph node, u represents an associated graph node of the sampling graph node v, and k represents the sampling layer number. Let N(v) represent the first initial embedding representation of the associated graph node u in the (k-1)th layer, N(v) represent the set of associated graph nodes of the sampled graph node v, and Mean represent the average aggregation process. Specifically, the first embedding representation of u is averaged and then concatenated with the first initial embedding representation of v, with two nonlinear transformations performed in between to fully learn the nonlinear characteristics of the multimedia material.

[0069] This application embodiment constructs a material relationship graph based on multiple first multimedia materials in a database; samples each graph node in the material relationship graph to construct a material relationship subgraph corresponding to each sampled graph node; inputs the material relationship subgraph into a trained target model for feature aggregation to obtain a first embedding representation of the sampled graph node; obtains at least one second multimedia material from the multiple first multimedia materials that has generated historical interaction behavior of the target user; determines a second embedding representation for representing the target user's interest features based on the first embedding representation of each second multimedia material; determines the interest value of a third multimedia material based on the multiple first embedding representations and the second embedding representation; sorts the third multimedia materials according to the interest value corresponding to each third multimedia material; and selects target third multimedia materials that meet preset conditions from the sorting results for material recommendation. This application uses subgraphs for feature aggregation, which reduces storage resource consumption and improves computation speed during the feature aggregation process.

[0070] In one implementation, determining the interest value corresponding to each third multimedia material based on the first embedded representation and the second embedded representation corresponding to each third multimedia material includes: for each third multimedia material, calculating the similarity between the first embedded representation and the second embedded representation corresponding to the third multimedia material; and using the similarity as the interest value of the third multimedia material.

[0071] Optionally, the angle between the first and second embedding representations can be calculated using cosine similarity. The smaller the angle, the more similar the first and second embedding representations are. For example, if the first embedding representation is v(5,3,4,4) and the second embedding representation is u(3,1,2,3), then the cosine similarity between the first embedding v and the second embedding representation u is:

[0072]

[0073]

[0074] Where, sim uv Let v represent the similarity between the first embedding v and the second embedding u, and let cos(u,v) represent the cosine angle between the first embedding v and the second embedding u.

[0075] Using the similarity as the target user's interest value for third-party multimedia materials, materials with high interest values ​​can be filtered from multiple third-party multimedia materials and recommended to the target user.

[0076] This application embodiment obtains the first initial embedding representations of the sampled graph nodes and each of the associated graph nodes in the material relationship subgraph; aggregates the first initial embedding representations of each of the associated graph nodes to generate a first intermediate aggregated embedding representation of the associated graph node; and concatenates the first intermediate aggregated embedding representation with the first initial embedding representation of the sampled graph node to obtain the first embedding representation of the sampled graph node. This method can aggregate features of multiple multimedia materials, saving storage resources. By aggregating features of the sampled graph nodes in the material relationship subgraph, the computational speed of the target model is improved.

[0077] Figure 4 This diagram illustrates a flowchart of the training method for the target model provided in an embodiment of this application. Figure 4 As shown, the training method may include the following steps.

[0078] S401: Construct a sample relationship diagram based on multiple multimedia materials from the user's historical interactions.

[0079] The sample relationship graph includes sample graph nodes and edges. The sample graph nodes represent the multimedia materials from the user's historical interactions, and the edges indicate that two connected sample graph nodes have an association relationship. This step can be referenced... Figure 1 The difference in the description of step S101 in the embodiment is that the association relationship described in this embodiment is the multimedia materials of a single user's historical interaction. For example, a user interacts with 3 multimedia materials during a certain period of time, and there is an association relationship between these 3 multimedia materials.

[0080] S402: Sample each sample graph node in the sample relationship graph and construct a sample relationship subgraph corresponding to each sampled sample graph node.

[0081] The sample relationship subgraph includes the corresponding sample graph node and the associated sample graph node that is related to it.

[0082] Optionally, a certain number of associated graph nodes can be sampled for feature aggregation for each sample graph node. For example, if the required number of associated graph nodes is M, i.e., the sampling quantity is M, if the number of associated graph nodes is less than M, the sampled associated graph nodes can be repeatedly sampled until M associated graph nodes are sampled. If the number of associated graph nodes is greater than M, then sampling M associated graph nodes is sufficient.

[0083] Based on the sampled graph nodes and their associated sampled graph nodes obtained from the sampling, construct a sample relationship subgraph corresponding to each sampled graph node.

[0084] S403: Input the sample relationship subgraph into the initial target model, perform feature aggregation on the sampled sample graph nodes and each associated sample graph node in the sample relationship subgraph, and output the first embedding representation of the sampled sample graph node.

[0085] This step can be referred to. Figure 2 In the description of step S203 in the embodiment, specifically, the sample graph nodes in the sample relationship subgraph contain the feature identifiers of the corresponding multimedia materials. Considering that learning the features of multimedia materials through feature identifiers may lead to insufficient feature learning, the feature information of the corresponding multimedia materials can be supplemented in the form of feature tags before feature aggregation. For example, each sample graph node carries a feature matrix, which stores the feature tags of the multimedia materials. For example, the feature matrix [1,2,3,4] indicates that the multimedia materials have four feature tags: history 1, cultural relics 2, archaeology 3, and auction 4.

[0086] Optionally, mini-batch training can be used to train the material relationship graph, using only one subgraph at a time. For example, if the material relationship graph has 5000 nodes, 100 nodes can be selected each time to calculate the loss function and update the model parameters. After 50 iterations, training the entire material relationship graph is complete. Because multiple subgraphs are used to calculate the loss in each update, the loss calculation and model parameter updates are more representative, and the result of a single subgraph will not affect the loss calculation and model parameter updates. Furthermore, this training method results in a more stable decrease in loss, and mini-batch training using material relationship subgraphs also reduces computational resource consumption.

[0087] S404: Adjust the parameters of the target model based on the first embedding representation to obtain a trained target model.

[0088] Specifically, each sample relationship subgraph is input as a mini-batch of samples into the target model for iterative training. During each iteration, the parameters of the target model are updated. After training, the target model's performance needs to be tested using a validation set. If the target model does not meet the test performance threshold, the training parameters need to be adaptively adjusted based on the test results, and the target model is retrained using sample data. This process continues until the target model passes the test performance threshold, resulting in a well-trained target model that improves the accuracy of feature learning.

[0089] The target model training method provided in this invention constructs a sample relationship graph based on multiple multimedia materials interacting with in the user's history. The sample relationship graph includes sample graph nodes and edges. The sample graph nodes represent the multimedia materials interacting with in the user's history, and the edges indicate that two connected sample graph nodes have an association relationship. Each sample graph node in the sample relationship graph is sampled to construct a sample relationship subgraph corresponding to each sampled sample graph node. The sample relationship subgraph includes the corresponding sampled sample graph node and associated sample graph nodes. The sample relationship subgraph is input into an initial target model. Feature aggregation is performed on the sampled sample graph nodes and each associated sample graph node in the sample relationship subgraph to output a first embedding representation of the sampled sample graph node. The parameters of the target model are adjusted based on the first embedding representation to obtain a trained target model. This method enables model training using sample relationship subgraphs, improving training efficiency.

[0090] Figure 5 The diagram shows a schematic of the structure of an apparatus for recommending multimedia materials provided in an embodiment of this application. The apparatus 500 includes: a first processing module 501, a second processing module 502, a first determining module 503, a calculation module 504, and a second determining module 505.

[0091] The first processing module 501 is used to construct a material relationship diagram based on multiple first multimedia materials in the database. The material relationship diagram includes graph nodes and edges, wherein the graph nodes are used to represent the first multimedia materials, and the edges are used to indicate that two connected graph nodes have an association relationship.

[0092] The second processing module 502 is used to input the material relationship diagram into the trained target model to perform feature aggregation and obtain a first embedding representation corresponding to each first multimedia material. The first embedding representation aggregates the features of the corresponding first multimedia material and the features of other first multimedia materials that are related to the first multimedia material.

[0093] The first determining module 503 is used to determine a second embedded representation for representing the interest characteristics of the target user based on at least one second multimedia material corresponding to the target user, wherein the second multimedia material is a material among a plurality of first multimedia materials in which the target user has generated historical interaction behavior;

[0094] The calculation module 504 is used to determine the interest value corresponding to each third multimedia material based on the first embedding representation and the second embedding representation corresponding to each third multimedia material, wherein the third multimedia material is the material in which the target user has not generated historical interaction behavior among a plurality of first multimedia materials;

[0095] The second determining module 505 is used to determine target third multimedia materials whose interest values ​​meet preset conditions based on the interest values ​​corresponding to each of the third multimedia materials, so as to recommend materials.

[0096] In one optional implementation, the second processing module 502 includes:

[0097] The sampling submodule 5021 is used to sample each graph node in the material relationship diagram and construct a material relationship subgraph corresponding to each sampled graph node. The material relationship subgraph includes the corresponding sampled graph node and the associated graph node with which it is related.

[0098] The aggregation submodule 5022 is used to input the material relationship subgraph into the trained target model for feature aggregation to obtain the first embedding representation of the sampled graph node. The first embedding representation aggregates the features of the sampled graph node and each of the associated graph nodes.

[0099] In one optional implementation, the aggregation submodule 5022 is specifically used to obtain the first initial embedding representation of the sampled graph node and each of the associated graph nodes in the material relationship subgraph; aggregate the first initial embedding representations of each of the associated graph nodes to generate a first intermediate aggregated embedding representation of the associated graph node; and concatenate the first intermediate aggregated embedding representation and the first initial embedding representation of the sampled graph node to obtain the first embedding representation of the sampled graph node.

[0100] In one alternative implementation, the sampling includes at least one layer, and the aggregation submodule 5022 is specifically used for the first embedded representation of the sampled graph nodes. Determined by the following formula:

[0101]

[0102] Where v represents the sampling graph node, u represents an associated graph node of the sampling graph node v, and k represents the sampling layer number. Let N(v) represent the first initial embedding representation of the associated graph node u in the (k-1)th layer, N(v) represent the set of associated graph nodes of the sampled graph node v, and Mean represents the average aggregation process.

[0103] In one optional implementation, the first determining module 503 includes:

[0104] The acquisition submodule 5031 is used to acquire at least one second multimedia material from a plurality of first multimedia materials in which the target user has generated historical interaction behavior.

[0105] The determination submodule 5032 is used to determine a second embedding representation for representing the target user's interest features based on the first embedding representation of each of the second multimedia materials.

[0106] In one optional implementation, the calculation module 504 is specifically used to calculate the similarity between the first embedded representation and the second embedded representation corresponding to each third multimedia material; and to use the similarity as the interest value of the third multimedia material.

[0107] In one optional implementation, the second determining module 505 is specifically used to sort each of the third multimedia materials according to the interest value corresponding to each of the third multimedia materials; and to select target third multimedia materials that meet preset conditions from the third multimedia materials according to the sorting result.

[0108] In one implementation, the apparatus further includes a model training module 506, configured to construct a sample relationship graph based on multiple multimedia materials interacting with in the user's history. The sample relationship graph includes sample graph nodes and edges. The sample graph nodes represent the multimedia materials interacting with in the user's history, and the edges represent the association between two connected sample graph nodes. The module samples each sample graph node in the sample relationship graph to construct a sample relationship subgraph corresponding to each sampled sample graph node. The sample relationship subgraph includes the corresponding sampled sample graph node and associated sample graph nodes that are related to it. The module inputs the sample relationship subgraph into an initial target model, performs feature aggregation on the sampled sample graph nodes and each associated sample graph node in the sample relationship subgraph, and outputs a first embedding representation of the sampled sample graph node. Based on the first embedding representation, the parameters of the target model are adjusted to obtain a trained target model.

[0109] The device 500 provided in this application embodiment can execute the methods described in the preceding method embodiments and achieve the functions and beneficial effects of the methods described in the preceding method embodiments, which will not be repeated here.

[0110] Figure 6This diagram illustrates the hardware structure of an electronic device executing the recommended multimedia materials provided in the embodiments of this application. Referring to the diagram, at the hardware level, the electronic device includes a processor and optionally, an internal bus, a network interface, and a memory. The memory may include RAM, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk storage device. Of course, the electronic device may also include other hardware required for other services.

[0111] The processor, network interface, and memory can be interconnected via an internal bus, which can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be categorized as an address bus, data bus, control bus, etc. For ease of illustration, only a single bidirectional arrow is used in this diagram, but this does not imply that there is only one bus or one type of bus.

[0112] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0113] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a device for locating the target user at the logical level. The processor executes the program stored in memory and specifically performs the actions described in the preceding method embodiments. Figure 1 , 2 The methods described in section 4, and the methods implemented in the preceding embodiments, are also applicable. Figure 1 , 2 The functions and beneficial effects of the methods described in section 4 will not be repeated here.

[0114] The above is as stated in this application. Figure 1 , 2The methods disclosed in the embodiments shown in Figure 4 can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0115] The electronic device can also perform the methods described in the preceding embodiments. Figure 1 , 2 The methods described in section 4, and the methods implemented in the preceding embodiments, are also applicable. Figure 1 , 2 The functions and beneficial effects of the methods described in section 4 will not be repeated here.

[0116] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0117] This application also proposes a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the methods described in the preceding embodiments. Figure 1 , 2 The methods described in section 4, and the methods implemented in the preceding embodiments, are also applicable. Figure 1 , 2 The functions and beneficial effects of the methods described in section 4 will not be repeated here.

[0118] The computer-readable storage medium includes read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, etc.

[0119] Furthermore, this application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, implement the method described in the preceding embodiments. Figure 1 , 2 The functions and beneficial effects of the methods described in section 4 will not be repeated here.

[0120] In summary, the above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0121] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0122] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0123] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0124] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0125] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method of recommending multimedia material, characterized by, The application comprises the following steps: constructing a material relationship graph according to a plurality of first multimedia materials in a database, the material relationship graph comprising graph nodes and edges, wherein the graph nodes are used to represent the first multimedia materials, and the edges are used to represent that two connected graph nodes have a correlation relationship; inputting the material relationship graph into a trained target model for feature aggregation to obtain a first embedding representation corresponding to each first multimedia material, the first embedding representation aggregating the features of the corresponding first multimedia material and the features of other first multimedia materials having a correlation relationship with the first multimedia material; determining a second embedding representation used to represent the interest features of a target user according to at least one second multimedia material corresponding to the target user, the second multimedia material being a material in which the target user generates a historical interaction behavior among the plurality of first multimedia materials; determining a corresponding interest value of each third multimedia material according to the first embedding representation corresponding to each third multimedia material and the second embedding representation, the third multimedia material being a material in which the target user does not generate a historical interaction behavior among the plurality of first multimedia materials; determining a target third multimedia material whose interest value meets a preset condition to perform material recommendation; the step of inputting the material relationship graph into the trained target model for feature aggregation to obtain the first embedding representation corresponding to each first multimedia material comprises the following steps: sampling each graph node in the material relationship graph respectively to construct a material relationship subgraph corresponding to each sampled graph node, wherein the material relationship subgraph comprises the corresponding sampled graph node and associated graph nodes having a correlation relationship with the corresponding sampled graph node; inputting the material relationship subgraph into the trained target model for feature aggregation to obtain the first embedding representation of the sampled graph node, the first embedding representation aggregating the features of the sampled graph node and each associated graph node; the step of inputting the material relationship subgraph into the trained target model for feature aggregation to obtain the first embedding representation of the sampled graph node comprises the following steps: obtaining first initial embedding representations of the sampled graph node and each associated graph node in the material relationship subgraph; aggregating the first initial embedding representations of each associated graph node to generate a first intermediate aggregated embedding representation of the associated graph node; performing splicing processing on the first intermediate aggregated embedding representation and the first initial embedding representation of the sampled graph node to obtain the first embedding representation of the sampled graph node.

2. The method of claim 1, wherein, The sampling comprises at least one layer, and the step of inputting the material relationship subgraph into the trained target model for feature aggregation to obtain the first embedding representation of the sampled graph node comprises the following steps: a first embedding representation of the sampling graph node is determined by the following equation: wherein, denotes the sample graph node, denotes the sample graph node one associated graph node, denotes a number of sampling layers, denotes a first initial embedding of the associated graph node of a first layer, denotes a set of associated graph nodes of the sample graph node denotes an average aggregation process.​​ 3. The method of claim 1, wherein, The step of determining the second embedding representation used to represent the interest features of the target user according to at least one second multimedia material corresponding to the target user comprises the following steps: obtaining at least one second multimedia material in which the target user generates a historical interaction behavior from the plurality of first multimedia materials; determining the second embedding representation used to represent the interest features of the target user according to the first embedding representation of each second multimedia material.

4. The method of claim 1, wherein, The determining of the interest value corresponding to each third multimedia material comprises: The similarity between the first embedding representation corresponding to each third multimedia material and the second embedding representation is calculated, and the similarity is taken as the interest value of the third multimedia material.

5. The method of claim 1, wherein, The determining of the target third multimedia material with the interest value meeting the preset condition comprises: The third multimedia materials are sorted according to the interest values corresponding to the third multimedia materials; The target third multimedia material meeting the preset condition is selected from the third multimedia materials according to the sorting result.

6. The method of claim 1, wherein, The training method of the target model comprises: A sample relationship graph is constructed according to the user historical interaction multimedia materials, the sample relationship graph comprises sample graph nodes and edges, the sample graph nodes are used to represent the user historical interaction multimedia materials, and the edges are used to represent that two connected sample graph nodes have a correlation relationship; Each sample graph node in the sample relationship graph is sampled respectively, and a sample relationship subgraph corresponding to each sampled sample graph node is constructed, wherein the sample relationship subgraph comprises the corresponding sampled sample graph node and the associated sample graph node having a correlation relationship with the corresponding sampled sample graph node; The sample relationship subgraph is input into an initial target model, feature aggregation is performed on the sampled sample graph node and each associated sample graph node in the sample relationship subgraph, and a first embedding representation of the sampled sample graph node is output; The parameters of the target model are adjusted based on the first embedding representation, and a trained target model is obtained.

7. An apparatus for recommending multimedia materials, comprising: A first processing module configured to construct a material relationship graph according to a plurality of first multimedia materials in a database, wherein the material relationship graph comprises graph nodes and edges, the graph nodes are used to represent the first multimedia materials, and the edges are used to represent that two connected graph nodes have a correlation relationship; A second processing module configured to input the material relationship graph into a trained target model to perform feature aggregation, and obtain a first embedding representation corresponding to each first multimedia material, wherein the first embedding representation aggregates features of the corresponding first multimedia material and features of other first multimedia materials having a correlation relationship with the first multimedia material; A first determining module configured to determine a second embedding representation used to represent interest features of a target user according to at least one second multimedia material corresponding to the target user, wherein the second multimedia material is a material in the plurality of first multimedia materials that has generated a historical interaction behavior of the target user; A calculating module configured to determine an interest value corresponding to each third multimedia material according to a first embedding representation corresponding to each third multimedia material and a second embedding representation, wherein the third multimedia material is a material in the plurality of first multimedia materials that has not generated a historical interaction behavior of the target user. The second determining module is configured to determine a target third multimedia material whose interest value meets a preset condition according to the interest value corresponding to each third multimedia material, so as to perform material recommendation. The second processing module comprises: The acquisition sub-module is configured to sample each graph node in the material relationship graph respectively, and construct a material relationship sub-graph corresponding to each sampled graph node, wherein the material relationship sub-graph comprises the corresponding sampled graph node and associated graph nodes associated with the corresponding sampled graph node; The aggregation sub-module is configured to input the material relationship sub-graph into the trained target model to perform feature aggregation, and obtain a first embedding representation of the sampled graph node, wherein the first embedding representation aggregates the features of the sampled graph node and each associated graph node. The aggregation sub-module is specifically configured to obtain a first initial embedding representation of the sampled graph node and each associated graph node in the material relationship sub-graph; aggregate the first initial embedding representation of each associated graph node to generate a first intermediate aggregated embedding representation of the associated graph node; and perform splicing processing on the first intermediate aggregated embedding representation and the first initial embedding representation of the sampled graph node to obtain the first embedding representation of the sampled graph node. 8.An electronic device comprising: a processor; and a memory arranged to store computer-executable instructions that, when executed, cause the processor to perform the steps of the method of recommending a multimedia material according to any one of claims 1-6. 9.A computer-readable medium storing one or more programs, which when executed by an electronic device comprising multiple applications, cause the electronic device to perform the steps of the method of recommending a multimedia material according to any one of claims 1-6.

Images