Multimedia resource display method, device, storage medium and electronic device

Abstract

The embodiments of the present application provide a method, device, storage medium and electronic device for displaying multimedia resources, wherein the method includes: predicting the temporal relationship characteristics of the target account based on the first browsing information of the target account, and predicting the transfer relationship characteristics of the target account based on the first browsing information; generating the target account characteristics of the target account based on the temporal relationship characteristics, the transfer relationship characteristics and the resource browsing characteristics of the target account; and displaying the target multimedia resources matching the target account characteristics to the target account. Through this application, the problem of low matching between the multimedia resources recommended for the target account and the current resource preference of the target account for multimedia resources is solved, so as to achieve the effect of improving the matching between the multimedia resources recommended for the target account and the current resource preference of the target account for multimedia resources.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of multimedia resource recommendation, and more specifically, to a method, device, storage medium and electronic device for displaying multimedia resources. Background Art

[0002] As the amount of information on the Internet grows explosively, users have more and more ways to obtain information. Multimedia resources, due to their rich forms of expression, intuitive information expression, and clear subject content, have become increasingly important in information retrieval and network communication.

[0003] In order to facilitate users to obtain the required multimedia resources, numerous multimedia resource recommendation engines have emerged, and their essence is to recommend multimedia resources to users based on their preferences for multimedia resources. In order to achieve the function of recommending multimedia resources based on users' preferences for multimedia resources, the relevant technology predicts the user's current preference for multimedia resources by obtaining the multimedia resources that the user has currently browsed. The multimedia resource preferences determined in this way only focus on the temporal relationship between the multimedia resources that the user has browsed, and cannot adapt to the user's more complex multimedia resource recommendation needs. Therefore, the multimedia resources recommended to the user cannot fully meet the user's actual current preferences for multimedia resources. Summary of the invention

[0004] The embodiments of the present application provide a method, device, storage medium and electronic device for displaying multimedia resources, so as to at least solve the problem in the related art that the multimedia resources recommended for a target account and the target account's current resource preference for the multimedia resources have a low matching degree.

[0005] According to an embodiment of the present application, a method for displaying multimedia resources is provided, comprising: predicting a temporal relationship feature of the target account based on first browsing information of the target account, and predicting a transfer relationship feature of the target account based on the first browsing information, wherein the first browsing information is used to indicate the multimedia resource browsing situation of the target account in multiple reference time periods before the current moment, the temporal relationship feature is used to indicate the temporal changes in the browsing preference of the target account for multimedia resources, and the transfer relationship feature is used to indicate the current browsing preference of the target account under the influence of the multimedia resources browsed in each of the reference time periods; generating a target account feature of the target account based on the temporal relationship feature, the transfer relationship feature and the resource browsing feature of the target account, wherein the resource browsing feature is used to indicate the multimedia resource browsing preference of the target account under the influence of the multimedia resource browsing information of a reference account associated with the target account, and the target account feature is used to indicate the multimedia resource browsing preference of the target account at the current moment; and displaying a target multimedia resource matching the target account feature to the target account.

[0006] Optionally, predicting the temporal relationship feature of the target account based on the first browsing information of the target account includes: filtering out the current target time period to be processed from the multiple reference time periods according to the time sequence of the multiple reference time periods; obtaining the first browsing feature of the first sub-browsing information of the target account in the target time period, wherein the first browsing feature is used to indicate the browsing preference of the target account in the target time period; predicting the second temporal relationship feature in the target time period based on the first temporal relationship feature in the candidate time period and the first browsing feature, wherein the candidate time period is adjacent to the target time period and is located before the target time period in the multiple reference time periods, and the first temporal relationship feature is used to indicate the influence of the browsing preference of the target account in the time period before the candidate time period on the browsing preference of the target account in the candidate time period; when the target time period is the last one of the multiple reference time periods, determining the second temporal relationship feature as the temporal relationship feature.

[0007] Optionally, the obtaining of the first browsing feature of the first sub-browsing information of the target account within the target time period includes: filtering out the second multimedia resource to be processed from multiple first multimedia resources in a browsing order, wherein the first sub-browsing information includes multiple first multimedia resources browsed by the target account within the target time period in the browsing order; predicting the second preference feature of the target account after browsing the second multimedia resource based on the first preference feature of the target account and the initial resource feature of the second multimedia resource, wherein the first preference feature is used to indicate the browsing preference of the target account after browsing the second multimedia resource under the influence of the multimedia resource browsed before the second multimedia resource, and the initial resource feature is used to indicate the account feature of the account browsing the second multimedia resource; when the second multimedia resource is the last multimedia resource browsed by the target account within the target time period, the second preference feature is determined as the first browsing feature.

[0008] Optionally, predicting the second preference feature of the target account after browsing the second multimedia resource based on the first preference feature of the target account and the initial resource feature of the second multimedia resource includes: using a first reset gate to extract first adjustment information from the first preference feature, wherein the first adjustment information is used to indicate the degree of influence of the first preference feature on the browsing preference of the target account; calling a first update gate to update the initial resource feature according to the first adjustment information to obtain the second preference feature.

[0009] Optionally, before obtaining the first browsing feature of the first sub-browsing information of the target account within the target time period, the method also includes: detecting a third multimedia resource browsed by the target account before the second multimedia resource in the first sub-browsing information; when the third multimedia resource exists in the first sub-browsing information, determining the browsing preference feature of the target account after browsing the third multimedia resource as the first preference feature; when the third multimedia resource does not exist in the first sub-browsing information, determining the first temporal relationship feature as the first preference feature.

[0010] Optionally, the filtering out the second multimedia resource to be processed currently from multiple first multimedia resources in a browsing order includes: calculating a reference resource parameter of each of the first multimedia resources, wherein the reference resource parameter is used to indicate a degree of dependency between the current multimedia resource and other multimedia resources among the multiple first multimedia resources except the current multimedia resource; filtering out a fourth multimedia resource whose reference resource parameter is greater than or equal to a target resource parameter from the multiple first multimedia resources; and filtering out the second multimedia resource to be processed currently from the multiple fourth multimedia resources in a browsing order.

[0011] Optionally, predicting the second temporal relationship feature within the target time period based on the first temporal relationship feature within the candidate time period and the first browsing feature includes: using a second reset gate to extract second adjustment information from the first temporal relationship feature, wherein the second adjustment information is used to indicate the degree of influence of the first temporal relationship feature on the browsing preference of the target account; calling a second update gate to update the first browsing feature according to the second adjustment information to obtain the second temporal relationship feature.

[0012] Optionally, predicting the transfer relationship feature of the target account based on the first browsing information includes: obtaining the second browsing feature of the second sub-browsing information within each of the reference time periods, wherein the second browsing feature is used to indicate the browsing preference of the target account within the reference time period; using the second browsing feature to calculate the browsing similarity between any two of the plurality of second sub-browsing information; generating a random walk sequence for the target sub-browsing information in the plurality of second sub-browsing information based on the browsing similarity, wherein the random walk sequence takes the target sub-browsing information as a starting point, and the random walk sequence records the association relationship between a target number of second sub-browsing information in the plurality of second sub-browsing information, wherein the target sub-browsing information is any sub-browsing information included in the plurality of second sub-browsing information; predicting the initial transfer relationship feature of the target sub-browsing information corresponding to the random walk sequence based on the random walk sequence, wherein the initial transfer relationship feature is used to indicate the browsing preference of the target account under the influence of the corresponding target sub-browsing information; generating the transfer relationship feature based on the initial transfer relationship features of the plurality of second sub-browsing information.

[0013] Optionally, generating a random walk sequence for a target sub-browsing information in a plurality of second sub-browsing information according to the browsing similarity includes: constructing a first relationship graph of the plurality of second sub-browsing information according to the browsing similarity, wherein the first relationship graph includes a plurality of first vertices and a first directed edge between any two first vertices, each of the first vertices corresponds to one second sub-browsing information, and the first directed edge records the association relationship between the second sub-browsing information; extracting a first vertex sequence with the second vertex as a starting point for each second vertex from the first relationship graph, wherein the second vertex is any one of the plurality of first vertices, and the first vertex sequence records the association relationship between the target number of first vertices in the plurality of first vertices; and sorting the second sub-browsing information corresponding to the first vertices in the first vertex sequence according to the association relationship between the first vertices recorded in the first vertex sequence to obtain the random walk sequence.

[0014] Optionally, extracting a first vertex sequence with the second vertex as the starting point for each second vertex from the first relationship graph includes: in the process of extracting the first vertex sequence, searching the first relationship graph for a fourth vertex that has an association relationship with the third vertex, wherein the third vertex is a vertex for searching for the next associated vertex, and the outgoing end of the first directed edge connecting the fourth vertex and the third vertex points to the fourth vertex; allocating a screening weight to each of the fourth vertices according to the vertex association degree indicated by the first directed edge; randomly screening out a fifth vertex that is located after the third vertex in the vertex sequence from multiple fourth vertices according to the screening weight; detecting a reference number of vertices included in the currently screened vertex sequence; and when the reference number is equal to the target number, determining the currently screened vertex sequence as the first vertex sequence.

[0015] Optionally, constructing a first relationship graph of multiple second sub-browsing information according to the browsing similarity includes: creating the first vertex corresponding to each second sub-browsing information, and configuring the vertex association between the first vertices corresponding to the second sub-browsing information according to the browsing similarity between the second sub-browsing information; creating a first connecting edge between the first vertices whose vertex association is greater than or equal to the target association; configuring the pointing relationship of the first connecting edge according to the timing relationship of the occurrence of the second sub-browsing information corresponding to the first vertex, to obtain the first directed edge, wherein the incident end of the first directed edge is connected to the vertex with an earlier occurrence timing, and the outgoing end of the first directed edge is connected to the vertex with a later occurrence timing; and determining the relationship graph of the first directed edges that records the connections between multiple first vertices and the first vertices as the first relationship graph.

[0016] Optionally, predicting the initial transfer relationship characteristics of the target sub-browsing information corresponding to the random walk sequence based on the random walk sequence includes: calculating the target timing characteristics of each random walk sequence in multiple random walk sequences of each second sub-browsing information, wherein the timing characteristics are used to indicate the timing changes in the browsing preference of the target account in the random walk sequence; assigning a sequence weight to each random walk sequence according to the proportion of the target timing characteristics of each random walk sequence in the timing characteristics and values ​​of multiple random walk sequences; and performing weighted summation of multiple timing characteristics according to the sequence weight to obtain the initial transfer relationship characteristics.

[0017] Optionally, the calculation of the target timing feature of each random walk sequence in the multiple random walk sequences of each second sub-browsing information includes: determining the sixth sub-browsing information to be processed from the random walk sequence according to the occurrence timing of the sub-browsing information; predicting the fourth preference feature of the target account after browsing the sixth sub-browsing information based on the third preference feature of the target account and the second browsing feature corresponding to the sixth sub-browsing information, wherein the third preference feature is used to indicate the browsing preference of the target account before browsing the sixth sub-browsing information in the random walk sequence; and when the sixth sub-browsing information is the last sub-browsing information in the random walk sequence, determining the fourth preference feature as the target timing feature of the random walk sequence.

[0018] Optionally, predicting the fourth preference feature of the target account after browsing the sixth sub-browsing information based on the third preference feature of the target account and the second browsing feature corresponding to the sixth sub-browsing information includes: using a third reset gate to extract second adjustment information from the third preference feature, wherein the second adjustment information is used to indicate the degree of influence of the third preference feature on the browsing preference of the target account; calling a third update gate to update the second browsing feature according to the second adjustment information to obtain the fourth preference feature.

[0019] Optionally, generating the transfer relationship feature based on the initial transfer relationship features of multiple second sub-browsing information includes one of the following: determining the average of multiple initial transfer relationship features as the transfer relationship feature; determining the maximum feature value among multiple initial transfer relationship features as the transfer relationship feature; assigning a corresponding feature weight to each initial transfer relationship feature according to the feature relationship between the initial transfer relationship features; and performing weighted sum calculation on the initial transfer relationship features using the feature weights to obtain the transfer relationship feature.

[0020] Optionally, before generating the target account characteristics of the target account based on the timing relationship characteristics, the transfer relationship characteristics and the resource browsing characteristics of the target account, the method also includes: obtaining second browsing information of the reference account, wherein the second browsing information is used to indicate the multimedia resource browsing status of the reference account within multiple reference time periods; and predicting the resource browsing characteristics of the target account based on the first browsing information and the second browsing information.

[0021] Optionally, predicting the resource browsing features of the target account based on the first browsing information and the second browsing information includes: constructing a second relationship graph based on the first browsing information and the second browsing information, wherein the second relationship graph includes multiple sixth vertices, multiple seventh vertices, a first edge and a second edge, wherein the sixth vertex corresponds to the target account and the reference account one-to-one, the seventh vertex corresponds to the multimedia resources included in the first browsing information and the multimedia resources included in the second browsing information one-to-one, the first edge is used to indicate the association relationship between accounts, and the second edge is used to indicate the access relationship of the account to the multimedia resources; using a feature extraction model including multiple feature extraction layers connected in sequence to extract target vertex features of the target vertex in the second relationship graph, wherein the target vertex is any vertex in the second relationship graph, each feature extraction layer is used to generate vertex features of the target vertex according to the initial vertex features of the reference vertex in the current second relationship graph, and use the vertex features of the target vertex output by the current feature extraction layer to update the vertex features of the target vertex in the second relationship graph, and the reference vertex is a vertex in the second relationship graph that has an association relationship with the target vertex; determining the target vertex features of the vertex corresponding to the target account as the resource browsing features.

[0022] Optionally, the use of a feature extraction model including multiple sequentially connected feature extraction layers to extract target vertex features of target vertices in the second relationship graph includes: calling a target feature extraction layer to extract a target vertex set of the target vertex from the second relationship graph, wherein the target vertex set records vertices in the second relationship graph that are connected to the target vertex through the first edge or the second edge, and the target feature extraction layer is the extraction layer currently to perform feature extraction on the target vertices in the second relationship graph; calling the target feature extraction layer to perform feature aggregation on initial vertex features currently possessed by vertices included in the target vertex set to obtain reference vertex features of the target vertex output by the target feature extraction layer; and when the target feature extraction layer is the last layer of multiple sequentially connected feature extraction layers, determining the reference vertex features as the target vertex features.

[0023] Optionally, calling the target feature extraction layer to perform feature aggregation on the initial vertex features currently possessed by the vertices included in the target vertex set to obtain the reference vertex features of the target vertex output by the target feature extraction layer includes: detecting the vertex type of the target vertex; when the vertex type indicates that the target vertex is a vertex corresponding to an account, performing feature aggregation on the initial vertex features of the sixth vertex included in the target vertex set to obtain a first aggregated feature, and performing feature aggregation on the initial vertex features of the seventh vertex in the target vertex set to obtain a second aggregated feature; assigning corresponding aggregated feature weights to the first aggregated feature and the second aggregated feature according to the feature relationship between the first aggregated feature and the second aggregated feature; performing weighted sum calculation on the first aggregated feature and the second aggregated feature using the aggregated feature weight to obtain the reference vertex feature; when the vertex type indicates that the target vertex is a vertex corresponding to a multimedia resource, aggregating the initial vertex features of the vertices included in the target vertex set to obtain the reference vertex feature.

[0024] Optionally, the calling target feature extraction layer extracts the target vertex set of the target vertex from the second relationship graph, including: extracting the initial vertex set of the target vertex from the second relationship graph, wherein the initial vertex set includes vertices in the second relationship graph connected to the target vertex through the first edge or the second edge; screening a candidate number of candidate vertices from the initial vertex set according to the current vertex features of the vertices in the initial vertex set to obtain the target vertex set, wherein, in the case where the target vertex is a vertex corresponding to an account, the candidate vertices include vertices corresponding to the account and vertices corresponding to multimedia resources, and in the case where the target vertex is a vertex corresponding to a multimedia resource, the candidate vertices include vertices corresponding to the account, and the dependency relationship between the vertices in the target vertex set and the target vertex meets the target dependency condition.

[0025] Optionally, the method of screening out a candidate number of candidate vertices from the initial vertex set according to the vertex features currently possessed by the vertices in the initial vertex set to obtain the target vertex set includes: using the vertex features possessed by each vertex included in the first vertex set in the current screening round to calculate the incremental value of each vertex to the vertex set information, wherein the vertex set information is used to indicate the importance of the vertex features of all vertices in the second vertex set, and the incremental value is used to indicate the amount of change in the vertex set information of the second vertex set after the corresponding vertex in the first vertex set is added to the second vertex set, and the initial vertex set includes vertices in the first vertex set and vertices in the second vertex set; extracting vertices whose incremental values ​​are greater than or equal to the target incremental value from the first vertex set and adding them to the second vertex set; when the number of candidate vertices included in the second vertex set to which the vertices are added is the candidate number, and the candidate vertices include both vertices corresponding to accounts and vertices corresponding to multimedia resources, determining the second vertex set as the target vertex set.

[0026] Optionally, the generating the target account feature of the target account based on the temporal relationship feature, the transfer relationship feature and the resource browsing feature of the target account includes: performing feature splicing on the temporal relationship feature, the transfer relationship feature and the resource browsing feature to obtain a spliced ​​feature; performing feature fusion on the spliced ​​feature using a feature fusion model to obtain the target account feature output by the feature fusion model, wherein the feature fusion model records the conversion relationship between the spliced ​​feature and the account feature.

[0027] Optionally, displaying the target multimedia resources matching the target account characteristics to the target account includes: calculating the target similarity between the target account characteristics and the target resource characteristics of each multimedia resource in the multimedia resource library; filtering out the target multimedia resources whose target similarity is greater than or equal to a preset similarity from the multimedia resource library; and displaying the target multimedia resources to the target account.

[0028] Optionally, the calculation of the target similarity between the target account feature and the target resource features of each multimedia resource in the multimedia resource library includes: using a feature generation model to generate a first resource feature of the multimedia resource in the first browsing information and a second resource feature of the multimedia resource in the multimedia resource library; detecting feature difference information between the first resource feature and the second resource feature; using a feature adjustment amount corresponding to the feature difference information to perform feature adjustment on the initial resource feature of the multimedia resource in the first browsing information to obtain the target resource feature, wherein the initial resource feature is used to indicate the account feature of the account browsing the multimedia resource in the first browsing information; and calculating the target similarity between the target account feature and the target resource feature.

[0029] According to another embodiment of the present application, a multimedia resource display device is provided, comprising: a first prediction module, configured to predict a temporal relationship feature of a target account based on first browsing information of the target account, and to predict a transfer relationship feature of the target account based on the first browsing information, wherein the first browsing information is used to indicate the multimedia resource browsing situation of the target account in multiple reference time periods before the current moment, the temporal relationship feature is used to indicate the temporal change of the browsing preference of the target account for multimedia resources, and the transfer relationship feature is used to indicate the current browsing preference of the target account under the influence of the multimedia resources browsed in each of the reference time periods; a generation module, configured to generate a target account feature of the target account based on the temporal relationship feature, the transfer relationship feature and the resource browsing feature of the target account, wherein the resource browsing feature is used to indicate the multimedia resource browsing preference of the target account under the influence of the multimedia resource browsing information of a reference account associated with the target account, and the target account feature is used to indicate the multimedia resource browsing preference of the target account at the current moment; and a display module, configured to display a target multimedia resource matching the target account feature to the target account.

[0030] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0031] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0032] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0033] Through the present application, by predicting the temporal relationship feature for indicating the temporal change of the browsing preference of the target account for multimedia resources according to the first browsing information of the target account, and predicting the transfer relationship feature for indicating the multimedia resources browsed in each reference time period that affect the browsing preference of the target account, and then generating the target account feature of the target account according to the temporal relationship feature, the transfer relationship feature and the resource browsing feature of the target account, since the resource browsing feature of the target account is used to indicate the multimedia resource browsing preference of the target account under the influence of the multimedia resource browsing information of the reference account associated with the target account, the temporal relationship feature, the transfer relationship feature and the resource browsing feature under the influence of the social relationship of the target account are integrated into the generated target account feature, so that the target account feature is closer to the user's current real preference for multimedia resources, and then the target multimedia resources matching the target account feature displayed to the target account can better meet the target account's current demand for multimedia resources. The problem of low matching between the multimedia resources recommended for the target account and the target account's current resource preference for multimedia resources can be solved, so as to improve the matching effect between the multimedia resources recommended for the target account and the target account's current resource preference for multimedia resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a hardware structure block diagram of a server device of a multimedia resource display method according to an embodiment of the present application;

[0035] Figure 2 is a flowchart of a method for displaying multimedia resources according to an embodiment of the present application;

[0036] Figure 3 is an optional video recommendation schematic diagram according to an embodiment of the present application;

[0037] Figure 4 is an optional GRU cycle application network diagram according to an embodiment of the present application;

[0038] Figure 5 is a schematic diagram of a general heterogeneous accelerator design of an optional FPGA according to an embodiment of the present application;

[0039] Figure 6 is a schematic diagram of an optional recommended device hardware according to an embodiment of the present application;

[0040] Figure 7It is a structural block diagram of a multimedia resource display device according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0042] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0043] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a multimedia resource display method according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0044] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for displaying multimedia resources in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0045] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0046] In this embodiment, a method for displaying multimedia resources is provided. Figure 2 is a flowchart of a method for displaying multimedia resources according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0047] Step S202, predicting a temporal relationship feature of the target account based on first browsing information of the target account, and predicting a transfer relationship feature of the target account based on the first browsing information, wherein the first browsing information is used to indicate the browsing situation of multimedia resources of the target account in multiple reference time periods before the current moment, the temporal relationship feature is used to indicate the temporal change of the browsing preference of the target account for multimedia resources, and the transfer relationship feature is used to indicate the current browsing preference of the target account under the influence of the multimedia resources browsed in each of the reference time periods;

[0048] Step S204, generating a target account feature of the target account according to the temporal relationship feature, the transfer relationship feature and the resource browsing feature of the target account, wherein the resource browsing feature is used to indicate the multimedia resource browsing preference of the target account under the influence of the multimedia resource browsing information of the reference account associated with the target account, and the target account feature is used to indicate the multimedia resource browsing preference of the target account at the current moment;

[0049] Step S206: Display target multimedia resources matching the characteristics of the target account to the target account.

[0050] Through the above steps, the target account characteristics are generated according to the target account's first browsing information by predicting the temporal relationship characteristics used to indicate the temporal changes of the target account's browsing preference for multimedia resources, and the transfer relationship characteristics used to indicate the multimedia resources browsed in each reference time period that affect the target account's browsing preference. Since the target account's resource browsing characteristics are used to indicate the target account's multimedia resource browsing preference under the influence of the multimedia resource browsing information of the reference account associated with the target account, the generated target account characteristics are integrated with the target account's temporal relationship characteristics, transfer relationship characteristics, and resource browsing characteristics under the influence of social relationships, so that the target account characteristics are closer to the user's current real preference for multimedia resources, and the target multimedia resources that match the target account characteristics displayed to the target account are more in line with the target account's current demand for multimedia resources. The problem of low matching between the multimedia resources recommended for the target account and the target account's current resource preference for multimedia resources can be solved, so as to improve the matching between the multimedia resources recommended for the target account and the target account's current resource preference for multimedia resources.

[0051] Optionally, in the embodiment of the present application, multimedia resources refer to resources that can transmit information in various forms, including text, pictures, audio, video, etc. These resources can be used on various media platforms, such as the Internet, television, movies, radio, etc.

[0052] In the technical solution provided in the above step S202, the temporal relationship feature is the multimedia resource browsing preference feature currently possessed by the target account under the influence of the temporal relationship of multimedia resources browsed within multiple reference time periods. In an embodiment of the present application, the method for predicting the temporal relationship feature may be to aggregate the multimedia resource preference features of each reference time period in chronological order, for example, obtain the first feature of the target time period currently to be processed, wherein the first feature is used to indicate the browsing preference of the target account within the target time period, and the target time period is any time period among the multiple reference time periods; perform feature aggregation on the first feature and the initial aggregation feature to obtain the target aggregation feature, wherein the initial aggregation feature is obtained by performing feature aggregation on the multimedia resource browsing preference features of the target account in other time periods within the multiple reference time periods whose timing is before the target time period; when it is detected that the target time period is the last time period among the multiple reference time periods, the target aggregation feature is determined as the temporal relationship feature of the target account. In the above embodiment, when the first feature and the initial aggregated feature are aggregated, the feature merging method can be directly adopted, or the weight of the first feature can be assigned according to the importance of the first feature relative to the target account in the target time period, and the first feature is weighted according to the weight, and the weighted feature and the initial aggregated feature are used to perform feature aggregation, so as to obtain the target aggregated feature. The importance of the first feature relative to the target account can be determined by calculating the difference between the current first feature and the browsing partial features of the target account in the reference time period other than the target time period in multiple reference time periods. The smaller the difference, the higher the importance of the current first feature relative to the target account, and the first feature has a positive influence on the temporal relationship feature of the predicted target account. Therefore, the corresponding weight can be configured to be higher or the corresponding weight can be configured to be positive weight. When the difference is smaller, the importance of the current first feature relative to the target account is lower, and the first feature has a small influence on the temporal relationship feature of the predicted target account or even has a negative influence. Therefore, the corresponding weight can be configured to be lower or the corresponding weight can be configured to be negative weight. In the above embodiment, since the user will browse multiple multimedia resources in chronological order in each reference time period, the temporal relationship and transfer relationship of browsing multiple multimedia resources in the target time period can also be considered when determining the first feature of the target time period. In the above embodiment, when predicting the first feature of the target time period, it is necessary to assign a multimedia resource feature to each multimedia resource browsed by the target account in the target time period. In order to integrate the influence of the social relationship of the target account into the temporal relationship, the browsing information of the multimedia resources by the reference account in the social relationship network of the target account can be given to predict the resource features of the multimedia resources browsed by the target account in each target time period. The core principle is to use the account features that browsed the multimedia resource to reflect the resource features of the multimedia resource.

[0053] Optionally, in the embodiment of the present application, the temporal relationship feature is obtained by learning the temporal relationship between the resource browsing features of the target account in multiple reference time periods in the first browsing information. The temporal relationship feature can be, but is not limited to, extracted from the first browsing information using a temporal relationship learning network, which mainly consists of two parts: a session preprocessor and a two-level recurrent neural network based on GRU. The former is responsible for extracting the temporal relationship from each user. Multiple active sessions of the target account (reference time period, multimedia resource browsing characteristics) The preprocessor removes redundant or invalid active sessions; the latter is responsible for learning the temporal relationship of active sessions from the preprocessed sessions to obtain session representation.

[0054] Session preprocessing procedure: This procedure can be implemented with the help of a neighbor sampling function based on submodule theory. To simplify the design of a two-stage recurrent neural network based on GRU, the cardinality of the basic sessions obtained by sampling is fixed to , then the session sampling problem based on submodule theory can be modeled as follows:

[0055] ,

[0056] ,

[0057] The submodule function can be any function that can measure the "value" or "importance" of a set, which is not limited in this embodiment. Similarly, the above problem can be solved by the REGSA algorithm, which is not described in detail here. The historical session (including the multimedia resource browsing information of the target account in each reference time period in the multiple reference time periods in the first browsing information) The fixed length obtained by the session preprocessor is Basic activity session . Because the preprocessor may disrupt the underlying active session The multimedia resource access timing in Reorder the timing in and record the sorted basic activity session as .

[0058] Two-level recurrent neural network based on GRU: In order to extract the temporal relationship between sessions, this embodiment designs a two-level recurrent neural network based on GRU. Figure 3 As shown in Figure 1, the network consists of a session preprocessor, an intra-session recurrent neural network, an inter-session recurrent neural network, and a session representation database. Each session Remove redundant or invalid videos from the session to lengthen the session Convert to a fixed length Basic conversation ; The intra-session recurrent neural network is responsible for extracting session representations from the video sequence in each basic session of the user in turn, and inputting them into the session representation database for storage. The session representation of the last session is the output of the entire two-level recurrent neural network; the inter-session recurrent neural network generates the initial hidden state of the intra-session recurrent neural network according to the vector representation sequence of all sessions in the session database every time a data write operation occurs in the session representation database, that is, every time the intra-session recurrent neural network generates a session representation and writes it into the session representation database, that is, the output of the inter-session recurrent neural network is used as the initial state of the intra-session recurrent neural network. The reason for this design is that although the intra-session recurrent neural network can achieve good performance, it does not know any information about the user at the beginning of each session, and learns the user's interests during the session, but all this information is discarded at the end of the session. The inter-session recurrent neural network can improve the intra-session recurrent neural network because it takes into account the user's previous sessions and provides information to the intra-session recurrent neural network at the beginning of each new session.

[0059] Optionally, in an embodiment of the present application, the multimedia browsing information of the target account in each reference time period among multiple reference time periods has a different effect on the browsing resource preference of the target account at the current moment, that is, the multimedia resources browsed in different reference time periods may have a complex dependency relationship such as an incentive or inhibitory effect on the resource browsing preference of the user at the current moment, that is, the influence of the user's multimedia resource browsing preferences in different time periods on the user's current resource browsing preferences may have a complex dependency relationship such as an incentive or inhibitory effect, and by predicting the transfer relationship characteristics of the target account based on the first browsing information, the multimedia resource browsing preference is obtained under the incentive or inhibitory effect of the multimedia information browsed by the target account in each reference time period.

[0060] Optionally, in an embodiment of the present application, the transfer relationship feature can be understood as a feature that takes into account the high-level structural information (i.e., high-order feature relationship) between the multimedia resource browsing information in each reference time period. In order to predict the transfer relationship feature based on the first browsing information, a transfer relationship learning network model based on a weighted directed graph autoencoder can be used to learn the high-order transfer relationship between the multimedia resource browsing information in multiple reference time periods of the target account. In order to achieve this function, a weighted directed graph construction algorithm can be designed to model the multimedia resource browsing feature sequences of multiple reference time periods in the first browsing information and the transfer relationships between them as a weighted directed graph.

[0061] Construction of weighted directed graph. First, each user (target account) Browsing features of multiple reference time periods in the first browsing information Processed basic history session (i.e., the historical session is the multimedia resource browsing information within multiple reference time periods in the first browsing information, and the historical session includes multiple active sessions, each of which corresponds to the multimedia resource browsing information of the target account within a reference time period) is modeled as a weighted directed graph In this graph, each vertex Represents the browsing time of multimedia resources in a reference time period. Each directed edge represents the influence relationship between multimedia features browsed within a reference time period; Represents a directed edge The weight of , which describes the semantic distance or similarity of multimedia resource browsing features in different reference time periods. History of conversations As an example, we introduce the Weighted and Directed Intra-Session Graph Construction Algorithm (WISGC). Assume , the following are the specific steps of the algorithm:

[0062] 1. Vertex set determination: historical session The vertex set of all different active session graphs that appear in . (For the above session sequence, the vertex set is ,

[0063] 2. Edge counting and weight calculation: For adjacent active session pairs in a session, the feature similarity between them is counted, thereby constructing the feature weights of multimedia resource browsing in different reference time periods.

[0064] Design of weighted directed graph autoencoder. This embodiment designs a weighted and directed graph autoencoder (WDGAE) to effectively learn the high-order transition relationship between different active sessions in the historical session. The network contains Layer weighted directed graph encoder, The graph pooling layer is composed of a weighted directed graph decoder and a graph pooling layer. Each weighted directed graph autoencoder is responsible for mapping the hidden layer representation of each vertex in the weighted directed historical conversation graph to a low-dimensional space and outputting the low-dimensional representation vector of the vertex; each weighted directed graph decoder reconstructs the low-dimensional representation vector of the vertex into the predicted value of the adjacency matrix and outputs the reconstructed adjacency matrix. The graph pooling layer performs a pooling operation on the hidden layer representation of the vertices output by the last weighted directed graph decoder to extract the graph-level representation and obtain the embedded vector representation of the entire weighted directed historical conversation graph.

[0065] To this end, we provide users Basic activity session Maintain a weighted directed historical conversation graph , which is used to update the vertex embedding vector representation and the active session representation. The update process requires information such as vertex features and the topological structure of the graph. According to the network structure of the weighted directed graph autoencoder, the following introduces the network design details in three parts: encoder, decoder, and graph pooling layer.

[0066] Encoder design. Let's assume that the user Basic history session Any active session in In the The hidden layer learned by the layer graph encoder is represented as Dimensional vector . The weighted directed graph encoder layer performs graph convolution operations on the multimedia resource browsing features of each active session and outputs the The active session hidden layer representation of the layer:

[0067] ,

[0068] in, Is the basic history session of all active sessions in The matrix composed of layer hidden layer representation; is the adjacency matrix with self-connection, where represents the identity matrix, all diagonal elements are 1 and the rest are zero; yes The degree matrix of is the parameter matrix to be learned, and satisfies The above operations gradually map multimedia resource features from high-dimensional space to low-dimensional space.

[0069] Decoder design. The operation of the weighted directed graph decoder layer is similar. The weighted directed graph decoder layer performs graph convolution operations on the multimedia resource browsing features of each active session and outputs the The active session hidden layer representation of the layer:

[0070] ,

[0071] in, is the parameter matrix to be learned, and satisfies The above operations gradually map the active session features from the low-dimensional space back to the high-dimensional space. In addition, the last weighted directed graph decoder layer also performs an adjacency matrix reconstruction operation to generate neighbor matrix prediction values ​​based on the hidden layer representation of the vertex :

[0072] ,

[0073] Graph pooling layer design. The graph pooling layer is a method for fusing all vertex features in a graph into an embedded vector representation. This method aims to capture the global features of the graph and can be used for tasks such as graph classification, graph-level prediction, and graph generation. In order to extract the representation of intra-session transfer relations from the weighted directed historical conversation graph, this paper proposes three graph pooling methods: global average pooling, global maximum pooling, and attention pooling. , let’s remember any multimedia resource output by the last layer of weighted directed graph attention layer The hidden layer is represented as , record the weighted directed historical conversation graph The embedding vector is represented as (I stands for intra, T stands for transitional), that is, the user Basic history session The intra-session transfer relation embedding vector representation of . Then the three pooling methods are introduced as follows:

[0074] (a). Global Average Pooling (GAP): Global Average Pooling is a simple and effective graph-level pooling method that averages the hidden layer representations of all vertices to obtain the representation of the entire weighted directed history session graph. :

[0075] ,

[0076] Similarly, we can get the global maximum pooling (Global Max Pooling, GMP):

[0077] ,

[0078] in, It means to take the maximum value of all input vectors by component and return a new vector.

[0079] (b) Attention Pooling: Attention pooling is a pooling method based on the attention mechanism. It dynamically aggregates vertex features by learning attention weights. The steps of this method are as follows:

[0080] ,

[0081] ,

[0082] ,

[0083] in, is the parameter matrix to be learned, is the activation function (such as ReLU), is the vertex The score, is the vertex The attention weight.

[0084] In the technical solution provided in the above step S204, the resource browsing feature is a multimedia resource browsing preference that takes into account the target account under the influence of the social space. The resource browsing feature can be, but is not limited to, obtained by extracting features based on the second browsing information of the reference account associated with the target account, wherein the second browsing information is used to indicate the multimedia resource browsing situation of the reference account in multiple reference time periods before the current moment. In the embodiment of the present application, the method of predicting the resource browsing feature of the target account based on the second browsing information can be to reflect the first account feature of the reference account through the multimedia resource browsing feature recorded in the second browsing information of each reference account, and then assign the account weight of the first account feature of each reference account according to the interactive relationship between the target account and each reference account, and use the account weight to perform weighted sum calculation on the first account feature, thereby obtaining the account resource feature of the target account. Alternatively, feature extraction may be performed based on the first browsing information and the second browsing information to obtain resource browsing features. In an embodiment of the present application, feature extraction for the first browsing information and the second browsing information may be performed by reflecting the first account features of the reference account through the multimedia resource browsing features recorded in the second browsing information of each reference account, and reflecting the second account features of the target account through the first browsing information. Then, account weights of the first account features of each reference account are allocated based on the interaction relationship between the target account and each reference account, and the first account features and the second account are weighted and summed using the account weights to obtain the account resource features of the target account.

[0085] Optionally, in the embodiment of the present application, the target account feature can be generated by directly splicing the temporal relationship feature, the transfer relationship feature and the resource browsing feature to obtain the target account feature. Alternatively, the attention mechanism can be used to assign a corresponding attention weight to each feature according to the influence of the temporal relationship feature, the transfer relationship feature and the resource browsing feature on the target account browsing preference, and the attention weight is used to perform a weighted summation of the temporal relationship feature, the transfer relationship feature and the resource browsing feature to obtain the target account feature.

[0086] In the technical solution provided in the above step S206, the target multimedia resource may be a multimedia resource whose matching degree is greater than the target matching degree obtained by matching the multimedia resources in the multimedia resource library based on the characteristics of the target account. The target multimedia resource may be one or more. When there are multiple target multimedia resources, the multiple target multimedia resources may be sorted according to the matching degree with the characteristics of the target account.

[0087] As an optional embodiment, predicting the temporal relationship feature of the target account according to the first browsing information of the target account includes:

[0088] Filtering out a target time period to be processed currently from the multiple reference time periods according to the time sequence of the multiple reference time periods;

[0089] Acquire a first browsing feature of the first sub-browsing information of the target account within the target time period, wherein the first browsing feature is used to indicate the browsing preference of the target account within the target time period;

[0090] Predicting a second temporal relationship feature within the target time period based on a first temporal relationship feature within a candidate time period and the first browsing feature, wherein the candidate time period is adjacent to the target time period and is located before the target time period in a plurality of reference time periods, and the first temporal relationship feature is used to indicate the influence of the browsing preference of the target account in the time period before the candidate time period on the browsing preference of the target account in the candidate time period;

[0091] In a case where the target time period is the last one of the multiple reference time periods, the second temporal relationship feature is determined as the temporal relationship feature.

[0092] Optionally, in an embodiment of the present application, the first browsing feature of the multimedia resources that the target account accesses in chronological order in the target time period in the first sub-browsing information may be a feature that considers the temporal relationship of the multimedia resources accessed by the target account in the target time period, or may also be a feature that considers the high-order transfer relationship between the multimedia resources accessed by the target account in the target time period, or may also be a feature that considers both the temporal relationship and the transfer relationship, and this scheme does not limit this. Therefore, in an embodiment of the present application, a temporal relationship extraction model may be used to extract the first temporal feature of the multimedia resources included in the first sub-browsing information from the first sub-information as the first browsing feature; or, a transfer relationship extraction model may be used to extract the first transfer feature of the multimedia resources included in the first sub-browsing information from the first sub-browsing information as the first browsing feature; or, a temporal relationship extraction model may be used to extract the first temporal feature of the multimedia resources included in the first sub-browsing information from the first sub-information, and a transfer relationship extraction model may be used to extract the first transfer feature of the multimedia resources included in the first sub-browsing information from the first sub-browsing information, and the first temporal relationship and the first transfer relationship may be combined to obtain the first browsing feature.

[0093] Optionally, in the above embodiment, the temporal relationship extraction model can be a two-stage recurrent neural network based on GRU, which mainly consists of two parts: a session preprocessor and a temporal perception capsule network. The former is responsible for extracting the temporal relationship from each user. Active sessions (first child browsing information of the reference time period) The first one removes redundant or invalid multimedia resources; the second one is responsible for learning the temporal relationship between multimedia resources from the preprocessed sessions to obtain the session representation.

[0094] Session preprocessing program: This program is implemented based on the neighbor sampling function of submodule theory. To simplify the design of time-aware capsule network, the cardinality of the basic session obtained by sampling is fixed to . Then the session sampling problem based on submodule theory can be modeled as follows:

[0095] ,

[0096] ,

[0097] The submodule function can be any function that can measure the "value" or "importance" of a set, and this embodiment does not limit this. Similarly, the above problem can be solved by the REGSA algorithm, which will not be repeated here. Active sessions The fixed length obtained by the session preprocessor is Basic activity session . Because the preprocessor may disrupt the underlying active session The access sequence of multimedia resources in Reorder the timing in and record the sorted basic activity session as .

[0098] Optionally, in the above embodiment, the transfer relationship extraction model can be an intra-session transfer relationship learning network, which is an intra-session transfer relationship learning network model based on a weighted directed graph autoencoder, and is used to learn high-order transfer relations between different multimedia resources in the current basic session of each user. The transfer relationship extraction model can be a two-level recurrent neural network based on GRU.

[0099] Optionally, in an embodiment of the present application, a method for predicting a second temporal relationship feature based on a first temporal relationship feature and a first browsing feature may be to assign a merging weight to the first browsing feature according to the importance of the first browsing feature to the temporal relationship feature, and to weight the first browsing feature according to the merging weight and merge it with the first temporal relationship feature, thereby obtaining a second temporal relationship feature, wherein the importance of the first browsing feature to the temporal relationship feature may be, but is not limited to, determined by calculating the degree of difference between the first browsing feature and browsing features of other time periods in a plurality of reference time periods except the target time period. The greater the degree of difference, the greater the influence of the first browsing feature on the temporal relationship feature, and the more important it is; conversely, the smaller the influence of the first browsing feature on the temporal relationship feature, and the less important it is.

[0100] Through the above-mentioned embodiments, the prediction method of the temporal relationship features adopts a method of cyclically extracting the browsing features of multiple reference time periods in chronological order, thereby capturing the complex dependencies such as excitation or inhibition between the browsing features of multiple reference time periods, and improving the accuracy of the temporal relationship feature representation between sessions (between multiple reference time periods).

[0101] As an optional embodiment, the acquiring the first browsing feature of the first sub-browsing information of the target account within the target time period includes:

[0102] Filtering out a second multimedia resource to be processed currently from a plurality of first multimedia resources in a browsing order, wherein the first sub-browsing information includes a plurality of the first multimedia resources browsed by the target account in the browsing order within the target time period;

[0103] Predicting a second preference feature of the target account after browsing the second multimedia resource based on the first preference feature of the target account and the initial resource feature of the second multimedia resource, wherein the first preference feature is used to indicate the browsing preference of the target account after browsing the second multimedia resource under the influence of the multimedia resource browsed before the second multimedia resource, and the initial resource feature is used to indicate the account feature of the account browsing the second multimedia resource;

[0104] In a case where the second multimedia resource is the last multimedia resource browsed by the target account within the target time period, the second preference feature is determined as the first browsing feature.

[0105] Optionally, in an embodiment of the present application, the method of predicting the second preference feature based on the first preference feature and the initial resource feature can be that the attention mechanism scores an attention weight for each multimedia resource in the first sub-browsing information, and weights the initial resource feature according to the attention weight corresponding to the second multimedia resource, and merges the weighted feature value with the first preference feature to obtain the second preference feature.

[0106] Through the above content, a method of performing cyclic feature extraction on multiple multimedia resources included in the first sub-browsing information in chronological order is adopted to capture complex dependencies such as excitation or inhibition between multiple multimedia resources, thereby improving the accuracy of temporal relationship representation.

[0107] As an optional embodiment, predicting the second preference feature of the target account after browsing the second multimedia resource according to the first preference feature of the target account and the initial resource feature of the second multimedia resource includes:

[0108] extracting first adjustment information from the first preference feature using a first reset gate, wherein the first adjustment information is used to indicate the degree of influence of the first preference feature on the browsing preference of the target account;

[0109] The first update gate is called to update the initial resource feature according to the first adjustment information to obtain the second preference feature.

[0110] As an optional embodiment, before predicting the second preference feature of the target account after browsing the second multimedia resource based on the first preference feature of the target account and the initial resource feature of the second multimedia resource, the method further includes:

[0111] Detecting, in the first sub-browsing information, a third multimedia resource browsed by the target account before the second multimedia resource;

[0112] In a case where the third multimedia resource exists in the first sub-browsing information, determining the browsing preference feature of the target account after browsing the third multimedia resource as the first preference feature;

[0113] In a case where the third multimedia resource does not exist in the first sub-browsing information, the first temporal relationship feature is determined as the first preference feature.

[0114] Through the above content, by using a cyclic feature extraction method, the temporal feature is extracted from each multimedia resource in the first sub-browsing information in turn, that is, the session representation is extracted from the video sequence in each basic session of the user in turn, and when there is no third multimedia resource between the second multimedia resources to be processed, it is proved that the current second multimedia resource is the multimedia resource with the first sorting position in the first browsing information. At this time, the first temporal relationship feature is determined as the first preference feature, thereby linking the sub-browsing information of two adjacent reference time periods, because the sub-browsing information in the previous reference time period is considered to provide information for the temporal relationship of the sub-browsing information in the new reference time period after the new reference time period starts, thereby ensuring the accuracy of the temporal relationship feature extraction of the first sub-browsing information.

[0115] As an optional embodiment, the step of selecting the second multimedia resource to be processed from the plurality of first multimedia resources in a browsing order includes:

[0116] Calculating a reference resource parameter of each of the first multimedia resources, wherein the reference resource parameter is used to indicate a degree of dependency between the current multimedia resource and other multimedia resources among the plurality of the first multimedia resources except the current multimedia resource;

[0117] Filter out a fourth multimedia resource whose reference resource parameter is greater than or equal to a target resource parameter from the plurality of first multimedia resources;

[0118] The second multimedia resource to be processed is selected from the plurality of the fourth multimedia resources in a browsing order.

[0119] Optionally, in an embodiment of the present application, the dependency relationship between multimedia resources is used to indicate the correlation between multimedia resources, which can be, but is not limited to, reflected by the feature similarity between resource features of the multimedia resources. When the feature similarity is greater than or equal to the target similarity, it can be determined that there is a dependency relationship between the multimedia resources; or it can also be determined whether there is a dependency relationship between the multimedia resources by calculating the mutual information between the features of the multimedia resources.

[0120] Optionally, in the embodiment of the present application, the reference resource parameter is used as a parameter for measuring the "value" or "importance" of a session to which the multimedia resource belongs (among multiple multimedia resources browsed within the same reference time period), and the reference resource parameter of each first multimedia resource can be calculated based on the submodule theory. The submodule function is a function defined on a set, which is usually used to measure the "value" or "importance" of a set. A function is called a submodule function if for any set and any element , the following conditions are met:

[0121] ,

[0122] Intuitively, this means that the marginal gain of adding elements to a smaller set decreases as the size of the set increases. Submodule functions have many useful properties in optimization problems, such as being able to approximate the optimal solution through a greedy algorithm, and being able to be used to maximize or minimize a certain objective function. Common submodule functions include mutual information functions. Specifically in an embodiment of the present application, the submodule function may be, but is not limited to, a mutual information function. Mutual information is an important indicator in information theory used to measure the correlation between two random variables. It can be used to describe the degree of mutual dependence or the amount of shared information between two random variables. Suppose there are two random variables and , whose joint probability distribution is , and the marginal probability distributions are and , mutual information It can be defined as:

[0123]

[0124] in, and They are random variables and If and Independent of each other, , at this time the mutual information , in other words, a mutual information of zero indicates that the two random variables are completely independent. and If there is a dependency relationship between The larger the mutual information value, the and The higher the degree of connection between them, or the more information they share.

[0125] Through the above content, the reference resource parameter is used as a parameter for measuring the "value" or "importance" of multimedia resources. Since the multimedia resources browsed within a reference time period may include redundant multimedia resources or multimedia resources mistakenly browsed by the target account, the "value" of the redundant or mistakenly browsed multimedia resources is relatively low, and the degree of dependence between them and other multimedia resources in the session belonging to the current time period is relatively small. In this way, the screening of multimedia resources with lower value in the current session is achieved, thereby ensuring the accuracy of the calculation result of the first browsing feature.

[0126] As an optional embodiment, predicting the second temporal relationship feature within the target time period according to the first temporal relationship feature within the candidate time period and the first browsing feature includes:

[0127] extracting second adjustment information from the first temporal relationship feature using a second reset gate, wherein the second adjustment information is used to indicate the degree of influence of the first temporal relationship feature on the browsing preference of the target account;

[0128] The second update gate is called to update the first browsing feature according to the second adjustment information to obtain the second temporal relationship feature.

[0129] As an optional embodiment, predicting the transfer relationship feature of the target account according to the first browsing information includes:

[0130] Acquire a second browsing feature of the second sub-browsing information within each of the reference time periods, wherein the second browsing feature is used to indicate a browsing preference of the target account within the reference time period;

[0131] Calculate the browsing similarity between any two of the second sub-browsing information in the plurality of the second sub-browsing information by using the second browsing feature;

[0132] Generating a random walk sequence for target sub-browsing information in the plurality of second sub-browsing information according to the browsing similarity, wherein the random walk sequence takes the target sub-browsing information as a starting point, and the random walk sequence records the association relationship between the target number of second sub-browsing information in the plurality of second sub-browsing information, and the target sub-browsing information is any sub-browsing information included in the plurality of second sub-browsing information;

[0133] Predicting, according to the random walk sequence, an initial transfer relationship feature of the target sub-browsing information corresponding to the random walk sequence, wherein the initial transfer relationship feature is used to indicate the browsing preference of the target account under the influence of the corresponding target sub-browsing information;

[0134] The transfer relationship feature is generated according to the initial transfer relationship features of a plurality of the second sub-browsing information.

[0135] Optionally, in the embodiment of the present application, the browsing similarity can be identified by, but not limited to, pre-similarity, Gaussian distance, Manhattan distance, etc., and this solution does not limit this.

[0136] Optionally, in an embodiment of the present application, in the temporal relationship, there is a complex transfer relationship between adjacent or distant sessions (video browsing information within a reference time period is a session), and this relationship is very important for modeling users' long-term interest preferences.

[0137] Optionally, in an embodiment of the present application, a random walk sequence is generated for the target sub-browsing information in the second sub-browsing information based on the theory of random walk, thereby generating a sequence starting from the target sub-browsing information in the second sub-browsing information in multiple reference time periods, including multiple second sub-browsing information that have an associated relationship with the target sub-browsing information, thereby traversing the possible browsing information sequence combinations starting from the target sub-browsing information in this way, and thus the influence characteristics of the target sub-browsing information on the browsing preferences of the target user can be obtained.

[0138] Optionally, in an embodiment of the present application, since multiple random walk sequences are extracted for the target sub-browsing information by random walk, it is possible to extract features from each random walk sequence to obtain sequence features, and merge the sequence features of multiple random walk sequences to obtain initial transfer relationship features. In this embodiment, merging the sequence features of multiple random walk sequences can be a method of directly merging multiple sequence features, or it can also be based on an attention mechanism to assign weights to each sequence feature, and use the weights to perform weighted summation on multiple sequence features to obtain initial transfer relationship features.

[0139] Optionally, in an embodiment of the present application, the method of generating the transfer relationship feature based on the initial transfer relationship features of multiple second sub-browsing information may be to average multiple initial transfer relationship features to obtain the transfer relationship feature, or to extract the largest one from multiple initial transfer relationship features as the transfer relationship feature, or to assign weights to each initial transfer relationship feature based on an attention mechanism and use the weights to perform weighted summation of the initial transfer relationship features to obtain the transfer relationship feature, and this scheme does not limit this.

[0140] Through the above content, multiple random walk sequences are generated for each target sub-browsing information in a random walk manner, so as to traverse the influence relationship between the target sub-browsing information and other sub-browsing information in multiple second sub-browsing information, thereby reflecting the high-order influence characteristics of each target sub-browsing information on the browsing preference of the target account, thereby predicting the initial transfer relationship characteristics of the target sub-browsing information according to the random walk sequence, and predicting the transfer relationship characteristics of the target account based on the initial transfer relationship of each target sub-browsing information, thereby improving the accuracy of the transfer relationship characteristics.

[0141] As an optional embodiment, generating a random walk sequence for target sub-browsing information in a plurality of second sub-browsing information according to the browsing similarity includes:

[0142] Constructing a first relationship graph of a plurality of the second sub-browsing information according to the browsing similarity, wherein the first relationship graph includes a plurality of first vertices and a first directed edge between any two first vertices, each of the first vertices corresponds to one second sub-browsing information, and the first directed edge records the association relationship between the second sub-browsing information;

[0143] Extracting a first vertex sequence starting from the second vertex for each second vertex from the first relationship graph, wherein the second vertex is any vertex among the plurality of first vertices, and the first vertex sequence records association relationships between the target number of first vertices among the plurality of first vertices;

[0144] The second sub-browsing information corresponding to the first vertices in the first vertex sequence is sorted according to the association relationship between the first vertices recorded in the first vertex sequence to obtain the random walk sequence.

[0145] Through the above content, the first relationship graph of the second browsing information is constructed by the browsing similarity between the second sub-browsing information, and the random walk sequence is obtained by performing random sequence extraction on the first relationship graph, thereby achieving accurate and efficient extraction of the random walk sequence.

[0146] As an optional embodiment, extracting, from the first relationship graph, for each second vertex, a first vertex sequence starting from the second vertex includes:

[0147] In the process of extracting the first vertex sequence, searching the first relationship graph for a fourth vertex that has an association relationship with a third vertex, wherein the third vertex is a vertex for searching a next associated vertex, and an outgoing end of the first directed edge connecting the fourth vertex and the third vertex points to the fourth vertex;

[0148] Allocating a screening weight to each of the fourth vertices according to the vertex association degree indicated by the first directed edge;

[0149] Randomly select a fifth vertex located after the third vertex in the vertex sequence from the plurality of fourth vertices according to the screening weight;

[0150] Detect the reference number of vertices included in the currently filtered vertex sequence;

[0151] When the reference number is equal to the target number, the vertex sequence that has been currently screened out is determined as the first vertex sequence.

[0152] Optionally, in an embodiment of the present application, there is a positive correlation between the vertex association degree and the screening weight of the fourth vertex, that is, the greater the vertex association degree, the greater the screening weight.

[0153] Through the above content, the screening weight of each vertex to be screened is allocated according to the association between the vertices, thereby realizing random screening of vertices based on vertex association, avoiding screening errors caused by screening according to fixed rules, and allowing the screened first vertex sequence to traverse more possible sequence combinations.

[0154] As an optional embodiment, the step of constructing a plurality of first relationship graphs of the second sub-browsing information according to the browsing similarities includes:

[0155] Creating the first vertex corresponding to each of the second sub-browsing information, and configuring the vertex association between the first vertices corresponding to the second sub-browsing information according to the browsing similarity between the second sub-browsing information;

[0156] Creating a first connecting edge between the first vertices whose vertex association degree is greater than or equal to a target association degree;

[0157] According to the occurrence time sequence relationship of the second sub-browsing information corresponding to the first vertex, the pointing relationship of the first connecting edge is configured to obtain the first directed edge, wherein the incident end of the first directed edge is connected to the vertex that occurs earlier in the time sequence, and the outgoing end of the first directed edge is connected to the vertex that occurs later in the time sequence;

[0158] A relationship graph of the first directed edges that records connections between a plurality of the first vertices and the first vertices is determined as the first relationship graph.

[0159] Through the above content, by creating the first relationship graph, the association relationship and transfer relationship between the second sub-browsing information are more intuitively represented.

[0160] As an optional embodiment, predicting the initial transfer relationship feature of the target sub-browsing information corresponding to the random walk sequence according to the random walk sequence includes:

[0161] Calculating a target time series feature of each random walk sequence in a plurality of random walk sequences of each second sub-browsing information, wherein the time series feature is used to indicate a time series change of the browsing preference of the target account in the random walk sequence;

[0162] Allocating a sequence weight to each of the random walk sequences according to the proportion of the target time series feature of each of the random walk sequences in the time series features and values ​​of the plurality of random walk sequences;

[0163] The initial transfer relationship feature is obtained by weighted summing of the plurality of time series features according to the sequence weight.

[0164] As an optional embodiment, the calculating the target time series feature of each random walk sequence in the multiple random walk sequences of each second sub-browsing information includes:

[0165] Determine the sixth sub-browsing information to be processed from the random walk sequence according to the occurrence sequence of the sub-browsing information;

[0166] Predicting a fourth preference feature of the target account after browsing the sixth sub-browsing information according to the third preference feature of the target account and the second browsing feature corresponding to the sixth sub-browsing information, wherein the third preference feature is used to indicate the browsing preference of the target account before browsing the sixth sub-browsing information in the random walk sequence;

[0167] In a case where the sixth sub-browsing information is the last sub-browsing information in the random walk sequence, the fourth preference feature is determined as the target time series feature of the random walk sequence.

[0168] Optionally, in an embodiment of the present application, the method for predicting the fourth preference feature based on the third preference feature and the second browsing feature may be to directly merge the third preference feature and the second browsing feature, or to assign a weight to the sixth sub-browsing information according to the importance of the sixth sub-browsing information in the session, use the weight to weight the second browsing feature, and merge the weighted second browsing feature and the third preference feature to obtain the fourth preference feature.

[0169] As an optional embodiment, predicting a fourth preference feature of the target account after browsing the sixth sub-browsing information based on the third preference feature of the target account and the second browsing feature corresponding to the sixth sub-browsing information includes:

[0170] extracting second adjustment information from the third preference feature using a third reset gate, wherein the second adjustment information is used to indicate the degree of influence of the third preference feature on the browsing preference of the target account;

[0171] The third update gate is called to update the second browsing feature according to the second adjustment information to obtain the fourth preference feature.

[0172] As an optional embodiment, generating the transfer relationship feature according to the initial transfer relationship features of the plurality of second sub-browsing information includes one of the following:

[0173] Determining the average of the plurality of initial transfer relationship features as the transfer relationship feature;

[0174] Determining a maximum value of a feature among a plurality of the initial transfer relationship features as the transfer relationship feature;

[0175] A corresponding feature weight is assigned to each of the initial transfer relationship features according to the feature relationship between the initial transfer relationship features; and the initial transfer relationship features are weighted summed using the feature weights to obtain the transfer relationship features.

[0176] Through the above content and the above method, the features of all the second sub-browsing information are converted into overall features, thereby comprehensively describing the transfer relationship features of the target account within multiple reference time periods.

[0177] As an optional embodiment, before generating the target account feature of the target account according to the time sequence relationship feature, the transfer relationship feature and the resource browsing feature of the target account, the method further includes:

[0178] Acquire second browsing information of the reference account, wherein the second browsing information is used to indicate multimedia resource browsing status of the reference account within a plurality of the reference time periods;

[0179] The resource browsing characteristics of the target account are predicted according to the first browsing information and the second browsing information.

[0180] Optionally, in an embodiment of the present application, the resource browsing feature is a browsing preference feature that takes into account the social relationship influence dimension of the target account. Therefore, the multimedia resource browsing preference feature of each reference account can be predicted based on the second browsing information, and the influence relationship of each reference account on the multimedia resource browsing of the target account can be calculated based on the similarity relationship between the first browsing information and the second browsing information, and weights are assigned to the multimedia resource browsing preference features of multiple reference accounts based on the influence relationship, and the multimedia resource browsing preference features of multiple reference accounts are weighted and summed using the weights to obtain the resource browsing feature of the target account.

[0181] As an optional embodiment, predicting the resource browsing characteristics of the target account according to the first browsing information and the second browsing information includes:

[0182] Constructing a second relationship graph according to the first browsing information and the second browsing information, wherein the second relationship graph includes a plurality of sixth vertices, a plurality of seventh vertices, a first edge and a second edge, wherein the sixth vertex corresponds to the target account and the reference account in a one-to-one manner, the seventh vertex corresponds to the multimedia resources included in the first browsing information and the multimedia resources included in the second browsing information in a one-to-one manner, the first edge is used to indicate an association relationship between accounts, and the second edge is used to indicate an access relationship between an account and a multimedia resource;

[0183] Extracting target vertex features of a target vertex in the second relationship graph using a feature extraction model including a plurality of sequentially connected feature extraction layers, wherein the target vertex is any vertex in the second relationship graph, each of the feature extraction layers is used to generate vertex features of the target vertex according to initial vertex features of a reference vertex in the current second relationship graph, and updating vertex features of the target vertex in the second relationship graph using the vertex features of the target vertex output by the current feature extraction layer, wherein the reference vertex is a vertex in the second relationship graph that has an association relationship with the target vertex;

[0184] The target vertex feature of the vertex corresponding to the target account is determined as the resource browsing feature.

[0185] Optionally, in an embodiment of the present application, the feature extraction model is used to iteratively update the relationship between vertices in the second relationship graph through multiple sequentially connected feature extraction layers, that is, each feature extraction layer is used to update the features of the current vertex according to the features of vertices that have an associated relationship with the current vertex, thereby randomly iterating the feature extraction layer, so that the features of the current vertex can be influenced by the features of vertices that are farther away.

[0186] Optionally, in an embodiment of the present application, each feature extraction layer can perform a graph convolution operation on each central vertex, and then learn different weights for the graph convolution results of different central vertices based on the attention mechanism to alleviate the imbalance.

[0187] Through the above content, by using a feature extraction model including multiple sequentially connected feature extraction layers to perform multiple rounds of iterative extraction on the vertex features of the target vertices in the second relationship graph, the feature relationship between vertices with a large distance is extracted, thereby improving the feature information dimension of the target vertex features.

[0188] As an optional embodiment, extracting the target vertex feature of the target vertex in the second relationship graph using a feature extraction model including a plurality of sequentially connected feature extraction layers includes:

[0189] Calling a target feature extraction layer to extract a target vertex set of the target vertex from the second relationship graph, wherein the target vertex set records vertices in the second relationship graph that are connected to the target vertex through the first edge or the second edge, and the target feature extraction layer is an extraction layer that currently performs feature extraction on the target vertex in the second relationship graph;

[0190] Calling the target feature extraction layer to perform feature aggregation on the initial vertex features currently possessed by the vertices included in the target vertex set, to obtain the reference vertex features of the target vertices output by the target feature extraction layer;

[0191] In a case where the target feature extraction layer is the last layer of a plurality of sequentially connected feature extraction layers, the reference vertex feature is determined as the target vertex feature.

[0192] Optionally, in the embodiment of the present application, when the feature extraction layer is the first layer, the vertex features can be extracted using an existing feature extraction model. For example, the features of user vertices can be extracted using existing deep learning technologies such as Transformer, CNN, etc. from the text, pictures, and other information provided by the user when registering to extract a high-dimensional vector as the user's features. The same is true for the vertices corresponding to multimedia resources, except that the existing technologies used may be different, such as ViTTransformer, etc.

[0193] Optionally, in an embodiment of the present application, when the target vertex is a vertex of an account, the vertices in the target vertex set need to include two types of vertices: vertices corresponding to multimedia resources and vertices corresponding to the account.

[0194] Optionally, in an embodiment of the present application, in order to improve the reliability of information when extracting the target vertex set, maximum value extraction can be used for extraction, that is, the vertex with the highest correlation with the current target vertex and the greatest benefit in reflecting the characteristics of the target vertex is extracted from the second relationship graph. Therefore, in an embodiment of the present application, a greedy strategy can be used to perform neighbor sampling in the second relationship graph, and the importance relationship between vertices can be calculated according to the submodule function theory when performing neighbor sampling.

[0195] Optionally, in the embodiment of the present application, when the central vertex is the vertex corresponding to the account (the vertex corresponding to the user) When assuming that the user and any of its neighbor vertices The The hidden layer representations of the output of the layer-heterogeneous graph convolutional neural network are Dimensional vector and , these neighbor vertices include both user vertices and vertices corresponding to multimedia resources. , No. The operations of the layer-heterogeneous graph convolutional neural network layer can be formally described as follows:

[0196] Implement graph convolution operations for all neighbor vertices of type "account":

[0197] ,

[0198] For all neighbor vertices of type "Multimedia Resources":

[0199] ,

[0200] Based on the attention mechanism, weights are dynamically assigned to the aggregation results of the two types of vertices and fused:

[0201] ,

[0202] ,

[0203] in, Indicates user The set of all user vertices in the intersection of the neighbor set and the basic neighbor subset; Indicates user The set of all multimedia resource vertices in the intersection of the neighbor set and the basic neighbor subset; represents a nonlinear activation function, represents the power operation of natural exponents; All belong to the parameter matrix to be learned.

[0204] Optionally, in the embodiment of the present application, when the central vertex is the vertex corresponding to the multimedia resource When, according to the user-multimedia resource heterogeneous graph (the second relationship graph) As can be seen from the definition, it has only one type of neighbor, the user vertex. Assume that the multimedia resource vertex and any of its neighbor vertices The The hidden layer representations of the output of the layer-heterogeneous graph convolutional neural network are Dimensional vector and .because The neighbor vertex of The operation of a layer of a heterogeneous graph convolutional neural network can be defined as:

[0205] ,

[0206] in, All belong to the parameter matrix to be learned.

[0207] when At that time, Layer-heterogeneous graph convolutional neural network layer based on user feature matrix and multimedia resource feature matrix is the input. When , the heterogeneous graph convolutional neural network outputs the user embedding vector representation enhanced by social semantics and video embedding vector representation .in, Represents the dimension of the embedding vector representation of users and multimedia resources.

[0208] Through the above content, by calling the target feature extraction layer, global knowledge such as friend relationships in the social space and all historical interaction records between users and multimedia are captured, which greatly improves the accuracy and efficiency of modeling users' long-term interest preferences.

[0209] As an optional embodiment, calling the target feature extraction layer to perform feature aggregation on the initial vertex features currently possessed by the vertices included in the target vertex set to obtain the reference vertex features of the target vertices output by the target feature extraction layer includes:

[0210] Detecting the vertex type of the target vertex;

[0211] In the case where the vertex type indicates that the target vertex is a vertex corresponding to an account, feature aggregation is performed on the initial vertex features of the sixth vertex included in the target vertex set to obtain a first aggregate feature, and feature aggregation is performed on the initial vertex features of the seventh vertex in the target vertex set to obtain a second aggregate feature; corresponding aggregate feature weights are assigned to the first aggregate feature and the second aggregate feature according to a feature relationship between the first aggregate feature and the second aggregate feature; and weighted sum calculation is performed on the first aggregate feature and the second aggregate feature using the aggregate feature weight to obtain the reference vertex feature;

[0212] In a case where the vertex type indicates that the target vertex is a vertex corresponding to a multimedia resource, the initial vertex features of the vertices included in the target vertex set are aggregated to obtain the reference vertex features.

[0213] Through the above content, weights are dynamically allocated between different types of vertices according to the attention mechanism, which effectively solves the problem of information imbalance between different types of features.

[0214] As an optional embodiment, the calling the target feature extraction layer to extract the target vertex set of the target vertex from the second relationship graph includes:

[0215] Extracting an initial vertex set of the target vertex from the second relationship graph, wherein the initial vertex set includes vertices in the second relationship graph connected to the target vertex through the first edge or the second edge;

[0216] A number of candidate vertices are screened out from the initial vertex set according to the current vertex features of the vertices in the initial vertex set to obtain the target vertex set, wherein, when the target vertex is a vertex corresponding to an account, the candidate vertices include vertices corresponding to the account and vertices corresponding to multimedia resources, and when the target vertex is a vertex corresponding to multimedia resources, the candidate vertices include vertices corresponding to the account, and the dependency relationship between the vertices in the target vertex set and the target vertex satisfies the target dependency relationship condition.

[0217] As an optional embodiment, the step of selecting a candidate number of candidate vertices from the initial vertex set according to the current vertex features of the vertices in the initial vertex set to obtain the target vertex set includes:

[0218] Using the vertex features of each vertex included in the first vertex set of the current screening round, calculating the incremental value of each vertex to the vertex set information, wherein the vertex set information is used to indicate the importance of the vertex features of all vertices in the second vertex set, and the incremental value is used to indicate the amount of change of the vertex set information of the second vertex set after the corresponding vertex in the first vertex set is added to the second vertex set, and the initial vertex set includes vertices in the first vertex set and vertices in the second vertex set;

[0219] Extracting vertices whose increment values ​​are greater than or equal to the target increment value from the first vertex set and adding them to the second vertex set;

[0220] When the number of candidate vertices included in the second vertex set to which vertices are added is the candidate number, and the candidate vertices include both vertices corresponding to accounts and vertices corresponding to multimedia resources, the second vertex set is determined as the target vertex set.

[0221] Optionally, in an embodiment of the present application, a candidate number of candidate vertices are screened out from the initial vertex set according to the vertex features currently possessed by the vertices in the initial vertex set, and a method for obtaining the target vertex set can also be: randomly screen out a candidate number of vertices from the initial vertex set to obtain a first vertex set; use a mutual information function to calculate the vertex features currently possessed by the vertices in the first vertex set to obtain the target mutual information of the first vertex set; extract an alternative vertex from the remaining vertices in the initial vertex set; use the vertex features currently possessed by the alternative vertex to calculate the incremental value of the alternative vertex to the target mutual information; when the incremental value meets the target incremental value condition, use the alternative vertex to update the vertices in the first vertex set to obtain a second vertex set; detect the number of remaining vertices in the initial vertex set; and when it is determined that there are no remaining vertices in the initial vertex set, determine the second vertex set as the target vertex set.

[0222] Optionally, in the embodiment of the present application, mutual information function: Mutual information is an important indicator used in information theory to measure the correlation between two random variables. It can be used to describe the degree of mutual dependence or the amount of shared information between two random variables. Suppose there are two random variables and , whose joint probability distribution is , and the marginal probability distributions are and , mutual information It can be defined as:

[0223]

[0224] in, and They are random variables and If and Independent of each other, , at this time the mutual information In other words, a mutual information of zero indicates that the two random variables are completely independent. and If there is a dependency relationship between The larger the mutual information value, the and The higher the degree of correlation between them, or the more information they share. The calculation of the mutual information function can be regarded as a measure of the degree of information sharing between two random variables, and can be applied in many fields, including machine learning, information retrieval, signal processing, etc. In machine learning, mutual information is often used in tasks such as feature selection, cluster analysis, and semi-supervised learning to help evaluate the correlation and information between features. For egocentric graphs , based on the mutual information principle, the submodule function is defined as:

[0225]

[0226] in, Is a collection About Vertex Sets The complement of , .

[0227] Optionally, in the embodiment of the present application, the target vertex set is screened by using a redundant information removal approximate solution algorithm based on a greedy strategy, for each vertex in the second relationship graph Redundant neighbors and noise in the social space and / or item space are removed to obtain a set of basic neighbor subsets that are non-redundant, representative, and diverse, laying the foundation for effectively extracting user embeddings and multimedia resource embeddings (i.e., features) enhanced with social relations.

[0228] As an optional embodiment, generating the target account feature of the target account according to the time sequence relationship feature, the transfer relationship feature and the resource browsing feature of the target account includes:

[0229] Performing feature splicing on the temporal relationship feature, the transfer relationship feature and the resource browsing feature to obtain a spliced ​​feature;

[0230] The splicing features are fused using a feature fusion model to obtain the target account features output by the feature fusion model, wherein the feature fusion model records the conversion relationship between the splicing features and the account features.

[0231] As an optional embodiment, presenting the target multimedia resource matching the target account feature to the target account includes:

[0232] Calculating target similarity between the target account feature and target resource features of each multimedia resource in the multimedia resource library;

[0233] Filtering the target multimedia resources whose target similarity is greater than or equal to a preset similarity from the multimedia resource library;

[0234] The target multimedia resource is displayed to the target account.

[0235] As an optional embodiment, the calculating the target similarity between the target account feature and the target resource feature of each multimedia resource in the multimedia resource library includes:

[0236] Generate a first resource feature of the multimedia resource in the first browsing information and a second resource feature of the multimedia resource in the multimedia resource library using a feature generation model;

[0237] Detecting feature difference information between the first resource feature and the second resource feature;

[0238] Using the feature adjustment amount corresponding to the feature difference information, feature adjustment is performed on the initial resource feature of the multimedia resource in the first browsing information to obtain the target resource feature, wherein the initial resource feature is used to indicate the account feature of the account browsing the multimedia resource in the first browsing information;

[0239] The target similarity between the target account feature and the target resource feature is calculated.

[0240] When recommending multimedia resources, a recommendation method based on sessions (i.e., multimedia resource browsing information of an account within a reference time period) is often used. This recommendation method has the following problems: (a) Only the current session is used to implement the recommendation, ignoring the complex dependencies such as incentives or inhibitions between historical sessions. For example, if a user is interested in romantic comedy movies in a previous session, he may maintain this interest or even increase it in the current session, which is the incentive effect between sessions. Or if a user buys a new laptop in the last few sessions, he is likely not to be interested in buying another laptop in the current session, which is the inhibition effect between sessions. (b) There is a large amount of noise data in the session, such as items clicked by mistake or items clicked repeatedly. These noises seriously affect the modeling of user interest preferences, and thus affect the accuracy of recommendations. (c) There are many redundant calculations, and additional sessions from social friends need to be processed to make recommendations for the current session. (d) The underlying hardware deployed by the system usually uses GPU for training and reasoning, which has low computing efficiency, high power consumption, and low energy efficiency. The embodiment of the present application takes the multimedia resource to be recommended as a video as an example to explain the display logic of the above-mentioned multimedia resource display method.

[0241] The embodiment of the present application proposes a social, temporal, and transitional semantic-aware Video Recommendation system based on Historical Sessions (SVRHS). Figure 3 is an optional video recommendation schematic diagram according to an embodiment of the present application, such as Figure 3As shown, it consists of a user-video heterogeneous graph construction component, a heterogeneous graph embedding component, a personalized session embedding component, a user interest preference fusion component, and a prediction component. Among them, the user-video heterogeneous graph construction component is responsible for constructing a user-video heterogeneous graph containing two vertices and two connecting edges (relationships) from the friendship between users in the social space and the interaction between the social space and the project space. In the heterogeneous graph embedding component, the embodiment of the present application designs a redundant information removal approximate solution algorithm REGSA (Redundancy Elimination algorithm based on Greedy Strategy with Approximation) based on a greedy strategy, samples the neighbors of each vertex in the user-video heterogeneous graph, removes redundant information and noise; designs a heterogeneous graph convolutional neural network, and extracts social semantics enhanced user embedding and video embedding from the sampled user-video heterogeneous graph. The personalized session embedding component is further divided into a temporal relationship learning network between sessions and a transfer relationship learning network between sessions. The former uses a two-level recurrent neural network based on GRU (Gated Recurrent Unit) to learn the temporal relationship between sessions to extract the embedded vector representation of the temporal relationship between sessions; the latter uses a weighted directed graph random walk recurrent convolutional neural network to extract features from the weighted directed historical session graph obtained by the weighted directed graph construction algorithm to obtain an embedded vector representation of the transfer relationship between sessions. The embedded vector representation of the temporal relationship between sessions, the embedded vector representation of the transfer relationship between sessions, and the user embedding as the user's long-term interest preference are used to obtain the user's comprehensive interest preference through the user interest preference fusion component. The prediction component calculates the probability of the user watching all videos that have not been visited based on the learned user comprehensive interest preference, and returns the video with the highest probability or a group of videos to the user as the recommendation result. In order to improve the energy efficiency of training and inference, the embodiment of the present application also designs a general heterogeneous accelerator based on FPGA for heterogeneous graph convolutional neural networks, two-level recurrent neural networks based on GRU, and weighted directed graph random walk recurrent convolutional neural networks.

[0242] The proposed recommendation method comprehensively considers the social relations in the social space, the temporal relations between historical conversations of user-video interactions, and the influence of high-order transfer relations on user interest preferences. It innovatively proposes a user-video heterogeneous graph construction algorithm to model the two interaction relationships between the two entities, users (i.e., accounts, including the target accounts and reference accounts mentioned above) and videos (corresponding to the multimedia resources mentioned above) in the video recommendation system; innovatively proposes a neighbor sampling algorithm based on sub-module theory to remove information redundancy and noise in the social space and project space; designs a heterogeneous graph convolutional neural network to mine the complex interaction relationships between users and videos in the social space and project space, and learns user embedding and video embedding enhanced by social semantics. Among them, the user embedding enhanced by social semantics accurately depicts the user's long-term interest preferences; the innovative two-level recurrent neural network based on GRU is proposed to learn the temporal relationship between different sessions from the user's historical sessions with redundancy and noise removed, and extract the temporal relationship embedding vector representation between sessions; the weighted directed historical session graph construction algorithm is innovatively proposed to convert the user's historical sessions into weighted directed graphs to model the high-order transfer relationship between sessions; the inter-session transfer relationship learning component based on the weighted directed graph random walk recurrent convolutional neural network is innovatively proposed to extract features from the weighted directed historical session graph and generate the inter-session transfer relationship embedding vector representation. The inter-session temporal relationship embedding vector representation and the inter-session transfer relationship embedding vector representation respectively depict the user's dynamic interest preferences from the temporal and transfer relations between sessions. After being integrated with the user's long-term interest preferences, they obtain the user's comprehensive interest preferences enhanced by social, temporal and transfer semantics, which more accurately and personally depicts the user's interests and can greatly improve the accuracy of video recommendations. The proposed method only uses historical user conversations to generate social semantics to enhance user and video embeddings during heterogeneous graph embedding. When making recommendations for historical conversations, it avoids repeated learning of social friends' historical conversations to integrate social influences, greatly reducing the computational complexity of the network model. In addition, we have also designed a general heterogeneous accelerator based on FPGA to improve the training and reasoning efficiency of heterogeneous graph convolutional neural networks, GRU-based two-level recurrent neural networks, and weighted directed graph random walk recurrent convolutional neural networks, and can significantly save energy consumption and reduce costs.

[0243] First, the basic symbols used in the embodiments of the present application and the problems to be solved are introduced. Specifically, all vectors and scalars are represented by bold lowercase letters (such as ) and lowercase letters (such as ) and use bold, italic, and uppercase letters to denote matrices (e.g. ); Use italic capital letters to represent tuples (such as ); capital cursive letters to denote sets (e.g. ); In particular, use uppercase Euclid Math Two font to represent the range of variables (such as the and the set of integers ).

[0244] let and Respectively represent the user set and video set in the recommendation system VRSTT. There is a friend relationship between users; there is an interactive relationship between users and videos (when users Visited the video ,say and There are interactive relationships). Graphs are usually used to model the relationships between these entities.

[0245] Definition 1 (Graph) is a tuple consisting of a vertex set and a connecting edge set, which is used to describe the relationship between pairs of vertices. It is usually denoted as Among them, the vertex set Non-empty, each element represents a vertex; connected edge set Represents the connection relationship between a pair of vertices, and each element represents a connecting edge.

[0246] In addition to representing the graph as a set of two-tuples, we can also use the adjacency matrix Represents the association relationship between vertices. Row, No. Column Elements Represents a vertex and The relationship between them. Vertices directly connected by connecting edges For its first-order neighbors, all first-order neighbors form the vertex Neighborhood set , so there is . Vertex Neighborhood Set Cardinality Vertex The degree of .

[0247] Depending on whether the connecting edges have weights, graphs can be divided into weighted graphs and unweighted graphs. For weighted graphs, each edge All with weight , which is used to describe the similarity between a pair of vertices connected by the edge. Adjacency matrix of weighted graph Middle Row, No. Column Elements Represents a vertex and The similarity between and When there are connecting edges between ;otherwise For an unweighted graph, when and When there are connecting edges between ;otherwise .

[0248] Depending on whether the connecting edges have directions, the graph can be divided into directed graphs and undirected graphs, and recorded and The directed and undirected edges between them are and . when When it is a directed graph, the vertices Neighborhood set It can be divided into the incident neighbor set and the outgoing neighbor set . So there is .in, , . Accordingly, the vertex The degree is further divided into in-degree and out-degree , and are defined as and .

[0249] According to the combination principle, graphs can also be divided into weighted directed graphs, weighted undirected graphs, unweighted undirected graphs and unweighted directed graphs. Figure 4 kind.

[0250] Definition 2 (Ego Graph) is used to describe the relationship between a vertex (called the central vertex) and its first-order neighbor vertices in the graph. Given an undirected graph , each vertex of Egocentric map Can be defined as The subgraph with the central vertex , including the central vertex And all the neighboring vertices directly connected to it and the edges between them. Among them, , . Correspondingly, the egocentric graph of each vertex in the directed graph can also be defined, which is omitted here.

[0251] Definition 3 (Social Graph) is a graph that models the friendship between users. It consists of a set of user vertices and a set of connected edges that represent friendship between users. It is usually expressed as .in, Represents a collection of friend relationships between users.

[0252] In the social graph In each user Both Dimensional Features , this feature can be extracted from the user's own description information through pre-trained deep learning models such as Bert, Word2Vec, etc. The features of all users constitute the user feature matrix .user There are several friends, and the set of these friends is , and call them From neighbors in the social space. Since the friend relationship is a symmetric relationship, that is, if the user Yes Friends, then Too friends, thus the social graph is an undirected graph. Obviously, for any undirected graph, we can convert it into a directed graph by replacing each of its edges with a pair of directed edges in opposite directions.

[0253] In addition, each user It is also associated with a length of Sequence of conversations .in, Indicates the current moment; Indicates user In time period The possible behaviors include video clicks, favorites, and ratings. They form a time-ordered session, and the video Viewed before , and said and There is a transfer relationship, denoted as The length of the session is recorded as ,The session lengths of different users may be the same or different. ,The interaction relationship between users and videos is usually modeled with the help of ,user-video bipartite graph.

[0254] Definition 4 (User-video Bipartite Graph) is a graph that models the interaction relationship between users and videos. It consists of a set of user vertices and a set of video vertices as well as a set of connected edges representing the interaction relationship between users and videos. It is usually expressed as . in, represents a vertex set; represents the set of user-video interaction edges.

[0255] In the user-video bipartite graph In each user The neighbor set of is composed of all videos that the user has watched, which can be recorded as , then ; Each video The neighbor set of is only composed of all users who have watched the video. Let's remember these neighbor sets from the social space as , then . Although the fact that the interaction action is directional, the user Videos they watched The connecting edge between them should be a directed edge, but in order to facilitate the subsequent propagation and fusion of the friend relationship in the social space and the transfer relationship in the project space in the two spaces through the interactive relationship, this embodiment considers that the user-video bipartite graph is an undirected graph.

[0256] For each video , extract the visual, audio and text features of the video through existing deep learning technologies such as ResMNet, SoundNet and GloVe, and fuse them to obtain Dimensional Features , the features of all videos form the video feature matrix .

[0257] Definition 5 (Video Transition Graph) is a directed graph that models the transition relationship between videos. It consists of a set of video vertices and a set of directed connected edges that describe the transition relationship between videos. It is usually expressed as . in, is the set of vertices consisting of all videos; Represents the transition relationship between videos.

[0258] In the video recommendation system, a video transition graph can be constructed based on the historical access records of all users. Specifically, for each user Sequence of conversations , find all consecutively accessed video pairs and ,in is the starting video of the visit. is the video that is accessed later. Each video and Both represent a directed edge , the directed edges obtained from all video pairs of all users constitute the directed edge set of the video transfer relationship graph .video Neighborhood set By the incident neighbor set and the outgoing neighbor set Together they constitute .in, , .

[0259] Definition 6 (User-video Heterogeneous Graph) is a graph containing two types of vertices: users and videos, and two types of edges: social edges between users and interaction edges between users and videos. It is used to model the friendship relationship in the social space and the interaction relationship in the interaction space.

[0260] The user-video heterogeneous graph can be formed by merging the social graph and the user-video bipartite graph. Formally, the user-video heterogeneous graph can be represented as a four-tuple . in, and Represent the vertex set and the connected edge set respectively; and Vertex type mapping function and connection edge type mapping function respectively; and They are vertex type sets and edge type sets respectively. Contains two types of vertices: user and video; It includes the social relationship edges between users (social) and the interaction relationship edges between users and videos (interaction). According to Definitions 2 and 3, users Neighborhood set ; For video For example, its neighbors are only user-type vertices, so its neighbor set .

[0261] In order to construct a user-video heterogeneous graph from the social relationships between users and the interactive relationships between users and items in the video recommendation system, this embodiment proposes a user-video heterogeneous graph construction algorithm (User-video Heterogeneous Graph Construction Algorithm, UHGC), and the specific process is as follows:

[0262] 1. Initialization: Create an empty user-video heterogeneous graph , and define a set of vertex types and the set of connecting edge types ;

[0263] 2. Add vertices: Traverse all users and videos in the video recommendation system and add each user and each video as a single vertex to the vertex set in the graph and add them to the user vertex set according to the vertex type. and video vertex set , build vertex type mapping function ;

[0264] 3. Add edges: According to the social relationships between users and the interactive relationships between users and videos, add corresponding edges to the graph. Specifically:

[0265] (a) For the social relationship edges between users, traverse the social relationship data, for each pair of friend relationships, if the user Is a user 's friends, then in the picture Midpoint and Add an undirected social edge connecting the two. Add to connected edge set middle;

[0266] (b) For the interaction edge between a user and a video, we traverse all the historical sessions of all users. For a record of interaction between a user and a video, if the user With Video If there is interaction, then in the figure Midpoint and Add an undirected interaction edge between the two and Add to connected edge set middle;

[0267] (c) Construct a connection edge type mapping function based on the type of each edge ;

[0268] 4. Return result: Return the user-video heterogeneous directed graph .

[0269] The technical problem solved by the embodiments of the present application is the problem of social-aware video recommendation based on conversations. It is defined as follows:

[0270] Definition 7 (Session-based Socially Aware Video Recommendation Problem) Given a user set , video collection and each user Historical conversation sequence and social networks .in, Indicates user The task of the session-based social recommendation problem is to predict the user’s video viewing behavior history and social information based on the above video viewing behavior history and social information. In the new session, that is, the current session In the next time step Most likely to visit videos , and the video has been visited before, it satisfies . Common recommendation algorithms return a A recommendation list of videos, which are ordered from high to low according to the user's viewing probability. recommend.

[0271] The following is a detailed description of each invention part of the VRSET:

[0272] Heterogeneous Graph Convolutional Neural Networks:

[0273] Generally speaking, there are problems of information imbalance, redundancy and failure in video recommendation systems:

[0274] 1) There are usually more interactive relationships between users and videos than social relationships. Direct vertex aggregation may lead to information imbalance between social knowledge and historical interactions, resulting in weaker impact on social relationships.

[0275] 2) There are often some redundant or invalid relationships in the interaction between users and videos. The following are some situations and examples that may cause redundancy or invalidity in the interaction between users and videos: (a) Invalid or repeated click problems of some videos: Some videos may be considered invalid by users due to poor content quality, not to the user's taste, or other reasons. For example, some videos may be low-quality advertisements or very unpopular content. (b) Repeated clicks: Users may click on the same video multiple times, which may be because the user forgets that he has watched the video or makes a mistake. Repeated clicks may cause the recommendation system to over-recommend the same video, affecting the user experience.

[0276] 3) There may be some redundant or invalid relationships in the social relationship network. The following are some situations and examples that may lead to redundancy or invalidity in social relationships: (a) Invalid social connections: Some social connections may be invalid because they do not reflect real social relationships or interactions. For example, on some social platforms, users may form connections due to misoperation or meaningless attention, and such connections do not represent actual social relationships. (b) Redundant social interactions: Some social interactions may be redundant because they do not provide additional information or contribute value. For example, users frequently have similar interactions or repeated exchanges, but these interactions do not add new insights or information. (c) Outdated or invalid social relationships: Some social relationships may be outdated or invalid because they reflect past connections or interactions but no longer have actual meaning or value. For example, the interactions between users may decrease or stop over time, but these relationships still exist in the social network. (d) False or fictitious social relationships: In some cases, social relationships may be false or fictitious because they are formed based on false information or fake behavior. For example, some users may artificially increase the number of their social connections by purchasing followers or using bots, which are not real or valid. (e) Irrelevant or meaningless social interactions: Sometimes the interactions in social connections may be irrelevant or meaningless because they are not related to the user’s real interests or purposes. For example, some social interactions may be insignificant comments or likes without real communication or interaction.

[0277] From (1) to (3), we can see that both the user-video interaction network and the social relationship network have information imbalance, redundancy or invalidity problems. These problems may affect the modeling and understanding of user interests, item representations and social relationships, reduce recommendation diversity and accuracy, and thus affect user experience.

[0278] To solve the above problems, this embodiment draws on the graph neural network and submodule theory to design a heterogeneous graph convolutional neural network (HGCN) to effectively learn the relationship and feature representation between different types of vertices. The network contains The layer-by-layer heterogeneous graph convolutional neural network consists of two functional components: neighbor sampling and aggregation update. The former is based on the submodule theory and selects a set of basic neighbor subsets that are non-redundant, representative, and diverse from the neighbor set of each type of central vertex. The aggregation update component performs graph convolution operations on each type of central vertex by type, and then learns different weights for the graph convolution results of different types of central vertices based on the attention mechanism to alleviate the imbalance. To this end, we maintain a user-video heterogeneous graph It is used to update the embedding vector representation of users or videos. The update process requires information from the neighboring vertices of users and videos. and Video In the The learned hidden layer representations are and For users , its neighbor set Contains two types of vertices: user vertices and video vertices; for videos , its neighbor set Contains only one type of vertex: the user vertex that accesses it, and thus its neighbor set .

[0279] Neighbor sampling: Neighbor sampling refers to sampling from each type of central vertex The neighbor set of In the above example, we select a set of basic neighbor subsets that are non-redundant, representative, and diverse. , and satisfy the cardinality of the subset No more than When the neighbors of the central vertex have multiple types, it is required that there is at least one vertex of each type. In order to reduce the complexity of the network model and optimize the model application scenario, it is advisable to take the basic neighbor subset of the neighbor set of all vertices The upper limit of the cardinality is The above problem can be abstracted into an optimization problem: the optimization target is the basic neighbor subset The representativeness and diversity of the elements in; the constraint is the basic neighbor subset The base number does not exceed , and there is at least one vertex of each type.

[0280] A submodule function is a function defined on a collection, usually used to measure the "value" or "importance" of the collection. is called a submodule function if for any set and any element , the following conditions are met:

[0281] , (1)

[0282] Intuitively, this means that the marginal gain of adding elements to a smaller set decreases as the size of the set increases. Submodular functions have many useful properties in optimization problems, such as being able to approximate the optimal solution through greedy algorithms and being able to be used to maximize or minimize some objective function. Common submodular functions include mutual information functions.

[0283] Inspired by the submodule function, this embodiment models the neighbor sampling problem of the central vertex as a cardinality-constrained subset selection problem: given a user-video heterogeneous graph , from each central vertex The egocentric subgraph of Choose a base number not exceeding Subset of , and there is at least one vertex of each type, and the submodule function achieves the maximum value. This problem can be formally expressed as:

[0284] , (2)

[0285] (3)

[0286] Wherein, formulas (2) and (3) represent the objective function and constraint conditions of the optimization problem respectively; the central vertex can be either a user vertex or a video vertex; "Judge whether the two operand types are consistent and return a Boolean variable (true or false); Indicates forced type conversion, converting a Boolean variable to an integer variable.

[0287] Mutual Information Function: Mutual Information is an important indicator used in information theory to measure the correlation between two random variables. It can be used to describe the degree of mutual dependence or the amount of shared information between two random variables. Suppose there are two random variables and , whose joint probability distribution is , and the marginal probability distributions are and , mutual information It can be defined as:

[0288] (4)

[0289] in, and They are random variables and If and Independent of each other, , at this time the mutual information In other words, a mutual information of zero indicates that the two random variables are completely independent. and If there is a dependency relationship between The larger the mutual information value, the and The higher the degree of correlation between them, or the more information they share. The calculation of the mutual information function can be regarded as a measure of the degree of information sharing between two random variables, and can be applied in many fields, including machine learning, information retrieval, signal processing, etc. In machine learning, mutual information is often used in tasks such as feature selection, cluster analysis, and semi-supervised learning to help evaluate the correlation and information between features. For egocentric graphs , based on the mutual information principle, the submodule function is defined as:

[0290] , (5)

[0291] in, Is a collection About Vertex Sets The complement of , .

[0292] Submodule functions It can be the mutual information function defined by the above formulas (4)-(5), or any other function that can measure the "value" or "importance" of a set, which is not limited in this embodiment. The above cardinality-constrained maximization submodule function problem is an NP-hard problem, which can usually be solved using a greedy algorithm. This embodiment proposes a redundant information removal approximate solution algorithm REGSA (Redundancy Elimination algorithm based on Greedy Strategy with Approximation). Since the neighbor sampling problem solving process of different center vertex types is similar, it is better to use the center vertex as the center vertex. Taking the time as an example, the proposed approximate solution algorithm flow is as follows:

[0293] 1. Initialization: will be used to record the basic neighbor subset Initialized to an empty set; will be used to record the unselected vertex set (maybe called the candidate set) Initialize to ;

[0294] 2. Iterative greedy selection:

[0295] (a). For each vertex in the unselected vertex set , calculated according to the following formula and added The increment of the posterior objective function:

[0296] (6)

[0297] (b) Select the vertex that maximizes the increment and satisfies the constraints. , and change it from Delete and add ; If there are multiple vertices that meet the requirements, one of them is randomly selected.

[0298] (c). Update the basic neighbor subset and the candidate set: ; ;

[0299] (d). Check whether the constraints are satisfied; if so, exit the loop;

[0300] 3. Output result: get user The sampled base neighbor subset .

[0301] Similarly, we can also use the REGSA algorithm to solve the video The sampled base neighbor subset It should be pointed out that our framework is applicable to any submodule functions, such as mutual information function, vertex cover function, edge cover function and facility location function.

[0302] Through the redundant information removal approximate solution algorithm based on greedy strategy, we Redundant neighbors and noise in the social space and / or item space are removed to obtain a set of basic neighbor subsets that are non-redundant, representative, and diverse, laying the foundation for effectively extracting user embeddings and item embeddings enhanced with social relationships.

[0303] Aggregate Update: For the User-Video Heterogeneous Graph , whose vertex type set is , a set of connection edge types For each edge type , when the central vertex When assuming that the user and any of its neighbor vertices The The hidden layer representations of the output of the layer-heterogeneous graph convolutional neural network are Dimensional vector and , these neighbor vertices include both user vertices and video vertices. , No. The operations of the layer-heterogeneous graph convolutional neural network layer can be formally described as follows:

[0304] 1. Implement graph convolution operations for all neighbor vertices of type "user":

[0305] (7)

[0306] 2. For all neighbor vertices of type "video":

[0307] (8)

[0308] 3. Based on the attention mechanism, dynamically assign weights to the aggregation results of the two types of vertices and merge them:

[0309] (9)

[0310] (10)

[0311] in, Indicates user The set of all user vertices in the intersection of the neighbor set and the basic neighbor subset; Indicates user The set of all video vertices in the intersection of the neighbor set and the base neighbor subset; represents a nonlinear activation function, represents the power operation of natural exponents; All of them belong to the parameter matrix to be learned.

[0312] When the center vertex When, according to the user-video heterogeneous graph As defined, it has only one type of neighbor, the user vertex. and any of its neighbor vertices The The hidden layer representations of the output of the layer-heterogeneous graph convolutional neural network are Dimensional vector and .because The neighbor vertex of is only the user vertex, similar to formula (7), The operation of a layer of a heterogeneous graph convolutional neural network can be defined as:

[0313] , (11)

[0314] in, All belong to the parameter matrix to be learned.

[0315] when At that time, Layer-heterogeneous graph convolutional neural network layer based on user feature matrix and the video feature matrix is the input. When , the heterogeneous graph convolutional neural network outputs the user embedding vector representation enhanced by social semantics and video embedding vector representation .in, Denotes the dimension of the user and video embedding vector representation.

[0316] The above user-video heterogeneous graph embedding captures global knowledge such as friend relationships in the social space and all historical interaction records between users and videos at one time, greatly improving the accuracy and efficiency of modeling users' long-term interest preferences. In addition, we designed a vertex-type graph convolution operation for heterogeneous directed graphs, and dynamically allocated weights between different types of vertices based on the attention mechanism, effectively solving the information imbalance problem in the video recommendation system. However, the above user-video heterogeneous graph embedding does not capture the contextual information of a specific session, let alone complex dependencies such as incentives and inhibitions between sessions. However, for a personalized video recommendation system based on a session, user behavior in historical sessions is very important for capturing users' dynamic interests.

[0317] To this end, this embodiment designs a personalized session embedding component to extract user dynamic interest preferences from historical sessions, which is mainly composed of a temporal relationship learning network between sessions and a transfer relationship network between sessions. The following will introduce each network one by one in two subsections.

[0318] Inter-session temporal relationship learning network:

[0319] In order to extract the temporal relationship between sessions, this application designs a two-level recurrent neural network based on GRU. Figure 4 is an optional GRU cycle application network diagram according to an embodiment of the present application, such as Figure 4 As shown in Figure 1, the network consists of a session preprocessor, an intra-session recurrent neural network, an inter-session recurrent neural network, and a session representation database. Each session Remove redundant or invalid videos from the session to lengthen the session Convert to a fixed length Basic conversation ; The intra-session recurrent neural network is responsible for extracting session representations from the video sequences in each basic session of the user (multimedia resources browsed in each reference time period) in turn, and inputting them into the session representation database for storage. The session representation of the last session is the output of the entire two-level recurrent neural network; The inter-session recurrent neural network generates the initial hidden state of the intra-session recurrent neural network according to the vector representation sequence of all sessions in the session database every time a data write operation occurs in the session representation database, that is, every time the intra-session recurrent neural network generates a session representation and writes it into the session representation database, that is, the output of the inter-session recurrent neural network is used as the initial state of the intra-session recurrent neural network. The reason for this design is that although the intra-session recurrent neural network can achieve good performance, it does not know any information about the user at the beginning of each session, and learns the user's interests during the session, but all this information is discarded at the end of the session. The inter-session recurrent neural network can improve the intra-session recurrent neural network because it takes into account the user's previous sessions and provides information to the intra-session recurrent neural network at the beginning of each new session.

[0320] Session preprocessing program: This program can be implemented based on the neighbor sampling function of submodule theory. The upper limit of the cardinality of the basic session obtained by the agreed sampling is , then the session sampling problem based on submodule theory can be modeled as follows:

[0321] , (12)

[0322] , (13)

[0323] Among them, the submodule function It can be the mutual information function defined by the above formulas (4)-(5), or any other function that can measure the "value" or "importance" of a set, which is not limited in this embodiment. Similarly, the above problem can be solved by the REGSA algorithm mentioned above, which will not be described here. Each session in the historical session (including information about multimedia resources browsed in each reference time period in multiple reference time periods) The maximum length obtained by the session preprocessor is Basic conversation , and meet . Because the preprocessor may disrupt the underlying session The order in which the videos are accessed in Rearrange the order in and record the sorted basic sessions as .

[0324] For Users Each basic session ,make The input of the inter-session recurrent neural network is a sequence of all session representations currently stored in the session representation database. The initial hidden state of the RNN within the session Set as the final output of the inter-session recurrent neural network. is the dimension of the video embedding vector representation.

[0325] GRU-based intra-session recurrent neural network: for users Each basic session , with the output of the GRU-based inter-session recurrent neural network As initial state , the GRU updates its hidden state video by video:

[0326] (14)

[0327] (15)

[0328] (16)

[0329] (17)

[0330] And the output of the last step As the embedded vector representation of the underlying session , and added to the session to represent the storage in the database. and Respectively represent Reset the gate and update the gate state in time steps; and are the parameter matrix and vector to be learned respectively. When The first basic session Initial state The random initialization method is used to complete the initialization; when When, that is, for users The last basic session of , takes the output of its last time step as the output of the entire two-stage recurrent neural network, and is recorded as (E stands for external and S stands for sequential), which is the embedding vector representation of the temporal relationship between sessions.

[0331] GRU-based session recurrent neural network: Each time a session representation is added, it can be recorded as , all the sessions are represented as a sequence of all the sessions stored in the database As input, the network state is randomly initialized, and the output is calculated according to formulas (14)~(17) and used as the initial hidden state of the GRU-based intra-session recurrent neural network .

[0332] The above-mentioned two-level GRU-based recurrent neural network continuously extracts historical session representations from historical session representations with the help of the inter-session recurrent neural network and uses it as the initial state of the intra-session recurrent neural network, so as to capture the complex dependencies such as incentives or inhibition between historical sessions, thereby improving the accuracy of the representation of temporal relationships between sessions.

[0333] Inter-session transfer relation learning network:

[0334] Since the inter-session sequential learning network effectively extracts the temporal relationship between sessions, it fails to capture the high-level structural information between sessions. There are complex transfer relationships between adjacent or distant sessions, which are very important for modeling users' long-term interest preferences. To this end, this embodiment proposes an inter-session transfer relationship learning network. First, we need to design a weighted directed historical session graph construction algorithm to model all user sessions and the transfer relationships between them as a weighted directed historical session graph.

[0335] Construct a weighted directed historical conversation graph. All conversations And the transfer relationship between them is modeled as a weighted directed graph In this graph, each vertex Represents a session, the characteristics of each vertex can be directly obtained from the session representation database in the previous article; each directed edge Represents a session and There is a transfer relationship between them; Represents a directed edge The weight of , which describes the semantic distance or similarity between the conversation pairs, can be obtained by the Directed and Weighted IntEr-SessionGraph Construction Algorithm (DESGC) proposed in this embodiment. The following are the specific steps of the algorithm:

[0336] 1. Initialization. The weighted directed historical conversation graph of The vertex set Initialize to , each vertex With attributes ; Connect the edge set Initialize to an empty set; connect edge weight set Initialize to an empty set; initialize the connection edge weight threshold ;

[0337] 2. Iterate and calculate the similarity between all pairs of vertices. For any pair of vertices and , this embodiment uses cosine similarity as the similarity measure between them:

[0338] (18)

[0339] in, Of course, other similarity measurement methods, such as Gaussian distance, Manhattan distance, etc., can also be applied, which is not limited in this embodiment.

[0340] 3. Normalize the similarity of all vertex pairs. Use the following formula for any pair of vertices calculated in step 2: and Similarity between Normalize to get the normalized similarity :

[0341] , (19)

[0342] 4. Determine the set of connected edges and weights. Determine each pair of vertices and The normalized similarity between and edge weight threshold The size relationship is when the former is greater than the latter and Accessed before , then at the vertex and Add directed edges between ; when the former is greater than the latter and Access time is later than , then at the vertex and Add directed edges between ; In both cases, the weight of the connecting edge is the normalized similarity .

[0343] 5. Output weighted directed historical session graph construction, where vertices represent sessions and edges represent dependencies between sessions.

[0344] After constructing the weighted directed historical conversation graph, this embodiment designs a weighted and directed graph convolutional recurrent neural network with random walk (WDGCR) to construct a weighted directed historical conversation graph. The network consists of The first layer is a weighted directed graph random walk recurrent convolutional neural network layer and the second layer is a graph pooling layer. Each weighted directed graph random walk recurrent convolutional neural network layer includes two functional components: random walk and aggregation update. The first layer is based on the theory of random walk and generates a random walk for each vertex. The length of the generated strip is Random walk sequence. The aggregation update component extracts the representation of each random walk sequence based on the GRU recurrent neural network, and then learns different weights for each random walk sequence representation based on the attention mechanism, and takes the weighted sum of these random walk sequence representations as the hidden layer representation of the vertex. The graph pooling layer is used to fuse all vertex features in the graph into an embedded vector representation as the user's dynamic interest preference. To this end, we Maintain a weighted directed historical conversation graph , used to update the embedding vector representation of the video, and the update process requires information from the user and the video neighbor vertices. The following will first introduce the weighted directed graph random walk algorithm.

[0345] Weighted directed graph random walk algorithm:

[0346] 1. Initialization. For each vertex As the starting vertex, initialize the current vertex for .

[0347] 2. Random walk sampling. For each vertex , when the time step When executing:

[0348] Calculate from the current vertex To all outgoing neighbor vertices The transition probability.

[0349] Randomly select the next vertex based on the transition probability .

[0350] ·The vertices Join the current random walk sequence.

[0351] ·The vertices Set as Current Vertex .

[0352] 3. Repeat the above steps until each vertex has The length of the generated strip is Random walk sequence.

[0353] In this algorithm, the calculation of transition probability can be achieved by normalizing the edge weights from each vertex to its outgoing neighbor vertices. Assume that the current vertex The set of outgoing neighbor vertices of , the transition probability It can be calculated as:

[0354] (20)

[0355] in, From the vertex To the top Through this algorithm, we can perform random walk sampling in a weighted directed graph and generate the edge weights of each vertex. A random walk sequence.

[0356] Aggregate update component. Let's assume that there is a weighted directed historical conversation graph Each vertex Generated by a weighted directed graph random walk algorithm The walking sequence is ,in , set the vertex In the The learned hidden layer representations are According to formulas (14) to (17), GRU is used to extract Extract the sequence representation from ; Then, based on the attention mechanism, a dynamic weight is assigned to each sequence representation , and take the weighted sum of these sequence representations as the vertex In the The output of the layer of the weighted directed graph random walk recurrent convolutional neural network :

[0357] (twenty one)

[0358] (twenty two)

[0359] Graph pooling layer. The graph pooling layer is a method for fusing all vertex features in a graph into an embedded vector representation. This method aims to capture the global features of the graph and can be used for tasks such as graph classification, graph-level prediction, and graph generation. In order to extract the representation of the inter-session transfer relationship from the weighted directed historical conversation graph, this embodiment proposes three graph pooling methods: global average pooling, global maximum pooling, and attention pooling. , let’s remember the arbitrary session output by the last layer of weighted directed graph random walk recurrent convolutional neural network layer The hidden layer is represented as , record the weighted directed historical conversation graph The embedding vector is represented as (E stands for external, T stands for transitional), it can be calculated by the following pooling method:

[0360] 1. Global Average Pooling (GAP): Global Average Pooling is a simple and effective graph-level pooling method that averages the hidden layer representations of all vertices to obtain the representation of the entire weighted directed history session graph. :

[0361] (twenty three)

[0362] Similarly, we can get the global maximum pooling (Global Max Pooling, GMP):

[0363] (twenty four)

[0364] in, It means to take the maximum value of all input vectors by component and return a new vector.

[0365] 2. Attention Pooling: Attention pooling is a pooling method based on the attention mechanism. It dynamically aggregates vertex features by learning attention weights. The steps of this method are as follows:

[0366] (25)

[0367] (26)

[0368] (27)

[0369] in, is the parameter matrix to be learned, is the activation function (such as ReLU), It is a conversation The score, It is a conversation The attention weight.

[0370] The design of the weighted directed graph random walk recurrent convolutional neural network can more effectively capture the complex transfer relationship and information transmission between adjacent and distant sessions in the directed graph, greatly improving the expressiveness and applicability of the model. The graph pooling layer fuses all vertex features in the weighted directed graph into graph-level features, comprehensively and accurately describing the complex transfer relationship between videos in a session.

[0371] User interest preference fusion component:

[0372] The user embedding output by the heterogeneous graph embedding component incorporates knowledge such as the influence of friends in the social space and all historical conversations between users and videos, and can be regarded as the user's long-term interest preference. Many previous studies have shown that user long-term interest preferences are useful for predicting the next video that users will watch. To this end, we designed a user interest preference fusion component to fuse the user's long-term interest preference output by the heterogeneous directed graph embedding component with the embedding vector representation of the temporal relationship between sessions and the embedding vector representation of the transfer relationship between sessions output by the personalized session embedding component to obtain a comprehensive user interest preference representation. :

[0373] (28)

[0374] in, are the network parameters to be learned; Represents a vector concatenation operation.

[0375] The above-mentioned user interest preference fusion component integrates user long-term interest preferences and personalized session embedding to obtain the user's comprehensive interest preferences that fully consider the influence of friends in the social space and the temporal relationship and transfer relationship between sessions contained in historical sessions, which more accurately depicts the user's behavior characteristics.

[0376] Video recommendation:

[0377] To generate the probability distribution of the next time step video, for each video with embedding , calculate the similarity score between its embedding vector representation and the user's final interest preference representation:

[0378] (29)

[0379] The score vector for all videos Then, the score vector is normalized by softmax to obtain the probability distribution:

[0380] , (30)

[0381] Among them, the probability distribution Characterizes the user The probability of watching each video in the next time step of the current session. Sort the probability distribution from high to low and return the previous The high-probability videos are selected as candidate sets for the next time step video. To avoid repeated recommendations, the returned videos can be constrained to be videos that the user has not watched.

[0382] Model training method:

[0383] set up Represents the true probability distribution of the next video, which is a vector using one-hot encoding. Loss function Defined as predicted value With the true value The cross entropy between:

[0384] (31)

[0385] Then, according to specific strategies such as Stochastic Gradient Descent (SGD), Momentum Gradient Descent (MGD), Nesterov Momentum, AdaGrad, RMSprop and Adam (Adaptive Moment Estimation) or Batch Gradient Descent (BGD), the parameters of the proposed VRSTT network are modified and updated to optimize the loss function value until the loss function reaches the minimum value or the number of training times reaches the specified maximum value, the training ends.

[0386] General purpose heterogeneous accelerator design based on FPGA:

[0387] This application FPGA designs a general heterogeneous accelerator for heterogeneous graph convolutional neural networks, two-level recurrent neural networks based on GRU, and weighted directed graph random walk recurrent convolutional neural networks. In order to maximize the reuse of FPGA computing and storage resources, this application uniformly maps the graph convolution of the heterogeneous graph attention neural network, the reset operation and update operation in the two-level recurrent neural network based on GRU, and the encoding, decoding and pooling operations in the weighted directed graph autoencoder, so that multiple types of operations can be completed using the same computing array. In terms of timing structure design, reconfigurable bit-serial matrix multiplication overlay (BISMO) is used. In terms of spatial structure, a three-dimensional pulsating array structure is adopted.

[0388] Figure 5 is a schematic diagram of a general heterogeneous accelerator design of an optional FPGA according to an embodiment of the present application, such as Figure 5 As shown in FIG. 1 , the FPGA-based general heterogeneous accelerator proposed in this embodiment mainly consists of a scheduling module, a general computing module, a nonlinear and normalization module, a pooling module, a data operation module, an off-chip memory, an input cache, and an output cache. The interconnection relationship between different modules is shown in FIG. Figure 5 As shown. Among them, the off-chip memory is used to store the instructions and data sent by the CPU to the heterogeneous accelerator via the PCIE bus; the scheduling module decodes the received instructions and distributes the decoded instructions to the data operation module, the nonlinear and normalization module, the general computing module or the pooling module; the general computing module is responsible for executing the graph convolution of the heterogeneous graph attention neural network, the reset operation and update operation in the two-level recurrent neural network based on GRU, and the encoding and decoding in the weighted directed graph autoencoder; the pooling module is responsible for executing the pooling operation in the weighted directed graph autoencoder; its results are finally output to the off-chip storage via the output cache; the data operation module is used to receive the scheduling core control signal, state feedback, data handling and process state control.

[0389] The main workflow of the proposed general heterogeneous accelerator is as follows: the host computer first loads the instruction set, weight data, feature data and / or graph topology data, writes them into the off-chip memory of the heterogeneous accelerator, and then starts running. After receiving the start command, the scheduling core pre-fetches some instructions from the off-chip memory to the instruction cache, then starts to read and decode the instructions from it, and dispatches the instructions to the general computing module, nonlinear and normalization module, pooling module, data operation module, etc. for execution according to the different types of instructions, until the program end (END) instruction issued by the host is encountered, and the operation ends.

[0390] The target device of the general heterogeneous accelerator designed in the embodiment of the present application is F26A FPGA, and the accelerator is integrated and implemented using Vivadov2021.1, and the system operating frequency is 200 MHz.

[0391] Due to the low power consumption, low latency, and customizability of FPGA, this embodiment customizes a general acceleration architecture by analyzing the core operations of heterogeneous graph convolutional neural networks, GRU-based two-level recurrent neural networks, and weighted directed graph random walk recurrent convolutional neural networks. This can greatly improve the training and reasoning efficiency of the proposed video recommendation system, significantly save energy consumption, and reduce costs.

[0392] Compared with the existing video recommendation methods, the video recommendation system based on social, temporal and transition relationship perception of historical conversations proposed in this embodiment has the following beneficial effects: (1) It comprehensively considers the influence of social relations in social space and temporal relations and transition relations hidden in historical interactive conversations between users and videos on user interest preferences, more accurately describes user interest preferences, and greatly improves the accuracy of video recommendation. (2) With the help of the redundant information removal approximate solution algorithm based on the greedy strategy, the information redundancy and noise in the heterogeneous graph and user historical conversation are removed, while improving the accuracy of user and video embedding, the diversity of video recommendation is increased. (3) The two-level recurrent neural network based on GRU continuously extracts the historical conversation representation from the historical conversation representation with the help of the inter-conversation recurrent neural network and uses it as the initial state of the intra-conversation recurrent neural network, so as to capture the complex dependency relationships such as incentives or inhibition between historical conversations, and improve the accuracy of the temporal relationship representation between conversations. (4) The design of the weighted directed graph random walk recurrent convolutional neural network can more effectively capture the complex transition relationship and information transmission between adjacent conversations and long-distance conversations in the directed graph, greatly improving the expression ability and applicability of the model. (5) The proposed method only uses the user's historical conversation to generate social semantics enhanced user embedding once in the heterogeneous graph embedding process, avoiding repeated learning of social friends' historical conversations to integrate social influence when recommending to users, greatly reducing the computational complexity of the network model. (6) With the help of low-latency, low-power FPGA, we designed a general heterogeneous accelerator for heterogeneous graph attention neural networks, time-aware capsule networks, and weighted directed graph autoencoders, which can significantly improve the training and inference efficiency of heterogeneous graph convolutional neural networks, GRU-based two-level recurrent neural networks, and weighted directed graph random walk recurrent convolutional neural networks, and can significantly save energy consumption and reduce costs.

[0393] Figure 6 is a schematic diagram of an optional recommended device hardware according to an embodiment of the present application, such as Figure 6 As shown, the host computer in the figure is responsible for running Figure 3The user directed graph construction component in the figure is used to obtain a user-video heterogeneous directed graph. The heterogeneous accelerator in the figure is responsible for running the heterogeneous directed graph convolutional neural network, the personalized session embedding component, the user interest preference fusion component and the prediction component. The implementation method can be: loading the video recommendation network model and the video recommendation data set from the host's disk to the main memory; further moving the video recommendation network model in the main memory to the hardware accelerator memory; the host preprocesses the video recommendation data set, constructs a user-video heterogeneous directed graph from the user's social network and browsing history, and divides the user browsing data into a training set, a test set and a validation set according to the time sequence and 0.6:0.2:0.2; the host sends a video recommendation network model training instruction to the hardware accelerator, and the hardware accelerator loads the training set and validation set data from the main memory in small batches to complete the training of the video recommendation network; the hardware accelerator continues to read the test set data from the host main memory, completes the performance test of the video recommendation network model, and returns the test results to the host; after receiving the recommendation results from the hardware accelerator, the host displays them to the user in a certain form.

[0394] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0395] In the present embodiment, a display device for multimedia resources is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0396] Figure 7 is a structural block diagram of a multimedia resource display device according to an embodiment of the present application, such as Figure 7 As shown, the device comprises:

[0397] A first prediction module is configured to predict a temporal relationship feature of the target account based on first browsing information of the target account, and to predict a transfer relationship feature of the target account based on the first browsing information, wherein the first browsing information is used to indicate the multimedia resource browsing situation of the target account in multiple reference time periods before the current moment, the temporal relationship feature is used to indicate the temporal changes in the browsing preference of the target account for multimedia resources, and the transfer relationship feature is used to indicate the current browsing preference of the target account under the influence of the multimedia resources browsed in each of the reference time periods; a generation module is configured to generate a target account feature of the target account based on the temporal relationship feature, the transfer relationship feature and the resource browsing feature of the target account, wherein the resource browsing feature is used to indicate the multimedia resource browsing preference of the target account under the influence of the multimedia resource browsing information of a reference account associated with the target account, and the target account feature is used to indicate the multimedia resource browsing preference of the target account at the current moment; a display module is configured to display target multimedia resources matching the target account feature to the target account.

[0398] Through the above device, by predicting the temporal relationship feature for indicating the temporal change of the browsing preference of the target account for multimedia resources according to the first browsing information of the target account, and predicting the transfer relationship feature for indicating the multimedia resources browsed in each reference time period that affect the browsing preference of the target account, and then generating the target account feature of the target account according to the temporal relationship feature, the transfer relationship feature and the resource browsing feature of the target account, since the resource browsing feature of the target account is used to indicate the multimedia resource browsing preference of the target account under the influence of the multimedia resource browsing information of the reference account associated with the target account, the temporal relationship feature, the transfer relationship feature and the resource browsing feature under the influence of the social relationship of the target account are integrated into the generated target account feature, so that the target account feature is closer to the user's current real preference for multimedia resources, and then the target multimedia resources matching the target account feature displayed to the target account can better meet the target account's current demand for multimedia resources. The problem of low matching between the multimedia resources recommended for the target account and the target account's current resource preference for multimedia resources can be solved, so as to achieve the effect of improving the matching between the multimedia resources recommended for the target account and the target account's current resource preference for multimedia resources.

[0399] Optionally, the first prediction module includes: a first screening unit, used to screen out the target time period to be processed from the multiple reference time periods according to the time sequence of the multiple reference time periods; a first acquisition unit, used to acquire a first browsing feature of the first sub-browsing information of the target account in the target time period, wherein the first browsing feature is used to indicate the browsing preference of the target account in the target time period; a first prediction unit, used to predict a second temporal relationship feature in the target time period based on the first temporal relationship feature in the candidate time period and the first browsing feature, wherein the candidate time period is adjacent to the target time period and is located before the target time period in the multiple reference time periods, and the first temporal relationship feature is used to indicate the influence of the browsing preference of the target account in the time period before the candidate time period on the browsing preference of the target account in the candidate time period; a determination unit, used to determine the second temporal relationship feature as the temporal relationship feature when the target time period is the last one of the multiple reference time periods.

[0400] Optionally, the first acquisition unit is used to: filter out the second multimedia resource to be processed from multiple first multimedia resources in a browsing order, wherein the first sub-browsing information includes multiple first multimedia resources browsed by the target account in the browsing order within the target time period; predict the second preference feature of the target account after browsing the second multimedia resource based on the first preference feature of the target account and the initial resource feature of the second multimedia resource, wherein the first preference feature is used to indicate the browsing preference of the target account after browsing the second multimedia resource under the influence of the multimedia resource browsed before the second multimedia resource, and the initial resource feature is used to indicate the account feature of the account browsing the second multimedia resource; when the second multimedia resource is the last multimedia resource browsed by the target account within the target time period, the second preference feature is determined as the first browsing feature.

[0401] Optionally, the first acquisition unit is used to: use a first reset gate to extract first adjustment information from the first preference feature, wherein the first adjustment information is used to indicate the degree of influence of the first preference feature on the browsing preference of the target account; call the first update gate to update the initial resource feature according to the first adjustment information to obtain the second preference feature.

[0402] Optionally, the device also includes: a detection module, used to predict the second preference feature of the target account after browsing the second multimedia resource and before the second preference feature of the target account, and detect in the first sub-browsing information a third multimedia resource browsed by the target account before the second multimedia resource; a first determination module, used to determine the browsing preference feature of the target account after browsing the third multimedia resource as the first preference feature if the third multimedia resource exists in the first sub-browsing information; and a second determination module, used to determine the first temporal relationship feature as the first preference feature if the third multimedia resource does not exist in the first sub-browsing information.

[0403] Optionally, the first acquisition unit is used to: calculate reference resource parameters of each of the first multimedia resources, wherein the reference resource parameters are used to indicate the degree of dependence between the current multimedia resource and other multimedia resources among the multiple first multimedia resources except the current multimedia resource; filter out a fourth multimedia resource whose reference resource parameter is greater than or equal to the target resource parameter from the multiple first multimedia resources; and filter out the second multimedia resource to be processed currently from the multiple fourth multimedia resources in browsing order.

[0404] Optionally, the first prediction unit is used to: use a second reset gate to extract second adjustment information from the first temporal relationship feature, wherein the second adjustment information is used to indicate the degree of influence of the first temporal relationship feature on the browsing preference of the target account; call the second update gate to update the first browsing feature according to the second adjustment information to obtain the second temporal relationship feature.

[0405] Optionally, the first prediction module includes: a second acquisition unit, used to acquire the second browsing feature of the second sub-browsing information within each of the reference time periods, wherein the second browsing feature is used to indicate the browsing preference of the target account within the reference time period; a first calculation unit, used to calculate the browsing similarity between any two of the second sub-browsing information in a plurality of the second sub-browsing information using the second browsing feature; a first generation unit, used to generate a random walk sequence for a target sub-browsing information in a plurality of the second sub-browsing information according to the browsing similarity, wherein the random walk sequence takes the target sub-browsing information as a starting point, and the random walk sequence records the association relationship between a target number of the second sub-browsing information in a plurality of the second sub-browsing information, wherein the target sub-browsing information is any sub-browsing information included in the plurality of the second sub-browsing information; a second prediction unit, used to predict the initial transfer relationship feature of the target sub-browsing information corresponding to the random walk sequence according to the random walk sequence, wherein the initial transfer relationship feature is used to indicate the browsing preference of the target account under the influence of the corresponding target sub-browsing information; a second generation unit, used to generate the transfer relationship feature according to the initial transfer relationship feature of the plurality of the second sub-browsing information.

[0406] Optionally, the first generating unit is used to: construct a first relationship graph of multiple second sub-browsing information according to the browsing similarity, wherein the first relationship graph includes multiple first vertices and a first directed edge between any two first vertices, each first vertex corresponds to one second sub-browsing information, and the first directed edge records the association relationship between the second sub-browsing information; extract a first vertex sequence with the second vertex as the starting point for each second vertex from the first relationship graph, wherein the second vertex is any vertex among the multiple first vertices, and the first vertex sequence records the association relationship between the target number of first vertices among the multiple first vertices; sort the second sub-browsing information corresponding to the first vertices in the first vertex sequence according to the association relationship between the first vertices recorded in the first vertex sequence to obtain the random walk sequence.

[0407] Optionally, the first generation unit is used to: in the process of extracting the first vertex sequence, search from the first relationship graph for a fourth vertex that has an association relationship with the third vertex, wherein the third vertex is the vertex for searching for the next associated vertex, and the output end of the first directed edge connecting the fourth vertex and the third vertex points to the fourth vertex; assign a screening weight to each of the fourth vertices according to the vertex association degree indicated by the first directed edge; randomly screen out a fifth vertex that is located after the third vertex in the vertex sequence from multiple fourth vertices according to the screening weight; detect a reference number of vertices included in the currently screened vertex sequence; and when the reference number is equal to the target number, determine the currently screened vertex sequence as the first vertex sequence.

[0408] Optionally, the first generation unit is used to: create the first vertex corresponding to each of the second sub-browsing information, and configure the vertex association between the first vertices corresponding to the second sub-browsing information according to the browsing similarity between the second sub-browsing information; create a first connecting edge between the first vertices whose vertex association is greater than or equal to the target association; configure the pointing relationship of the first connecting edge according to the timing relationship of the occurrence of the second sub-browsing information corresponding to the first vertex, to obtain the first directed edge, wherein the incident end of the first directed edge is connected to the vertex with an earlier occurrence timing, and the output end of the first directed edge is connected to the vertex with a later occurrence timing; determine the relationship graph of the first directed edges that records the connections between multiple first vertices and the first vertices as the first relationship graph.

[0409] Optionally, the second prediction unit is used to: calculate the target time series feature of each random walk sequence in the multiple random walk sequences of each second sub-browsing information, wherein the time series feature is used to indicate that the browsing preference of the target account in the random walk sequence is a time series change; assign a sequence weight to each random walk sequence according to the proportion of the target time series feature of each random walk sequence in the time series features and values ​​of the multiple random walk sequences; and perform weighted summation of the multiple time series features according to the sequence weight to obtain the initial transfer relationship feature.

[0410] Optionally, the second prediction unit is used to: determine the sixth sub-browsing information to be processed from the random walk sequence according to the occurrence sequence of the sub-browsing information; predict the fourth preference feature of the target account after browsing the sixth sub-browsing information based on the third preference feature of the target account and the second browsing feature corresponding to the sixth sub-browsing information, wherein the third preference feature is used to indicate the browsing preference of the target account before browsing the sixth sub-browsing information in the random walk sequence; when the sixth sub-browsing information is the last sub-browsing information in the random walk sequence, determine the fourth preference feature as the target timing feature of the random walk sequence.

[0411] Optionally, the second prediction unit is used to: use a third reset gate to extract second adjustment information from the third preference feature, wherein the second adjustment information is used to indicate the degree of influence of the third preference feature on the browsing preference of the target account; call the third update gate to update the second browsing feature according to the second adjustment information to obtain the fourth preference feature.

[0412] Optionally, the second prediction unit is used to perform one of the following operations: determining the mean of multiple initial transfer relationship features as the transfer relationship feature; determining the maximum feature value among multiple initial transfer relationship features as the transfer relationship feature; assigning a corresponding feature weight to each of the initial transfer relationship features according to the feature relationship between the initial transfer relationship features; and using the feature weights to perform weighted sum calculation on the initial transfer relationship features to obtain the transfer relationship feature.

[0413] Optionally, the device also includes: an acquisition module, used to obtain second browsing information of the reference account before generating the target account characteristics of the target account based on the timing relationship characteristics, the transfer relationship characteristics and the resource browsing characteristics of the target account, wherein the second browsing information is used to indicate the multimedia resource browsing status of the reference account within multiple reference time periods; a second prediction module, used to predict the resource browsing characteristics of the target account based on the first browsing information and the second browsing information.

[0414] Optionally, the second prediction module includes: a construction unit, which is used to construct a second relationship graph according to the first browsing information and the second browsing information, wherein the second relationship graph includes multiple sixth vertices, multiple seventh vertices, a first edge and a second edge, wherein the sixth vertex corresponds to the target account and the reference account one-to-one, and the seventh vertex corresponds to the multimedia resources included in the first browsing information and the multimedia resources included in the second browsing information one-to-one, the first edge is used to indicate the association relationship between the accounts, and the second edge is used to indicate the access relationship of the account to the multimedia resources; an extraction unit, which is used to extract the target vertex feature of the target vertex in the second relationship graph using a feature extraction model including multiple feature extraction layers connected in sequence, wherein the target vertex is any vertex in the second relationship graph, and each of the feature extraction layers is used to generate the vertex feature of the target vertex according to the initial vertex feature of the reference vertex in the current second relationship graph, and use the vertex feature of the target vertex output by the current feature extraction layer to update the vertex feature of the target vertex in the second relationship graph, and the reference vertex is a vertex in the second relationship graph that has an association relationship with the target vertex; a determination unit, which is used to determine the target vertex feature of the vertex corresponding to the target account as the resource browsing feature.

[0415] Optionally, the extraction unit is used to: call a target feature extraction layer to extract a target vertex set of the target vertex from the second relationship graph, wherein the target vertex set records the vertices in the second relationship graph that are connected to the target vertex through the first edge or the second edge, and the target feature extraction layer is the extraction layer that currently performs feature extraction on the target vertices in the second relationship graph; call the target feature extraction layer to perform feature aggregation on the initial vertex features currently possessed by the vertices included in the target vertex set to obtain reference vertex features of the target vertex output by the target feature extraction layer; and when the target feature extraction layer is the last layer of a plurality of feature extraction layers connected in sequence, determine the reference vertex features as the target vertex features.

[0416] Optionally, the extraction unit is used to: detect the vertex type of the target vertex; when the vertex type indicates that the target vertex is a vertex corresponding to an account, perform feature aggregation on the initial vertex features of the sixth vertex included in the target vertex set to obtain a first aggregate feature, and perform feature aggregation on the initial vertex features of the seventh vertex in the target vertex set to obtain a second aggregate feature; assign corresponding aggregate feature weights to the first aggregate feature and the second aggregate feature according to a feature relationship between the first aggregate feature and the second aggregate feature; use the aggregate feature weights to perform weighted sum calculation on the first aggregate feature and the second aggregate feature to obtain the reference vertex feature; when the vertex type indicates that the target vertex is a vertex corresponding to a multimedia resource, perform aggregation on the initial vertex features of the vertices included in the target vertex set to obtain the reference vertex feature.

[0417] Optionally, the extraction unit is used to: extract an initial vertex set of the target vertex from the second relationship graph, wherein the initial vertex set includes vertices in the second relationship graph that are connected to the target vertex through the first edge or the second edge; filter out a candidate number of candidate vertices from the initial vertex set according to vertex features currently possessed by the vertices in the initial vertex set to obtain the target vertex set, wherein, in a case where the target vertex is a vertex corresponding to an account, the candidate vertices include vertices corresponding to the account and vertices corresponding to multimedia resources; in a case where the target vertex is a vertex corresponding to a multimedia resource, the candidate vertices include vertices corresponding to the account, and the dependency relationship between the vertices in the target vertex set and the target vertex satisfies a target dependency condition.

[0418] Optionally, the extraction unit is used to: use the vertex features of each vertex included in the first vertex set of the current screening round to calculate the incremental value of each vertex to the vertex set information, wherein the vertex set information is used to indicate the importance of the vertex features of all vertices in the second vertex set, and the incremental value is used to indicate the amount of change of the vertex set information of the second vertex set after the corresponding vertex in the first vertex set is added to the second vertex set, and the initial vertex set includes vertices in the first vertex set and vertices in the second vertex set;

[0419] Extracting vertices whose increment values ​​are greater than or equal to the target increment value from the first vertex set and adding them to the second vertex set;

[0420] When the number of candidate vertices included in the second vertex set to which vertices are added is the candidate number, and the candidate vertices include both vertices corresponding to accounts and vertices corresponding to multimedia resources, the second vertex set is determined as the target vertex set.

[0421] Optionally, the generation module includes: a splicing unit, used to perform feature splicing on the temporal relationship feature, the transfer relationship feature and the resource browsing feature to obtain a spliced ​​feature; and a fusion module, used to perform feature fusion on the spliced ​​feature using a feature fusion model to obtain the target account feature output by the feature fusion model, wherein the feature fusion model records the conversion relationship between the splicing feature and the account feature.

[0422] Optionally, the display module includes: a second calculation unit, used to calculate the target similarity between the target account characteristics and the target resource characteristics of each multimedia resource in the multimedia resource library; a second screening unit, used to screen out the target multimedia resources whose target similarity is greater than or equal to a preset similarity from the multimedia resource library; and a display unit, used to display the target multimedia resources to the target account.

[0423] Optionally, the second computing unit is used to: use a feature generation model to generate a first resource feature of the multimedia resource in the first browsing information and a second resource feature of the multimedia resource in the multimedia resource library; detect feature difference information between the first resource feature and the second resource feature; use a feature adjustment amount corresponding to the feature difference information to perform feature adjustment on the initial resource feature of the multimedia resource in the first browsing information to obtain the target resource feature, wherein the initial resource feature is used to indicate the account feature of the account that browses the multimedia resource in the first browsing information; and calculate the target similarity between the target account feature and the target resource feature.

[0424] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0425] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0426] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0427] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0428] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0429] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0430] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0431] The embodiments of the present application also provide a computer program, which includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any one of the above method embodiments.

[0432] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0433] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0434] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for displaying multimedia resources, characterized in that: include: Predicting a temporal relationship feature of the target account based on first browsing information of the target account, and predicting a transfer relationship feature of the target account based on the first browsing information, wherein the first browsing information is used to indicate the multimedia resource browsing situation of the target account in multiple reference time periods before the current moment, and the temporal relationship feature is used to indicate the temporal change of the browsing preference of the target account for multimedia resources, and the temporal relationship feature is obtained by learning the temporal relationship between the resource browsing features of the target account in multiple reference time periods in the first browsing information, and the transfer relationship feature is used to indicate the current browsing preference of the target account under the influence of the multimedia resources browsed in each reference time period; generating a target account feature of the target account according to the temporal relationship feature, the transfer relationship feature and the resource browsing feature of the target account, wherein the resource browsing feature is used to indicate the multimedia resource browsing preference of the target account under the influence of the multimedia resource browsing information of the reference account associated with the target account, and the target account feature is used to indicate the multimedia resource browsing preference of the target account at the current moment; The target multimedia resources matching the characteristics of the target account are displayed to the target account.

2. The method according to claim 1, characterized in that The predicting the temporal relationship feature of the target account according to the first browsing information of the target account includes: Filtering out a target time period to be processed currently from the multiple reference time periods according to the time sequence of the multiple reference time periods; Acquire a first browsing feature of the first sub-browsing information of the target account within the target time period, wherein the first browsing feature is used to indicate the browsing preference of the target account within the target time period; Predicting a second temporal relationship feature within the target time period based on a first temporal relationship feature within a candidate time period and the first browsing feature, wherein the candidate time period is adjacent to the target time period and is located before the target time period in a plurality of reference time periods, and the first temporal relationship feature is used to indicate the influence of the browsing preference of the target account in the time period before the candidate time period on the browsing preference of the target account in the candidate time period; In a case where the target time period is the last one of the multiple reference time periods, the second temporal relationship feature is determined as the temporal relationship feature.

3. The method according to claim 2, characterized in that The step of obtaining the first browsing feature of the first sub-browsing information of the target account within the target time period includes: Filtering out a second multimedia resource to be processed currently from a plurality of first multimedia resources in a browsing order, wherein the first sub-browsing information includes a plurality of the first multimedia resources browsed by the target account in the browsing order within the target time period; Predicting a second preference feature of the target account after browsing the second multimedia resource based on the first preference feature of the target account and the initial resource feature of the second multimedia resource, wherein the first preference feature is used to indicate the browsing preference of the target account after browsing the second multimedia resource under the influence of the multimedia resource browsed before the second multimedia resource, and the initial resource feature is used to indicate the account feature of the account browsing the second multimedia resource; In a case where the second multimedia resource is the last multimedia resource browsed by the target account within the target time period, the second preference feature is determined as the first browsing feature.

4. The method according to claim 3, characterized in that The predicting, based on the first preference feature of the target account and the initial resource feature of the second multimedia resource, the second preference feature of the target account after browsing the second multimedia resource includes: extracting first adjustment information from the first preference feature using a first reset gate, wherein the first adjustment information is used to indicate the degree of influence of the first preference feature on the browsing preference of the target account; The first update gate is called to update the initial resource feature according to the first adjustment information to obtain the second preference feature.

5. The method according to claim 3, characterized in that: Before predicting the second preference feature of the target account after browsing the second multimedia resource based on the first preference feature of the target account and the initial resource feature of the second multimedia resource, the method further includes: Detecting, in the first sub-browsing information, a third multimedia resource browsed by the target account before the second multimedia resource; In a case where the third multimedia resource exists in the first sub-browsing information, determining the browsing preference feature of the target account after browsing the third multimedia resource as the first preference feature; In a case where the third multimedia resource does not exist in the first sub-browsing information, the first temporal relationship feature is determined as the first preference feature.

6. The method according to claim 3, characterized in that The step of selecting the second multimedia resource to be processed from the plurality of first multimedia resources in a browsing order includes: Calculating a reference resource parameter of each of the first multimedia resources, wherein the reference resource parameter is used to indicate a degree of dependency between the current multimedia resource and other multimedia resources among the plurality of the first multimedia resources except the current multimedia resource; Filter out a fourth multimedia resource whose reference resource parameter is greater than or equal to a target resource parameter from the plurality of first multimedia resources; The second multimedia resource to be processed is selected from the plurality of the fourth multimedia resources in a browsing order.

7. The method according to claim 2, characterized in that The predicting the second temporal relationship feature within the target time period according to the first temporal relationship feature within the candidate time period and the first browsing feature includes: extracting second adjustment information from the first temporal relationship feature using a second reset gate, wherein the second adjustment information is used to indicate the degree of influence of the first temporal relationship feature on the browsing preference of the target account; The second update gate is called to update the first browsing feature according to the second adjustment information to obtain the second temporal relationship feature.

8. The method according to claim 1, characterized in that The predicting the transfer relationship feature of the target account according to the first browsing information includes: Acquire a second browsing feature of the second sub-browsing information within each of the reference time periods, wherein the second browsing feature is used to indicate a browsing preference of the target account within the reference time period; Calculate the browsing similarity between any two of the second sub-browsing information in the plurality of the second sub-browsing information by using the second browsing feature; Generating a random walk sequence for target sub-browsing information in the plurality of second sub-browsing information according to the browsing similarity, wherein the random walk sequence takes the target sub-browsing information as a starting point, and the random walk sequence records the association relationship between the target number of second sub-browsing information in the plurality of second sub-browsing information, and the target sub-browsing information is any sub-browsing information included in the plurality of second sub-browsing information; Predicting, according to the random walk sequence, an initial transfer relationship feature of the target sub-browsing information corresponding to the random walk sequence, wherein the initial transfer relationship feature is used to indicate the browsing preference of the target account under the influence of the corresponding target sub-browsing information; The transfer relationship feature is generated according to the initial transfer relationship features of a plurality of the second sub-browsing information.

9. The method according to claim 8, characterized in that The step of generating a random walk sequence for target sub-browsing information in a plurality of second sub-browsing information according to the browsing similarity comprises: Constructing a first relationship graph of a plurality of the second sub-browsing information according to the browsing similarity, wherein the first relationship graph includes a plurality of first vertices and a first directed edge between any two first vertices, each of the first vertices corresponds to one second sub-browsing information, and the first directed edge records the association relationship between the second sub-browsing information; Extracting a first vertex sequence starting from the second vertex for each second vertex from the first relationship graph, wherein the second vertex is any vertex among the plurality of first vertices, and the first vertex sequence records association relationships between the target number of first vertices among the plurality of first vertices; The second sub-browsing information corresponding to the first vertices in the first vertex sequence is sorted according to the association relationship between the first vertices recorded in the first vertex sequence to obtain the random walk sequence.

10. The method according to claim 9, characterized in that The extracting, from the first relationship graph, for each second vertex, a first vertex sequence starting from the second vertex comprises: In the process of extracting the first vertex sequence, searching the first relationship graph for a fourth vertex that has an association relationship with a third vertex, wherein the third vertex is a vertex for searching a next associated vertex, and an outgoing end of the first directed edge connecting the fourth vertex and the third vertex points to the fourth vertex; Allocating a screening weight to each of the fourth vertices according to the vertex association degree indicated by the first directed edge; Randomly select a fifth vertex located after the third vertex in the vertex sequence from the plurality of fourth vertices according to the screening weight; Detect the reference number of vertices included in the currently filtered vertex sequence; When the reference number is equal to the target number, the vertex sequence that has been currently screened out is determined as the first vertex sequence.

11. The method according to claim 9, characterized in that The step of constructing a first relationship graph of a plurality of the second sub-browsing information according to the browsing similarities includes: Creating the first vertex corresponding to each of the second sub-browsing information, and configuring the vertex association between the first vertices corresponding to the second sub-browsing information according to the browsing similarity between the second sub-browsing information; Creating a first connecting edge between the first vertices whose vertex association degree is greater than or equal to a target association degree; According to the occurrence time sequence relationship of the second sub-browsing information corresponding to the first vertex, the pointing relationship of the first connecting edge is configured to obtain the first directed edge, wherein the incident end of the first directed edge is connected to the vertex that occurs earlier in the time sequence, and the outgoing end of the first directed edge is connected to the vertex that occurs later in the time sequence; A relationship graph of the first directed edges that records connections between a plurality of the first vertices and the first vertices is determined as the first relationship graph.

12. The method according to claim 8, characterized in that The predicting, based on the random walk sequence, the initial transfer relationship feature of the target sub-browsing information corresponding to the random walk sequence comprises: Calculating a target time series feature of each random walk sequence in a plurality of random walk sequences of each second sub-browsing information, wherein the time series feature is used to indicate a time series change of the browsing preference of the target account in the random walk sequence; Allocating a sequence weight to each of the random walk sequences according to the proportion of the target time series feature of each of the random walk sequences in the time series features and values ​​of the plurality of random walk sequences; The initial transfer relationship feature is obtained by weighted summing of the plurality of time series features according to the sequence weight.

13. The method according to claim 12, characterized in that The calculating of the target time series feature of each random walk sequence in the plurality of random walk sequences of each second sub-browsing information includes: Determine the sixth sub-browsing information to be processed from the random walk sequence according to the occurrence sequence of the sub-browsing information; Predicting a fourth preference feature of the target account after browsing the sixth sub-browsing information according to the third preference feature of the target account and the second browsing feature corresponding to the sixth sub-browsing information, wherein the third preference feature is used to indicate the browsing preference of the target account before browsing the sixth sub-browsing information in the random walk sequence; In a case where the sixth sub-browsing information is the last sub-browsing information in the random walk sequence, the fourth preference feature is determined as the target time series feature of the random walk sequence.

14. The method according to claim 13, characterized in that The predicting, based on the third preference feature of the target account and the second browsing feature corresponding to the sixth sub-browsing information, the fourth preference feature of the target account after browsing the sixth sub-browsing information includes: extracting second adjustment information from the third preference feature using a third reset gate, wherein the second adjustment information is used to indicate the degree of influence of the third preference feature on the browsing preference of the target account; The third update gate is called to update the second browsing feature according to the second adjustment information to obtain the fourth preference feature.

15. The method according to claim 8, characterized in that Generating the transfer relationship feature according to the initial transfer relationship features of the plurality of second sub-browsing information includes one of the following: Determining the average of the plurality of initial transfer relationship features as the transfer relationship feature; Determining a maximum value of a feature among a plurality of the initial transfer relationship features as the transfer relationship feature; A corresponding feature weight is assigned to each of the initial transfer relationship features according to the feature relationship between the initial transfer relationship features; and the initial transfer relationship features are weighted summed using the feature weights to obtain the transfer relationship features.

16. The method according to claim 1, characterized in that Before generating the target account feature of the target account according to the time sequence relationship feature, the transfer relationship feature and the resource browsing feature of the target account, the method further includes: Acquire second browsing information of the reference account, wherein the second browsing information is used to indicate multimedia resource browsing status of the reference account within a plurality of the reference time periods; The resource browsing characteristics of the target account are predicted according to the first browsing information and the second browsing information.

17. The method according to claim 16, characterized in that The predicting the resource browsing characteristics of the target account according to the first browsing information and the second browsing information includes: Constructing a second relationship graph according to the first browsing information and the second browsing information, wherein the second relationship graph includes a plurality of sixth vertices, a plurality of seventh vertices, a first edge and a second edge, wherein the sixth vertex corresponds to the target account and the reference account in a one-to-one manner, the seventh vertex corresponds to the multimedia resources included in the first browsing information and the multimedia resources included in the second browsing information in a one-to-one manner, the first edge is used to indicate an association relationship between accounts, and the second edge is used to indicate an access relationship between an account and a multimedia resource; Extracting target vertex features of a target vertex in the second relationship graph using a feature extraction model including a plurality of sequentially connected feature extraction layers, wherein the target vertex is any vertex in the second relationship graph, each of the feature extraction layers is used to generate vertex features of the target vertex according to initial vertex features of a reference vertex in the current second relationship graph, and updating vertex features of the target vertex in the second relationship graph using the vertex features of the target vertex output by the current feature extraction layer, wherein the reference vertex is a vertex in the second relationship graph that has an association relationship with the target vertex; The target vertex feature of the vertex corresponding to the target account is determined as the resource browsing feature.

18. The method according to claim 17, characterized in that The extracting the target vertex feature of the target vertex in the second relationship graph by using a feature extraction model including a plurality of sequentially connected feature extraction layers comprises: Calling a target feature extraction layer to extract a target vertex set of the target vertex from the second relationship graph, wherein the target vertex set records vertices in the second relationship graph that are connected to the target vertex through the first edge or the second edge, and the target feature extraction layer is an extraction layer that currently performs feature extraction on the target vertex in the second relationship graph; Calling the target feature extraction layer to perform feature aggregation on the initial vertex features currently possessed by the vertices included in the target vertex set, to obtain the reference vertex features of the target vertices output by the target feature extraction layer; In a case where the target feature extraction layer is the last layer of a plurality of sequentially connected feature extraction layers, the reference vertex feature is determined as the target vertex feature.

19. The method according to claim 18, characterized in that The calling of the target feature extraction layer to perform feature aggregation on the initial vertex features currently possessed by the vertices included in the target vertex set to obtain the reference vertex features of the target vertices output by the target feature extraction layer includes: Detecting the vertex type of the target vertex; In the case where the vertex type indicates that the target vertex is a vertex corresponding to an account, feature aggregation is performed on the initial vertex features of the sixth vertex included in the target vertex set to obtain a first aggregate feature, and feature aggregation is performed on the initial vertex features of the seventh vertex in the target vertex set to obtain a second aggregate feature; corresponding aggregate feature weights are assigned to the first aggregate feature and the second aggregate feature according to a feature relationship between the first aggregate feature and the second aggregate feature; and weighted sum calculation is performed on the first aggregate feature and the second aggregate feature using the aggregate feature weight to obtain the reference vertex feature; In a case where the vertex type indicates that the target vertex is a vertex corresponding to a multimedia resource, the initial vertex features of the vertices included in the target vertex set are aggregated to obtain the reference vertex features.

20. The method according to claim 18, characterized in that The calling of the target feature extraction layer to extract the target vertex set of the target vertex from the second relationship graph includes: Extracting an initial vertex set of the target vertex from the second relationship graph, wherein the initial vertex set includes vertices in the second relationship graph connected to the target vertex through the first edge or the second edge; A number of candidate vertices are screened out from the initial vertex set according to the current vertex features of the vertices in the initial vertex set to obtain the target vertex set, wherein, when the target vertex is a vertex corresponding to an account, the candidate vertices include vertices corresponding to the account and vertices corresponding to multimedia resources, and when the target vertex is a vertex corresponding to multimedia resources, the candidate vertices include vertices corresponding to the account, and the dependency relationship between the vertices in the target vertex set and the target vertex satisfies the target dependency relationship condition.

21. The method according to claim 20, characterized in that The step of selecting a candidate number of candidate vertices from the initial vertex set according to the current vertex features of the vertices in the initial vertex set to obtain the target vertex set includes: Using the vertex features of each vertex included in the first vertex set of the current screening round, calculating the incremental value of each vertex to the vertex set information, wherein the vertex set information is used to indicate the importance of the vertex features of all vertices in the second vertex set, and the incremental value is used to indicate the amount of change of the vertex set information of the second vertex set after the corresponding vertex in the first vertex set is added to the second vertex set, and the initial vertex set includes vertices in the first vertex set and vertices in the second vertex set; Extracting vertices whose increment values ​​are greater than or equal to the target increment value from the first vertex set and adding them to the second vertex set; When the number of candidate vertices included in the second vertex set to which vertices are added is the candidate number, and the candidate vertices include both vertices corresponding to accounts and vertices corresponding to multimedia resources, the second vertex set is determined as the target vertex set.

22. The method according to claim 1, characterized in that The generating the target account feature of the target account according to the time sequence relationship feature, the transfer relationship feature and the resource browsing feature of the target account includes: Performing feature splicing on the temporal relationship feature, the transfer relationship feature and the resource browsing feature to obtain a spliced ​​feature; The splicing features are fused using a feature fusion model to obtain the target account features output by the feature fusion model, wherein the feature fusion model records the conversion relationship between the splicing features and the account features.

23. The method according to claim 1, characterized in that The displaying of the target multimedia resource matching the target account feature to the target account includes: Calculating target similarity between the target account feature and target resource features of each multimedia resource in the multimedia resource library; Filtering the target multimedia resources whose target similarity is greater than or equal to a preset similarity from the multimedia resource library; The target multimedia resource is displayed to the target account.

24. The method according to claim 23, characterized in that The calculating the target similarity between the target account feature and the target resource feature of each multimedia resource in the multimedia resource library includes: Generate a first resource feature of the multimedia resource in the first browsing information and a second resource feature of the multimedia resource in the multimedia resource library using a feature generation model; Detecting feature difference information between the first resource feature and the second resource feature; Using the feature adjustment amount corresponding to the feature difference information, feature adjustment is performed on the initial resource feature of the multimedia resource in the first browsing information to obtain the target resource feature, wherein the initial resource feature is used to indicate the account feature of the account browsing the multimedia resource in the first browsing information; The target similarity between the target account feature and the target resource feature is calculated.

25. A multimedia resource display device, characterized in that: include: A first prediction module is used to predict a temporal relationship feature of the target account based on first browsing information of the target account, and to predict a transfer relationship feature of the target account based on the first browsing information, wherein the first browsing information is used to indicate the multimedia resource browsing situation of the target account in multiple reference time periods before the current moment, and the temporal relationship feature is used to indicate the temporal change of the browsing preference of the target account for multimedia resources, and the temporal relationship feature is obtained by learning the temporal relationship between the resource browsing features of the target account in multiple reference time periods in the first browsing information, and the transfer relationship feature is used to indicate the current browsing preference of the target account under the influence of the multimedia resources browsed in each reference time period; A generating module, configured to generate a target account feature of the target account according to a temporal relationship feature, a transfer relationship feature and a resource browsing feature of the target account, wherein the resource browsing feature is used to indicate a multimedia resource browsing preference of the target account under the influence of multimedia resource browsing information of a reference account associated with the target account, and the target account feature is used to indicate a multimedia resource browsing preference of the target account at a current moment; A display module is used to display target multimedia resources matching the characteristics of the target account to the target account.

26. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 24 when executed by a processor.

27. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 24 are implemented.

28. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 24 are implemented.

Images