Media resource pushing method and device and electronic equipment

Abstract

The invention discloses a media resource pushing method and device and electronic equipment. The method comprises the following steps: acquiring multi-modal data associated with a to-be-processed resource pushing task; in a graph structure in which each type of media resource data is mapped into nodes, determining an edge weight of an edge in the graph structure based on mutual information amount among the media resource data; under the condition that the edge weight of each edge in the graph structure is determined, determining a current mutual information total amount matched with the graph structure and a fusion feature vector corresponding to the graph structure by utilizing the data feature vector of the media resource data corresponding to each node in the graph structure and the edge weight; acquiring a resource recommendation coefficient corresponding to the fusion feature vector under the condition that the current mutual information total amount meets a convergence condition; and determining a target media resource matched with the resource pushing task according to the resource recommendation coefficient. The technical problem that the recommended content is inaccurate in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of data analysis, and more specifically, to a method, device and electronic device for pushing media resources. Background Art

[0002] Traditional content recommendation systems often recommend content to users based on simple statistical models or preset rules. These systems have obvious deficiencies in dealing with user personalized needs. They may not be able to accurately capture subtle changes in user preferences, resulting in the homogenization of recommended content. Users may therefore fall into an "information cocoon" and only be exposed to the types of information they are already familiar with or prefer, limiting their access to broader and more diverse content. In addition, these systems also face the "cold start" problem when dealing with new users or new items because there is a lack of sufficient historical data to make accurate recommendations;

[0003] That is, the current system is in a complex and changeable environment and is used to handle complex task requirements. However, system decisions usually depend on static parameters / data of a single dimension / single moment, which makes the system's decision on the resources to be pushed relatively single / limited and one-sided, resulting in inaccurate pushed resources. Therefore, there is a technical problem in the current related technologies that the recommended content is not accurate enough.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method, device and electronic device for pushing media resources, so as to at least solve the technical problem of inaccurate recommended content in the current related technologies.

[0006] According to one aspect of an embodiment of the present application, a method for pushing media resources is provided, comprising: obtaining multimodal data associated with a resource push task to be processed, wherein the multimodal data is used to indicate M types of media resource data collected in the resource push task, and M is a natural number greater than or equal to 2; in a graph structure in which each type of media resource data is mapped to a node, based on the mutual information between the media resource data of the i-th type and the media resource data of the j-th type in the multimodal data, determining the edge weight of the edge in the graph structure for connecting the i-th node corresponding to the media resource data of the i-th type and the j-th node corresponding to the media resource data of the j-th type, wherein the mutual information is used to indicate The data correlation between the media resource data of the i-th category and the media resource data of the j-th category, i and j are natural numbers greater than or equal to 1 and less than or equal to M; when the edge weight of each edge in the graph structure is determined, the current total amount of mutual information matching the graph structure and the fusion feature vector corresponding to the graph structure are determined using the data feature vector of the media resource data corresponding to each node in the graph structure, and the edge weight, wherein the edge weight of each edge in the graph structure is dynamically adjusted; when the current total amount of mutual information meets the convergence condition, the resource recommendation coefficient corresponding to the fusion feature vector is obtained; and the target media resource matching the resource push task is determined according to the resource recommendation coefficient.

[0007] According to another aspect of an embodiment of the present application, a media resource pushing device is also provided, including: a first acquisition unit, which acquires multimodal data associated with a resource push task to be processed, wherein the multimodal data is used to indicate M types of media resource data collected in the resource push task, and M is a natural number greater than or equal to 2; a first determination unit, which, in a graph structure in which each type of media resource data is mapped to a node, determines the edge weight of the edge in the graph structure for connecting the i-th node corresponding to the media resource data of the i-th category and the j-th node corresponding to the media resource data of the j-th category based on the mutual information between the media resource data of the i-th category and the media resource data of the j-th category in the multimodal data, wherein the mutual information is used to indicate the i-th category The data correlation between the media resource data of the jth category and the media resource data of the jth category, i and j are natural numbers greater than or equal to 1 and less than or equal to M; the second determination unit, after determining the edge weight of each edge in the graph structure, uses the data feature vector of the media resource data corresponding to each node in the graph structure, and the edge weight to determine the current total amount of mutual information matching the graph structure and the fusion feature vector corresponding to the graph structure, wherein the edge weights of each edge in the graph structure are dynamically adjusted; the second acquisition unit, when the current total amount of mutual information meets the convergence condition, acquires the resource recommendation coefficient corresponding to the fusion feature vector; the third determination unit determines the target media resource matching the resource push task according to the resource recommendation coefficient.

[0008] Optionally, the second acquisition unit includes: an input module, used to input the fused feature vector into a target decision model, wherein each model parameter in the target decision model is dynamically adjusted according to feedback information, and the feedback information is determined based on operation information of task feedback operations of historical resource push tasks counted within a first historical reference time period, and the first historical reference time period is before determining the current total amount of mutual information; obtaining a resource recommendation coefficient corresponding to the fused feature vector output by the target decision model, wherein the resource recommendation coefficient is used to indicate the push probability of each type of media resource data.

[0009] Optionally, the above-mentioned second acquisition unit includes: a third acquisition module, used to obtain reference feedback information corresponding to a reference total amount of mutual information before the current total amount of mutual information, wherein the reference total amount of mutual information is the total amount of mutual information determined before the current total amount of mutual information, and the reference feedback information is determined based on the operation information of the task feedback operation of the historical resource push task counted within the second historical reference time period, and the second historical reference time period is before the reference total amount of mutual information is determined; when the reference feedback information indicates that the resource push statistics result of the pushed reference media resource that meets the push condition reaches a statistical threshold, it is determined that the current total amount of mutual information meets the convergence condition; or the reference total amount of mutual information before the current total amount of mutual information is obtained, wherein the reference total amount of mutual information is the total amount of mutual information determined before the current total amount of mutual information; when the change between the reference total amount of mutual information and the current total amount of mutual information is less than the target threshold, it is determined that the current total amount of mutual information meets the convergence condition.

[0010] Optionally, the above-mentioned third acquisition module includes: an adjustment module, which is used to adjust the edge weight of the edge between the media resource data of the i-th category and the media resource data of the j-th category when the reference feedback information indicates that the resource push statistics of the pushed reference media resources meet the push conditions have not reached the statistical threshold; and adjust the edge weight of the edge between the media resource data of the i-th category and the media resource data of the j-th category when the change between the reference mutual information total amount and the current mutual information total amount is greater than or equal to the target threshold.

[0011] Optionally, the above-mentioned adjustment module also includes an update module, which is used to obtain a reference resource push conversion rate matching the reference feedback information, wherein the reference resource push conversion rate is positively correlated with the reference feedback information; based on the reference feedback information and the reference resource push conversion rate, determine the current resource push conversion rate matching the current feedback information; based on the current resource push conversion rate matching the current feedback information, update the i-th node weight matching the data feature vector of the media resource data of the i-th category to obtain an updated i-th node weight, and update the j-th node weight matching the data feature vector of the media resource data of the j-th category to obtain an updated j-th node weight; based on the updated i-th node weight and the updated j-th node weight, adjust the edge weight of the edge between the media resource data of the i-th category and the media resource data of the j-th category.

[0012] Optionally, the above-mentioned third acquisition module is also used to obtain information entropy matching each category of media resource data, wherein the information entropy is used to indicate the potential conversion probability of the media resource data being successfully converted; based on the i-th information entropy matching the i-th category of media resource data and the j-th information entropy matching the j-th category of media resource data, determine the mutual information between the i-th category of media resource data and the j-th category of media resource data in the multimodal data.

[0013] Optionally, the above-mentioned second determination unit also includes a distribution module, which is used to obtain historical feedback information matching the target media resource based on the historical operation data of the target media resource; adjust the display mode parameters of the target media resource according to the historical feedback information matching the target media resource to obtain the adjusted display mode parameters; encapsulate the target media resource and the adjusted display mode parameters to obtain a target media resource data packet; and distribute the target media resource data packet to each resource display device when the sending priority of the target media resource data packet is higher than the sending priority of other media resource data packets in the queue of data packets to be sent.

[0014] Optionally, the above-mentioned distribution module is also used to obtain the device address of a second server device serving as a backup device of the first server device when the first server device used to distribute the target media resource data packet is in a faulty state; and send the target media resource data packet and the current distribution progress information corresponding to the target media resource data packet to the second server device, so that the second server device continues to distribute the target media resource data packet according to the current distribution progress information.

[0015] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for pushing media resources when running.

[0016] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above method for pushing media resources.

[0017] According to another aspect of the embodiments of the present application, there is further provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-mentioned method for pushing media resources through the computer program.

[0018] In an embodiment of the present application, multimodal data associated with a resource push task to be processed is obtained; further, in a graph structure in which each category of media resource data is mapped to nodes, the edge weights of the edges in the graph structure are determined based on the mutual information between the media resource data; when the edge weights of each edge in the graph structure are determined, the current total amount of mutual information matching the graph structure and the fused feature vector corresponding to the graph structure are determined using the data feature vectors of the media resource data corresponding to each node in the graph structure, and the edge weights; thereby, when the current total amount of mutual information meets the convergence conditions, a resource recommendation coefficient corresponding to the fused feature vector is obtained; and it is achieved that the target media resource matching the resource push task is determined based on the resource recommendation coefficient, thereby solving the technical problem of inaccurate recommended content in the current related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 is a schematic diagram of an application environment of an optional method for pushing media resources according to an embodiment of the present application;

[0021] Figure 2 is a flow chart of an optional method for pushing media resources according to an embodiment of the present application;

[0022] Figure 3 is a flow chart of another optional method for pushing media resources according to an embodiment of the present application;

[0023] Figure 4 is a flow chart of another optional method for pushing media resources according to an embodiment of the present application;

[0024] Figure 5is a flow chart of another optional method for pushing media resources according to an embodiment of the present application;

[0025] Figure 6 is a schematic diagram of an optional method for pushing media resources according to an embodiment of the present application;

[0026] Figure 7 is a flow chart of another optional method for pushing media resources according to an embodiment of the present application;

[0027] Figure 8 is a structural diagram of an optional media resource push device according to an embodiment of the present application;

[0028] Fig. 9 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] According to one aspect of the embodiments of the present application, a method for pushing media resources is provided. Optionally, as an optional implementation, the method for pushing media resources can be applied to, but is not limited to, Figure 1 in the environment shown.

[0032] S102, the terminal device 102 sends a data collection instruction to the server 104 via the network 110;

[0033] The server executes S104-S112, S104, obtaining multimodal data associated with the resource push task to be processed, wherein the multimodal data is used to indicate M types of media resource data collected in the resource push task, where M is a natural number greater than or equal to 2;

[0034] S106, in a graph structure in which each category of media resource data is mapped to a node, based on the mutual information between the media resource data of the i-th category and the media resource data of the j-th category in the multimodal data, determining an edge weight of an edge in the graph structure for connecting an i-th node corresponding to the media resource data of the i-th category and a j-th node corresponding to the media resource data of the j-th category, wherein the mutual information is used to indicate a data association degree between the media resource data of the i-th category and the media resource data of the j-th category, and i and j are natural numbers greater than or equal to 1 and less than or equal to M;

[0035] S108, when the edge weight of each edge in the graph structure is determined, the current total amount of mutual information matching the graph structure and the fusion feature vector corresponding to the graph structure are determined using the data feature vector of the media resource data corresponding to each node in the graph structure and the edge weight, wherein the edge weight of each edge in the graph structure is dynamically adjusted;

[0036] S110, when the current total amount of mutual information meets the convergence condition, obtaining a resource recommendation coefficient corresponding to the fused feature vector;

[0037] S112, determining a target media resource that matches the resource push task according to the resource recommendation coefficient; the server 104 includes a database and a processing engine, and the terminal device 102 includes a display, a processor, and a memory;

[0038] S114, the server 104 sends a push result to the terminal device;

[0039] Then the terminal device executes S116 to display the target media resource;

[0040] Optionally, in this embodiment, the terminal device may be a terminal device configured with a target client, which may include but is not limited to at least one of the following: a mobile phone, a laptop, a tablet computer, a PAD, a smart TV, etc. The target client may be a video client, an instant messaging client, a browser client, an education client, etc. The network includes but is not limited to: wired networks such as local area networks and metropolitan area networks, and wireless networks such as Bluetooth and WIFI. The server may be a single server, a server cluster, or a cloud server. The above is only an example and is not limited to this.

[0041] Alternatively, if Figure 2 As shown, the method for pushing the above media resources includes:

[0042] S202, acquiring multimodal data associated with the resource push task to be processed, wherein the multimodal data is used to indicate M types of media resource data collected in the resource push task, where M is a natural number greater than or equal to 2;

[0043] S204, in a graph structure in which each category of media resource data is mapped to a node, based on the mutual information between the media resource data of the i-th category and the media resource data of the j-th category in the multimodal data, determining an edge weight of an edge in the graph structure for connecting an i-th node corresponding to the media resource data of the i-th category and a j-th node corresponding to the media resource data of the j-th category, wherein the mutual information is used to indicate a data association degree between the media resource data of the i-th category and the media resource data of the j-th category, and i and j are natural numbers greater than or equal to 1 and less than or equal to M;

[0044] S206, when the edge weight of each edge in the graph structure is determined, the current total amount of mutual information matching the graph structure and the fusion feature vector corresponding to the graph structure are determined using the data feature vector of the media resource data corresponding to each node in the graph structure and the edge weight, wherein the edge weight of each edge in the graph structure is dynamically adjusted;

[0045] S208, when the current total amount of mutual information meets the convergence condition, obtaining a resource recommendation coefficient corresponding to the fused feature vector;

[0046] S210: Determine a target media resource that matches the resource push task according to the resource recommendation coefficient.

[0047] In an optional implementation, in step S202, the multimodal data may be video data, such as images captured in real time by cameras at different camera positions; audio data, such as ambient sounds at each camera position, speeches by the host or guests, cheers from the audience, etc.; motion sensor data, such as motion data of a camera or device, such as panning, shaking, etc.; preset scene information, such as stage layout, locations of important areas (such as guest seats, performance areas), etc.; time information, such as timestamps, scheduled program flow, advertising time slots, etc.; facial expressions and emotions of characters in live broadcast images, such as the emotional state of the audience or performers; background noise and environment, such as the background noise level and other environmental changes in the detection environment, such as lighting, smoke, and other types of media resource data related to the push task.

[0048] Furthermore, in step S204, the relationship between different modal data is expressed using a graph structure, and the mutual information between the media resource data of the i-th category and the media resource data of the j-th category in the multimodal data can be calculated by the following formula:

[0049] I(xi , x j )=H(x i )+H(x j )-H(x i , x j );

[0050] H(x) represents the mode x i and x j The information entropy of represents the uncertainty of the modal information itself, H(x i , x j ) is the joint entropy between the modes, indicating the total uncertainty when they appear together. The higher the mutual information, the stronger the information complementarity between the modes, and the greater the weight of the edge between the two modes. Further calculating the edge weight, the above edge weight can indicate the complementarity of information between the modes, and the calculation formula of the edge weight can be: w ij =a*w i +b*w j +c,w i and w j is the weight of the feature data corresponding to each of the two connected nodes in the graph structure. Parameters a, b, and c are dynamically updated during the reinforcement learning process. Specifically, a and b can be used to indicate the importance (contribution) of each modal data in the output recommendation result. If the value of a is large, it means that the modal data corresponding to the feature data under the i-th node has a significant impact on w. ij The influence of is greater, which leads to an increase in the recommendation probability of the modal data corresponding to node i in the recommendation result; c represents a basic weight value, which is used to keep the activation function working in the nonlinear region and to prevent overfitting to a certain extent to maintain the balance of the neural network model.

[0051] In the above step S206, after obtaining the feature weights corresponding to the nodes in the graph structure and the weights of the edges between the nodes, the total mutual information of the graph structure is further calculated in combination with the mutual information calculated above, for example, by the following formula:

[0052] I total =∑ i,j w ij *I(x i , x j ); I(x i , x j ) represents the mutual dependence between node i and node j (the degree of information sharing), w ij *I(x i , x j ) represents the contribution of local mutual information to global mutual information; I totalIt is the weighted sum of all local mutual information, quantifies the mutual information between nodes in the graph structure, and uses edge weights to adjust the importance of this information, thereby improving the accuracy of the output results by maximizing the global mutual information.

[0053] In addition, in this embodiment, through the graph structure, in the graph neural network, each node not only uses its own feature information, but also exchanges information with other nodes through edge weights, thereby learning a richer feature representation containing other modal information, and using nonlinear activation functions to generate a fused feature vector through multi-layer information propagation, for example, it can be calculated according to the following formula:

[0054]

[0055] in, is the feature representation of node i at layer k, N(i) is the set of neighbor nodes of node i, σ is the nonlinear activation function, represents the weighted sum of the feature representations of node i’s neighbor node j at the kth layer, and represents the information received by node i from other neighbor nodes;

[0056] It should be noted that in this embodiment, in order to maximize the mutual information between the modalities, that is, to find the weight configuration with the highest information sharing degree between different modal data sources, the edge weights of the graph structure can be continuously optimized through technologies such as genetic algorithm (GA) or particle swarm optimization (PSO), and finally the weight combination that maximizes the total amount of mutual information is found.

[0057] In an optional implementation, in the above step S208, when the current total amount of mutual information meets the convergence condition, the resource recommendation coefficient corresponding to the fused feature vector is obtained; for example, the convergence condition may be I total Reaching the maximum value, the total amount of mutual information reaching the maximum value can be understood as gradually reducing the loss of information in the process of information propagation, that is, in the conversion process from the original graph structure to the node representation, the information loss is minimized, thereby retaining more useful information (key information) and reducing noise. The resource recommendation coefficient can be a push probability, which determines the push probability of different types of resource data corresponding to the fused feature vector obtained by the multimodal data.

[0058] Then, in step S210, the target media resource matching the resource push task is determined according to the resource recommendation coefficient, for example, the resource recommendation coefficient is input into a matching strategy recommendation model, for example, I total When the maximum value is reached, the resource recommendation coefficient corresponding to the video resource is the largest, and a strategy recommendation model with a high video resource weight value can be selected. For example, if the video resource weight value is higher than 0.6, video resources in 10S and image resources in 3S will be recommended.

[0059] The following is a specific process to illustrate the above process, such as Figure 3 As shown:

[0060] S302, environmental perception and data collection, including S302-1, collecting sensor data, S302-2, collecting image data, S302-3, collecting text data;

[0061] Further, S304 is executed according to the data in S302-1-S302-3 to collect data in real time;

[0062] S306, data fusion; specifically, data fusion includes cleaning, normalizing and standardizing the data of each data source, determining whether the information in different data sources is related, and how to associate the data. It can be but not limited to combining data from different sources according to a certain logic or rule through matching of timestamps, spatial locations or other attributes. In this embodiment, the fused feature vector is obtained through a graph structure.

[0063] S308, key information identification, useful information extraction, such as identifying audio with keywords and extracting information within a time period with a large number of comments; S310, adaptive learning model update; specifically, based on the results of key information identification, useful features are extracted, such as feedback information, new data samples are added to the original data set, and the model is retrained so that the model can adapt to the new data, that is, the training model value.

[0064] Further, according to the update of the parameters after the model is updated, S312 is continued to be executed to strengthen the learning algorithm, that is, according to the decision results and feedback information, the weights of each modal resource media in the graph structure model are dynamically adjusted. Specifically, the weight value of the node can be updated by dynamically adjusting the learning rate, so that the total amount of mutual information in the training model converges; corresponding to each training, the weight value is dynamically adjusted and S314 is further executed to update the decision model;

[0065] S316, decision execution; S318, result feedback;

[0066] It should be noted that after the result feedback, the process can be ended directly and resource recommendations can be made;

[0067] You can also continue to learn and execute S320, decision effect evaluation; S322, learning strategy adjustment, such as adjusting the learning rate to update the weight value; S324, system continuous learning and optimization; S326, decision effect optimization; end the process.

[0068] Through the above-mentioned implementation mode of the present application, multimodal data associated with the resource push task to be processed is obtained; further, in a graph structure in which each category of media resource data is mapped to a node, the edge weight of the edge in the graph structure is determined based on the mutual information between the media resource data; when the edge weight of each edge in the graph structure is determined, the current total amount of mutual information matching the graph structure and the fused feature vector corresponding to the graph structure are determined using the data feature vector of the media resource data corresponding to each node in the graph structure, and the edge weight; thereby, when the current total amount of mutual information meets the convergence condition, the resource recommendation coefficient corresponding to the fused feature vector is obtained; the target media resource matching the resource push task is determined according to the resource recommendation coefficient, thereby solving the technical problem of inaccurate recommended content in the current related technology.

[0069] In an optional implementation, when the current total amount of mutual information meets the convergence condition, obtaining a resource recommendation coefficient corresponding to the fused feature vector includes:

[0070] S1, inputting the fused feature vector into the target decision model, wherein each model parameter in the target decision model is obtained by dynamically adjusting according to the feedback information, and the feedback information is determined according to the operation information of the task feedback operation of the historical resource push task counted in the first historical reference time period, and the first historical reference time period is before the current total amount of mutual information is determined;

[0071] S2, obtaining a resource recommendation coefficient corresponding to the fused feature vector output by the target decision model, wherein the resource recommendation coefficient is used to indicate the push probability of each type of media resource data.

[0072] Optionally, in the first method, a weighted summation result of at least one decision result is obtained, and a live media resource stream with the highest decision probability is determined as a target media resource stream according to the weighted summation result;

[0073] As an optional implementation, the feedback information in step S1 includes positive feedback operations and negative feedback operations further performed by the user after receiving the recommended media resources, for example, positive feedback operations include likes, comments, forwarding, etc., and negative feedback operations include dislikes, unfollowing, etc.

[0074] After the fused feature vector is input into the target decision model, in step S2, a resource recommendation coefficient corresponding to the fused feature vector output by the target decision model is obtained, wherein the resource recommendation coefficient is used to indicate the push probability of each type of media resource data;

[0075] Through the above-mentioned implementation mode of the present application, the fused feature vector is input into the target decision model, and then the resource recommendation coefficient corresponding to the fused feature vector output by the target decision model is obtained, and the recommendation probability corresponding to each modal media data is determined, and then the weight of recommending each media resource can be determined according to the probability, thereby improving the problem of inaccurate recommended content.

[0076] In an optional implementation, after determining the current total amount of mutual information matching the graph structure and the fused feature vector corresponding to the graph structure, at least one of the following is further included:

[0077] Method 1: obtaining reference feedback information corresponding to a reference total amount of mutual information before a current total amount of mutual information, wherein the reference total amount of mutual information is a total amount of mutual information determined before the current total amount of mutual information, and the reference feedback information is determined based on operation information of task feedback operations of historical resource push tasks counted within a second historical reference time period, and the second historical reference time period is before the reference total amount of mutual information is determined; when the reference feedback information indicates that the resource push statistics result of the pushed reference media resource that meets the push condition reaches a statistical threshold, determining that the current total amount of mutual information meets the convergence condition;

[0078] Method 2: obtaining a reference total mutual information before a current total mutual information, wherein the reference total mutual information is the total mutual information determined before the current total mutual information; when a change between the reference total mutual information and the current total mutual information is less than a target threshold, determining that the current total mutual information meets the convergence condition.

[0079] In method one, as an optional implementation, reference feedback information corresponding to the reference mutual information total amount before the current mutual information total amount is obtained. For example, the operation information of the task feedback operation of the historical resource push task can be counted, such as the user's sharing behavior after each picture resource is pushed by the historical push task. If the number of sharing times of multiple historical pushed picture users reaches 10 times within 10 minutes, it can be determined that the reference feedback information indicates that the pushed reference media resource meets the push condition, and the resource push statistical result reaches the statistical threshold, and it is determined that the current total mutual information meets the convergence condition.

[0080] In another optional implementation, the change between two consecutive total mutual information amounts can be obtained. For example, the preset target threshold is 15%, and the change between the previous reference total mutual information amount and the current total mutual information amount is 10%. If it is less than the above target threshold, it is determined that the current total mutual information meets the convergence condition.

[0081] Through the above implementation method recorded in this application, when the total amount of mutual information corresponding to the graph structure meets the convergence condition, resource recommendations are made according to the current weight values ​​of each modality, thereby improving the accuracy of the recommendation results.

[0082] In an optional implementation, after determining the current total amount of mutual information matching the graph structure and the fused feature vector corresponding to the graph structure, the method further includes:

[0083] S1, when the reference feedback information indicates that the resource push statistics result of the pushed reference media resources that meet the push conditions does not reach the statistical threshold, adjust the edge weight of the edge between the media resource data of the i-th category and the media resource data of the j-th category;

[0084] S2: When the variation between the reference total mutual information and the current total mutual information is greater than or equal to the target threshold, the edge weight of the edge between the media resource data of the i-th category and the media resource data of the j-th category is adjusted.

[0085] In this embodiment, when the reference feedback information indicates that the resource push statistics of the pushed reference media resources that meet the push conditions do not reach the statistical threshold, or when the change between the reference mutual information total amount and the current mutual information total amount is greater than or equal to the target threshold, the edge weight of the edge between the media resource data of the i-th category and the media resource data of the j-th category is further adjusted.

[0086] Specifically, assume that the system needs to integrate the user's historical purchase records (text data), product images (image data), and the user's social interaction information (network data) to generate personalized shopping recommendations. This involves three modal data: Modality 1: text data (historical purchase records); Modality 2: image data (product images); Modality 3: network data (social interaction);

[0087] First, the weight of each modality is equally distributed to 1 / 3. For example, the mutual information calculation results are as follows: I (text, image) = 0.6, I (text, network) = 0.3, I (image, network) = 0.4;

[0088] Assuming that the total amount of mutual information has not converged, further update the weights of the modal data. For example, after updating, the text weight is 0.4, the image weight is 0.4, and the network weight is 0.2.

[0089] According to the edge weight calculation formula w ij =a*w i +b*w j +c updates the edge weights, and further calculates the total mutual information based on the updated edge weights, I total =∑ i,j w ij *I(x i , x j ) until the total amount of mutual information reaches a maximum value, so that the recommendation result best meets the user's interests. This is only an example and does not specifically limit the above training process.

[0090] In an optional implementation, adjusting the edge weight of the edge between the media resource data of the i-th category and the media resource data of the j-th category includes:

[0091] S1, obtaining a reference resource push conversion rate that matches the reference feedback information, wherein the reference resource push conversion rate is positively correlated with the reference feedback information;

[0092] S2, determining a current resource push conversion rate that matches the current feedback information based on the reference feedback information and the reference resource push conversion rate;

[0093] S3, based on the current resource push conversion rate matched by the current feedback information, the weight of the i-th node matched with the data feature vector of the media resource data of the i-th category is updated to obtain the updated weight of the i-th node, and the weight of the j-th node matched with the data feature vector of the media resource data of the j-th category is updated to obtain the updated weight of the j-th node;

[0094] S4: Based on the updated ith node weight and the updated jth node weight, adjust the edge weight of the edge between the ith type of media resource data and the jth type of media resource data.

[0095] As an optional implementation, in the above step S1, the reference resource push conversion rate matching the reference feedback information is obtained. The above reference feedback information may be a real-time feedback signal Rt, such as a task success rate, execution time, and other performance indicators that are positively correlated with the reference feedback information; the above reference resource push conversion rate may be a learning rate. It can be understood that when the feedback information indicates that the execution result is good, the learning rate will increase to promote faster parameter adjustment, otherwise the learning rate will decrease to prevent the model from falling into a local optimum.

[0096] Further in step S2, based on the reference feedback information and the reference resource push conversion rate, the current resource push conversion rate matching the current feedback information is determined, that is, the learning rate is dynamically adjusted according to the real-time feedback signal, for example, the learning rate is updated by the following formula:

[0097] α t+1 =α t *(1+β*Rt);

[0098] α t+1 is the learning rate at time step t+1, α t is the current learning rate, β is the feedback weight coefficient, which controls the impact of the feedback signal on the learning rate adjustment, and Rt is a performance indicator such as task success rate and execution time that is positively correlated with the reference feedback information.

[0099] As an optional implementation, in step S3, based on the current resource push conversion rate matched by the current feedback information, the i-th node weight matched with the data feature vector of the media resource data of the i-th category is updated to obtain the updated i-th node weight, and the j-th node weight matched with the data feature vector of the media resource data of the j-th category is updated to obtain the updated j-th node weight;

[0100] Specifically, the weight can be updated by the following formula:

[0101] Where A is the overall loss function L(w,x) relative to the weight w i The gradient of is the weight value of the i-th type of modal data at the t-th time step, is the weight value of the i-th type of modal data at the t+1th time step; α is the learning rate; where w i represents the weight of the i-th type of modal data, x i is the input feature of the i-th type of modal data, l(x i , y) is the calculation of the traditional loss function (such as mean square error or cross entropy) on the i-th type of modal data. It can be understood that x represents the predicted value and y represents the true value. By minimizing the loss function, the difference between the predicted value and the true value is reduced, thereby improving the accuracy of the model. Based on this, the weights of each node in the graph structure are dynamically adjusted according to the feedback information.

[0102] The following Figure 4 Take the process shown as an example:

[0103] S402, the first data collection, including executing S402-1, collecting sensor data, S402-2, collecting image data, S402-3, collecting text data; performing data screening, and further executing S404, the second data collection, performing S404-1, standardization processing to ensure data format and dimension consistency, S404-2, cleaning processing to remove noise and irrelevant information and improve data quality, S404-3, feature extraction processing;

[0104] Further executing S406, data encoding, converting the data into a specific format or standard form, including executing S406-1, feature-level fusion, combining data of different modalities (such as sensor data, image features, and text features) at the feature level; S406-2, decision-level fusion, combining time, push mode, push content, etc. Specifically, different decision models are selected for different combinations of multimodal data, for example: modality type 1 and modality type 2 are fused at the feature level to obtain output result A, and modality type 1 and modality type 3 are fused at the feature level to obtain output result B, and then output result A and output result B are fused at the decision level to obtain the final output result;

[0105] According to the results of the above steps, S408, model evaluation is performed; S410, environmental perception, monitoring and determining changes in data information; S412, result output; S414, feedback loop; and the above steps are repeated according to the feedback results.

[0106] Through the above implementation method, the reference resource push conversion rate matched by the reference feedback information is obtained, and based on the reference feedback information and the reference resource push conversion rate, the current resource push conversion rate matched by the current feedback information is determined, and then based on the current resource push conversion rate matched by the current feedback information, the weights of the nodes corresponding to each modal data in the graph structure are adjusted, thereby dynamically adjusting the edge weights, and dynamically adjusting the model parameters in real time according to the user's feedback information, thereby achieving the ability to quickly respond to environmental changes and adjust the decision model in real time to obtain more accurate recommendation results.

[0107] In an optional implementation, after acquiring the multimodal data associated with the resource push task to be processed, the method includes:

[0108] S1, obtaining information entropy respectively matching each type of media resource data, wherein the information entropy is used to indicate a potential conversion probability of the media resource data being successfully converted;

[0109] S2, determining the mutual information between the media resource data of the i-th category and the media resource data of the j-th category in the multimodal data based on the i-th information entropy matching the media resource data of the i-th category and the j-th information entropy matching the media resource data of the j-th category.

[0110] In the above steps S1-S2, information entropies matching each category of media resource data are obtained, and the mutual information between the media resource data of the i-th category and the media resource data of the j-th category in the multimodal data is determined according to the i-th information entropy matching the media resource data of the i-th category and the j-th information entropy matching the media resource data of the j-th category, which can be calculated, for example, by the following formula:

[0111] I(x i , xj )=H(x i )+H(x j )-H(x i , x j );

[0112] H(x) represents the mode x i and x j The information entropy of represents the uncertainty of the modal information itself, H(x i , x j ) is the joint entropy between the modes, indicating the total uncertainty when they appear together.

[0113] Through the above-mentioned implementation mode of the present application, the model parameters are adjusted or the optimal model is selected in real time according to the data, and the model with poor effect in the current scene model is disabled, thereby improving the accuracy of the model output results.

[0114] In an optional implementation, after determining the target media resource matching the resource push task according to the resource recommendation coefficient, the following steps are included:

[0115] S1, based on the historical operation data of the target media resource, obtaining historical feedback information matching the target media resource;

[0116] S2, adjusting the display mode parameters of the target media resource according to the historical feedback information matching the target media resource to obtain adjusted display mode parameters;

[0117] S3, encapsulating the target media resource and the adjusted display mode parameters to obtain a target media resource data packet;

[0118] S4: When the sending priority of the target media resource data packet is higher than the sending priority of other media resource data packets in the to-be-sent data packet queue, the target media resource data packet is distributed to each resource display device.

[0119] In this embodiment, in the above steps S1-S3, the display mode parameters of the target media resource are adjusted by pushing the feedback information obtained by the target media resource at the historical time, and the above display mode parameters include video playback speed, character close-up, audio volume, subtitle size, etc., and then the target media resource and the adjusted display mode parameters are packaged to obtain the target media resource data packet;

[0120] Further, in the above step S4, if the sending priority of the target media resource data packet is higher than the sending priority of other media resource data packets in the to-be-sent data packet queue, the target media resource data packet is distributed to each resource display device;

[0121] Specifically, the resource usage of each computing node can be calculated, such as the load coefficient of the computing node, load coefficient = current node load / maximum node capacity, and the priority coefficient of each media resource can be determined according to the timeliness and importance of the target media resources in the push task. For example, if historical data shows that the importance and timeliness of video resources are high, then the corresponding priority level is high. In addition, this solution will also consider the dependencies between tasks. For example, if audio push and subtitle push are highly dependent, the related tasks will be assigned to adjacent nodes to reduce the delay in character data transmission.

[0122] The specific implementation method can be as follows Figure 5 As shown:

[0123] S502-1, issuing data monitoring tasks; S502-2, the operation console obtains manual operation data;

[0124] Further executing S504, Openapi performs interface encapsulation to allow the system to interact with other applications or services; S506-1, the component reads configuration information, for example, through Hermes; S506-2, the blackhole alarm module reads configuration information, and the blackhole alarm module is used to ensure that the alarm is responded to in a timely manner;

[0125] S508-1 and S508-2 are executed according to the execution results of S506-1 and S506-2, wherein S508-1, the cloud monitoring middle station reads metadata information, specifically including metadata management (which can be used to read and use description data), background management (which can be used to maintain system configuration, user permissions and task scheduling), negative feedback information (used to improve decision-making) and monitoring alarm management; S508-2, the cloud monitoring middle station executes decision management (which can be used based on the collected data and analysis results, the system supports the decision-making process), automated task issuance, for example, automated task issuance is implemented according to task priority strategy and fault tolerance mechanism, including automated task 1, automated task 2, automated task 3, automated task 4, etc., ensuring that the system maintains stability while executing efficiently, and ensures the reasonable distribution of tasks among different nodes while ensuring the load balancing of task processing at each node.

[0126] In an optional implementation, distributing the target media resource data packet to each resource display device includes:

[0127] S1, when a first server device for distributing a target media resource data packet is in a fault state, obtaining a device address of a second server device that is a backup device for the first server device;

[0128] S2, sending the target media resource data packet and the current distribution progress information corresponding to the target media resource data packet to the second server device, so that the second server device continues to distribute the target media resource data packet according to the current distribution progress information.

[0129] In the above steps S1-S2, when the first server device used to distribute the target media resource data packet is in a faulty state, the device address of the second server device serving as a backup device for the first server device is obtained, and the target media resource data packet and the current distribution progress information corresponding to the target media resource data packet are sent to the second server device, so that the second server device continues to distribute the target media resource data packet according to the current distribution progress information.

[0130] Specifically, in this embodiment, the status of each node is monitored in real time, including CPU utilization, memory usage, network latency, etc. When a node failure is detected, the task will be automatically transferred to the backup node. Through the copy mechanism, the system sets up multiple execution copies for critical tasks. When a node fails, other copies can take over seamlessly. And during the task execution process, the system regularly saves the intermediate status and progress of the task. Through the data snapshot mechanism, the task execution progress can be restored after a node failure, avoiding the task from the beginning. In addition, this solution adopts the Paxos or Raft consensus algorithm to ensure that the consistency of the system state and the integrity of the data are maintained during the task redistribution and fault recovery process, avoiding repeated execution or task loss.

[0131] The key processing steps in the automated decision-making system are described in a specific implementation method:

[0132] S1, multimodal data processing: First, the system uses advanced multimodal data processing technology to integrate user behavior and preference data in different modalities, such as text, images, videos, etc., to enhance the system's ability to perceive the environment and obtain a more comprehensive user portrait and item representation.

[0133] S2, Adaptive learning capability: The system has adaptive learning capability and can dynamically adjust recommendation strategies based on users’ real-time feedback and behavior data, using technologies such as deep learning to continuously optimize users’ interest models.

[0134] S3, real-time decision and execution engine: Using the real-time decision engine, the system can quickly respond to user requests, combine the user's real-time behavior and contextual information, and provide fast and accurate personalized recommendation results.

[0135] S4, Intelligent Feedback and Optimization Mechanism: The system is designed with an intelligent feedback mechanism to collect feedback information through user interaction behaviors, such as clicks, purchases, or ratings, and use reinforcement learning and other technologies to optimize strategies.

[0136] S5, Explainability and Controllability: In order to improve the transparency of recommendations and user trust, the system focuses on the explainability and controllability of recommendation strategies, so that users and regulators can understand the recommendation logic and adjust and control the recommendation results.

[0137] S6, Security and Privacy Protection: When processing user data, we strictly abide by data protection regulations and adopt encryption, desensitization, access control and other measures to ensure user data security and privacy is not violated.

[0138] The specific privacy calculation diagram is as follows Figure 6 As shown: Privacy computing includes three aspects: trusted computing environment, secure multi-party computing, and privacy computing technology;

[0139] The trusted computing environment can be a TEE trusted computing environment (a secure, isolated execution environment that protects code and data from attacks at the operating system software and hardware layers), a security sandbox (for testing and running untrusted code), a safe house (for storing sensitive data and performing secure operations), etc.

[0140] Secure multi-party computing includes: secret sharing, oblivious transmission, obfuscated circuits, homomorphic encryption, private information retrieval, and zero-knowledge proof;

[0141] Privacy computing technologies include: differential privacy, K-anonymity algorithm, and L-diversity diversification;

[0142] The overall automated decision-making system execution process is as follows Figure 7 As shown:

[0143] The following steps are performed at the user level: S702, open the backend; S704, edit the project, target, and environment parameters; S706, start executing the task target; S708, give a decision prompt; S710, normal logic execution;

[0144] The following steps are performed on the server: S712, collecting the current environment, pictures, and text; S714, writing local cache files; S716, notifying optimization suggestions for long links;

[0145] The AI ​​big model executes the following steps: S718, reporting the start event to the data information exchanger; S720, starting to match the target template; S722, matching the resource strategy; S724, matching the control strategy; S726, passing? If the matching strategy is judged to be passed, S728 is executed to optimize the relevant goals; if the judgment is not passed, S710 is executed for normal logic execution; it should be noted that the judgment conditions include but are not limited to whether the data is complete and compliant, whether the risk of the decision is lower than the preset threshold, whether there are sufficient resources, and historical feedback from users.

[0146] The following steps are performed on the big data platform: S730, forwarding data collection and network information collection to Kafka. It should be noted that Kafka is an open source data processing platform and a distributed stream data platform that can process large-scale real-time data streams; S732, Flume collects data. It should be noted that Flume big data collection system is used to process large-scale data collection, transmission and storage; S734, HDFS file system storage. It should be noted that HDFS distributed file system is a distributed file system for storing large amounts of data, which can store and manage data on multiple machines in a cluster;

[0147] S736, MapReduce data cleaning, it should be noted that MapReduce data cleaning refers to the process of cleaning and processing large-scale data using the MapReduce programming framework. In this process, the data is divided into small blocks, then processed in parallel, and finally the processed data is merged into the final result. Data cleaning usually includes operations such as data deduplication, data filtering, and data conversion to ensure data quality and accuracy. Through MapReduce data cleaning, large-scale data sets can be processed efficiently and the speed and efficiency of data processing can be improved.

[0148] S738, Mahout algorithm for data processing, which can help users perform various processing on large-scale data sets, from data cleaning to model training, and provides rich functions and tools.

[0149] S740, final result storage database;

[0150] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0151] According to another aspect of the embodiment of the present application, a media resource push device for implementing the above-mentioned media resource push method is also provided. Figure 8 As shown, the device comprises:

[0152] A first acquisition unit 802 acquires multimodal data associated with a resource push task to be processed, wherein the multimodal data is used to indicate M types of media resource data collected in the resource push task, where M is a natural number greater than or equal to 2;

[0153] A first determining unit 804 determines, in a graph structure in which each category of media resource data is mapped to a node, an edge weight of an edge in the graph structure for connecting an i-th node corresponding to the i-th category of media resource data and a j-th node corresponding to the j-th category of media resource data based on the mutual information between the i-th category of media resource data and the j-th category of media resource data in the multimodal data, wherein the mutual information is used to indicate a data association degree between the i-th category of media resource data and the j-th category of media resource data, and i and j are natural numbers greater than or equal to 1 and less than or equal to M;

[0154] The second determination unit 806, after determining the edge weight of each edge in the graph structure, uses the data feature vector of the media resource data corresponding to each node in the graph structure and the edge weight to determine the current total amount of mutual information matching the graph structure and the fusion feature vector corresponding to the graph structure, wherein the edge weight of each edge in the graph structure is dynamically adjusted;

[0155] The second acquisition unit 808 acquires a resource recommendation coefficient corresponding to the fused feature vector when the current total amount of mutual information meets the convergence condition;

[0156] The third determining unit 810 determines a target media resource that matches the resource push task according to the resource recommendation coefficient.

[0157] Optionally, the second acquisition unit includes: an input module, used to input the fused feature vector into a target decision model, wherein each model parameter in the target decision model is dynamically adjusted according to feedback information, and the feedback information is determined based on operation information of task feedback operations of historical resource push tasks counted within a first historical reference time period, and the first historical reference time period is before determining the current total amount of mutual information; obtaining a resource recommendation coefficient corresponding to the fused feature vector output by the target decision model, wherein the resource recommendation coefficient is used to indicate the push probability of each type of media resource data.

[0158] Optionally, the above-mentioned second acquisition unit includes: a third acquisition module, used to obtain reference feedback information corresponding to a reference total amount of mutual information before the current total amount of mutual information, wherein the reference total amount of mutual information is the total amount of mutual information determined before the current total amount of mutual information, and the reference feedback information is determined based on the operation information of the task feedback operation of the historical resource push task counted within the second historical reference time period, and the second historical reference time period is before the reference total amount of mutual information is determined; when the reference feedback information indicates that the resource push statistics result of the pushed reference media resource that meets the push condition reaches a statistical threshold, it is determined that the current total amount of mutual information meets the convergence condition; or the reference total amount of mutual information before the current total amount of mutual information is obtained, wherein the reference total amount of mutual information is the total amount of mutual information determined before the current total amount of mutual information; when the change between the reference total amount of mutual information and the current total amount of mutual information is less than the target threshold, it is determined that the current total amount of mutual information meets the convergence condition.

[0159] Optionally, the above-mentioned third acquisition module includes: an adjustment module, which is used to adjust the edge weight of the edge between the media resource data of the i-th category and the media resource data of the j-th category when the reference feedback information indicates that the resource push statistics of the pushed reference media resources meet the push conditions have not reached the statistical threshold; and adjust the edge weight of the edge between the media resource data of the i-th category and the media resource data of the j-th category when the change between the reference mutual information total amount and the current mutual information total amount is greater than or equal to the target threshold.

[0160] Optionally, the above-mentioned adjustment module also includes an update module, which is used to obtain a reference resource push conversion rate matching the reference feedback information, wherein the reference resource push conversion rate is positively correlated with the reference feedback information; based on the reference feedback information and the reference resource push conversion rate, determine the current resource push conversion rate matching the current feedback information; based on the current resource push conversion rate matching the current feedback information, update the i-th node weight matching the data feature vector of the media resource data of the i-th category to obtain an updated i-th node weight, and update the j-th node weight matching the data feature vector of the media resource data of the j-th category to obtain an updated j-th node weight; based on the updated i-th node weight and the updated j-th node weight, adjust the edge weight of the edge between the media resource data of the i-th category and the media resource data of the j-th category.

[0161] Optionally, the above-mentioned third acquisition module is also used to obtain information entropy matching each category of media resource data, wherein the information entropy is used to indicate the potential conversion probability of the media resource data being successfully converted; based on the i-th information entropy matching the i-th category of media resource data and the j-th information entropy matching the j-th category of media resource data, determine the mutual information between the i-th category of media resource data and the j-th category of media resource data in the multimodal data.

[0162] Optionally, the above-mentioned second determination unit also includes a distribution module, which is used to obtain historical feedback information matching the target media resource based on the historical operation data of the target media resource; adjust the display mode parameters of the target media resource according to the historical feedback information matching the target media resource to obtain the adjusted display mode parameters; encapsulate the target media resource and the adjusted display mode parameters to obtain a target media resource data packet; and distribute the target media resource data packet to each resource display device when the sending priority of the target media resource data packet is higher than the sending priority of other media resource data packets in the queue of data packets to be sent.

[0163] Optionally, the above-mentioned distribution module is also used to obtain the device address of a second server device serving as a backup device of the first server device when the first server device used to distribute the target media resource data packet is in a faulty state; and send the target media resource data packet and the current distribution progress information corresponding to the target media resource data packet to the second server device, so that the second server device continues to distribute the target media resource data packet according to the current distribution progress information.

[0164] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned method for pushing media resources is also provided. The electronic device may be Figure 1 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a mobile phone or a computer as an example. Fig. 9 As shown, the electronic device includes a memory 902 and a processor 904. The memory 902 stores a computer program, and the processor 904 is configured to execute the steps in any of the above method embodiments through the computer program.

[0165] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0166] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0167] S1, obtaining multimodal data associated with a resource push task to be processed, wherein the multimodal data is used to indicate M types of media resource data collected in the resource push task, where M is a natural number greater than or equal to 2;

[0168] S2, in a graph structure in which each category of media resource data is mapped to a node, based on the mutual information between the media resource data of the i-th category and the media resource data of the j-th category in the multimodal data, determine the edge weight of the edge in the graph structure for connecting the i-th node corresponding to the media resource data of the i-th category and the j-th node corresponding to the media resource data of the j-th category, wherein the mutual information is used to indicate the data association between the media resource data of the i-th category and the media resource data of the j-th category, and i and j are natural numbers greater than or equal to 1 and less than or equal to M;

[0169] S3, after determining the edge weight of each edge in the graph structure, using the data feature vector of the media resource data corresponding to each node in the graph structure and the edge weight, determine the current total amount of mutual information matching the graph structure and the fusion feature vector corresponding to the graph structure, wherein the edge weight of each edge in the graph structure is dynamically adjusted;

[0170] S4, when the current total amount of mutual information meets the convergence condition, obtain the resource recommendation coefficient corresponding to the fusion feature vector;

[0171] S5, determining a target media resource that matches the resource push task according to the resource recommendation coefficient.

[0172] Alternatively, a person skilled in the art may understand that: Fig. 9 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Fig. 9 The structure of the electronic device is not limited. Fig. 9 More or fewer components (such as network interfaces, etc.) as shown in, or with Fig. 9 Different configurations are shown.

[0173] Among them, the memory 902 can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for pushing media resources in the embodiments of the present application. The processor 904 executes various functional applications and data processing by running the software programs and modules stored in the memory 902, that is, to implement the above-mentioned method for pushing media resources. The memory 902 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 902 may further include a memory remotely located relative to the processor 904, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 902 may be specifically, but not limited to, used to store information such as page elements and page styles. As an example, such as Fig. 9 As shown, the memory 902 may include, but is not limited to, the first acquisition unit 802, the first determination unit 804, the second determination unit 806, the second acquisition unit 808, and the third determination unit 810 in the device for pushing media resources. In addition, other module units in the device for pushing media resources may also be included but are not limited to, which will not be described in detail in this example.

[0174] Optionally, the transmission device 906 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired network and a wireless network. In one example, the transmission device 906 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 906 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0175] In addition, the electronic device further includes: a display 908 for displaying the target page; and a connection bus 910 for connecting various module components in the electronic device.

[0176] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. The nodes may form a point-to-point network, and any form of computing device, such as a server, terminal or other electronic device, may become a node in the blockchain system by joining the point-to-point network.

[0177] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in the above various optional implementations;

[0178] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0179] S1, obtaining multimodal data associated with a resource push task to be processed, wherein the multimodal data is used to indicate M types of media resource data collected in the resource push task, where M is a natural number greater than or equal to 2;

[0180] S2, in a graph structure in which each category of media resource data is mapped to a node, based on the mutual information between the media resource data of the i-th category and the media resource data of the j-th category in the multimodal data, determine the edge weight of the edge in the graph structure for connecting the i-th node corresponding to the media resource data of the i-th category and the j-th node corresponding to the media resource data of the j-th category, wherein the mutual information is used to indicate the data association between the media resource data of the i-th category and the media resource data of the j-th category, and i and j are natural numbers greater than or equal to 1 and less than or equal to M;

[0181] S3, after determining the edge weight of each edge in the graph structure, using the data feature vector of the media resource data corresponding to each node in the graph structure and the edge weight, determine the current total amount of mutual information matching the graph structure and the fusion feature vector corresponding to the graph structure, wherein the edge weight of each edge in the graph structure is dynamically adjusted;

[0182] S4, when the current total amount of mutual information meets the convergence condition, obtain the resource recommendation coefficient corresponding to the fusion feature vector;

[0183] S5, determining a target media resource that matches the resource push task according to the resource recommendation coefficient.

[0184] Optionally, in the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0185] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0186] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0187] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0188] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0189] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0190] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0191] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for pushing media resources, characterized in that: include: Acquire multimodal data associated with the resource push task to be processed, wherein the multimodal data is used to indicate M types of media resource data collected in the resource push task, where M is a natural number greater than or equal to 2; In a graph structure in which each category of media resource data is respectively mapped to a node, based on the mutual information between the media resource data of the i-th category and the media resource data of the j-th category in the multimodal data, an edge weight of an edge in the graph structure for connecting an i-th node corresponding to the media resource data of the i-th category and a j-th node corresponding to the media resource data of the j-th category is determined, wherein the mutual information is used to indicate the data association between the media resource data of the i-th category and the media resource data of the j-th category, and i and j are natural numbers greater than or equal to 1 and less than or equal to M; In the case where the edge weight of each edge in the graph structure is determined, the current total amount of mutual information matching the graph structure and the fusion feature vector corresponding to the graph structure are determined using the data feature vector of the media resource data corresponding to each node in the graph structure and the edge weight, wherein the edge weight of each edge in the graph structure is dynamically adjusted; When the total amount of the current mutual information satisfies a convergence condition, obtaining a resource recommendation coefficient corresponding to the fused feature vector; A target media resource matching the resource push task is determined according to the resource recommendation coefficient.

2. The method according to claim 1, characterized in that When the current total amount of mutual information satisfies a convergence condition, obtaining a resource recommendation coefficient corresponding to the fused feature vector includes: Inputting the fused feature vector into a target decision model, wherein each model parameter in the target decision model is obtained by dynamically adjusting according to feedback information, and the feedback information is determined according to operation information of task feedback operations of historical resource push tasks counted in a first historical reference time period, and the first historical reference time period is before the current total amount of mutual information is determined; The resource recommendation coefficient corresponding to the fusion feature vector output by the target decision model is obtained, wherein the resource recommendation coefficient is used to indicate the push probability of each type of the media resource data.

3. The method according to claim 1, characterized in that After determining the current total amount of mutual information matching the graph structure and the fused feature vector corresponding to the graph structure, the method further includes at least one of the following: Obtaining reference feedback information corresponding to a reference total amount of mutual information before the current total amount of mutual information, wherein the reference total amount of mutual information is a total amount of mutual information determined before the current total amount of mutual information, and the reference feedback information is determined based on operation information of task feedback operations of historical resource push tasks counted within a second historical reference time period, and the second historical reference time period is before the reference total amount of mutual information is determined; when the reference feedback information indicates that a resource push statistical result that the pushed reference media resource meets the push condition reaches a statistical threshold, determining that the current total amount of mutual information meets the convergence condition; Obtain a reference total amount of mutual information before the current total amount of mutual information, wherein the reference total amount of mutual information is the total amount of mutual information determined before the current total amount of mutual information; when a change between the reference total amount of mutual information and the current total amount of mutual information is less than a target threshold, determine that the current total amount of mutual information meets the convergence condition.

4. The method according to claim 3, characterized in that: After determining the total current mutual information matching the graph structure and the fused feature vector corresponding to the graph structure, the method further includes: When the reference feedback information indicates that the resource push statistics result of the pushed reference media resources that meet the push condition does not reach the statistical threshold, adjusting the edge weight of the edge between the media resource data of the i-th category and the media resource data of the j-th category; When a change between the reference total mutual information and the current total mutual information is greater than or equal to a target threshold, the edge weight of the edge between the media resource data of the i-th category and the media resource data of the j-th category is adjusted.

5. The method according to claim 4, characterized in that The adjusting the edge weight of the edge between the media resource data of the i-th category and the media resource data of the j-th category includes: Acquire a reference resource push conversion rate that matches the reference feedback information, wherein the reference resource push conversion rate is positively correlated with the reference feedback information; Determining a current resource push conversion rate matched by current feedback information based on the reference feedback information and the reference resource push conversion rate; Based on the current resource push conversion rate matched by the current feedback information, the i-th node weight matched with the data feature vector of the media resource data of the i-th category is updated to obtain an updated i-th node weight, and the j-th node weight matched with the data feature vector of the media resource data of the j-th category is updated to obtain an updated j-th node weight; Based on the updated i-th node weight and the updated j-th node weight, the edge weight of the edge between the i-th category of media resource data and the j-th category of media resource data is adjusted.

6. The method according to claim 3, characterized in that After the multimodal data associated with the resource push task to be processed is obtained, the following steps are included: Acquire information entropy respectively matching each type of the media resource data, wherein the information entropy is used to indicate a potential conversion probability of the media resource data being successfully converted; The mutual information amount between the media resource data of the i-th category and the media resource data of the j-th category in the multimodal data is determined based on the i-th information entropy matching the media resource data of the i-th category and the j-th information entropy matching the media resource data of the j-th category.

7. The method according to claim 1, characterized in that After determining the target media resource matching the resource push task according to the resource recommendation coefficient, the method includes: Based on the historical operation data of the target media resource, acquiring historical feedback information matching the target media resource; According to the historical feedback information matching the target media resource, the display mode parameters of the target media resource are adjusted to obtain the adjusted display mode parameters; Encapsulating the target media resource and the adjusted display mode parameters to obtain a target media resource data packet; In the case where the sending priority of the target media resource data packet is higher than the sending priorities of other media resource data packets in the to-be-sent data packet queue, the target media resource data packet is distributed to each resource presentation device.

8. The method according to claim 7, characterized in that Distributing the target media resource data packet to each resource display device includes: When a first server device for distributing the target media resource data packet is in a fault state, obtaining a device address of a second server device serving as a backup device for the first server device; The target media resource data packet and current distribution progress information corresponding to the target media resource data packet are sent to the second server device, so that the second server device continues to distribute the target media resource data packet according to the current distribution progress information.

9. A device for pushing media resources, characterized in that: include: A first acquisition unit is configured to acquire multimodal data associated with a resource push task to be processed, wherein the multimodal data is used to indicate M types of media resource data collected in the resource push task, where M is a natural number greater than or equal to 2; A first determination unit, in a graph structure in which each category of media resource data is respectively mapped to nodes, determines, based on the mutual information between the media resource data of the i-th category and the media resource data of the j-th category in the multimodal data, an edge weight of an edge in the graph structure for connecting an i-th node corresponding to the media resource data of the i-th category and a j-th node corresponding to the media resource data of the j-th category, wherein the mutual information is used to indicate a data association degree between the media resource data of the i-th category and the media resource data of the j-th category, and i and j are natural numbers greater than or equal to 1 and less than or equal to M; A second determination unit, after determining the edge weight of each edge in the graph structure, uses the data feature vector of the media resource data corresponding to each node in the graph structure and the edge weight to determine the current total amount of mutual information matching the graph structure and the fusion feature vector corresponding to the graph structure, wherein the edge weight of each edge in the graph structure is dynamically adjusted; A second acquisition unit, when the current total amount of mutual information satisfies a convergence condition, acquires a resource recommendation coefficient corresponding to the fused feature vector; The third determining unit determines a target media resource matching the resource push task according to the resource recommendation coefficient.

10. An electronic device comprising: A memory and a processor, wherein the memory is used to store a computer program, and wherein the processor is used to execute the steps of the method according to any one of claims 1 to 8 when calling the computer program.

Images