AI agent driven digital media content distribution optimization method and system

Abstract

The present application relates to the technical field of digital media content distribution, and particularly relates to an AI intelligent body driven digital media content distribution optimization method and system. User interaction behavior data is collected to extract sequences and cross-channel migration paths, a comprehensive state representation is generated by a recurrent neural network and a graph convolutional encoder, a channel allocation probability distribution is output by a global scheduling intelligent body, a target channel identifier is sampled, an adaptation degree is calculated by each channel execution intelligent body based on local load and content characteristics, feedback delay is fed back when the adaptation degree is lower than a threshold value, and resampling is performed, an un-delayed intelligent body performs pushing, and global and execution intelligent body parameters are updated respectively by using user feedback and effect deviation. The method improves the efficiency and adaptation degree of content distribution.

Description

Technical Field

[0001] This invention relates to the field of digital media content distribution technology, and in particular to an AI-driven digital media content distribution optimization method and system. Background Technology

[0002] In the field of digital media content distribution, existing practices typically rely on preset distribution rules or statistically based recommendation algorithms. For example, platforms collect behavioral data such as users' historical clicks and browsing time, use collaborative filtering or matrix factorization models to calculate the matching degree between content and users, and push the content to various distribution channels according to a fixed priority. The configuration of distribution channels is often based on experience, such as prioritizing push notifications to active users and reaching infrequent users through email channels.

[0003] The drawback of the above approach is that it ignores the behavioral migration patterns of users across multiple channels. The user's transition path between different channels (such as web pages, mobile applications, and social media) is not modeled, causing content distribution to fail to adapt to their dynamic behavioral trajectories. In reality, users may browse one channel and then switch to another to complete the interaction, and traditional methods cannot identify this cross-channel funnel, resulting in a disconnect between the pushed content and the user's current state. Furthermore, the distribution strategy lacks real-time adaptability to channel load. Existing technologies typically assume that each channel has sufficient resources; when a channel experiences congestion (such as server overload or a sudden drop in user activity), pushes are still pushed as planned, leading to content delivery delays or user frustration. Simultaneously, the execution actions of each channel are independent, lacking a global coordination mechanism, making it difficult to achieve a balance between timeliness, content adaptability, and resource efficiency. Summary of the Invention

[0004] This invention provides an AI-driven digital media content distribution optimization method and system, which can solve the problems in the prior art.

[0005] A first aspect of this invention provides an AI agent-driven method for optimizing digital media content distribution, comprising:

[0006] Collect interactive behavior data of target users across multiple distribution channels, and extract user behavior sequences and cross-channel migration paths;

[0007] The user behavior sequence is input into a recurrent neural encoder to generate a state latent vector, and the cross-channel migration path is input into a graph convolutional encoder to generate a path latent vector. The two are then mapped through a fully connected layer to form a comprehensive user state representation.

[0008] A global scheduling agent and multiple channel execution agents are set up. The global scheduling agent receives the user's comprehensive state representation and outputs a channel allocation probability distribution. It samples from the allocation probability distribution to generate the target channel identifier for each content item and passes it to the corresponding channel execution agent.

[0009] Each channel execution agent reads the local real-time load status encoding into a status vector, calculates the vector dot product with the content features to obtain the fit degree, and when the fit degree is lower than the fit threshold, generates a delay mark and sends it back to the global scheduling agent, and removes the corresponding channel from the allocation probability distribution and resamples.

[0010] For channels that have not received a delay flag, the execution agent will push content. The global scheduling agent parameters will be updated using policy gradients based on user feedback. The parameters of each channel's execution agent will be updated using time-series differentials based on the deviation between the fit value and the actual effect.

[0011] The user behavior sequence is input into a recurrent neural encoder to generate a state latent vector, and the cross-channel migration path is input into a graph convolutional encoder to generate a path latent vector. Both are then mapped through a fully connected layer to form a comprehensive user state representation, including:

[0012] The user behavior sequence is encoded by behavior type and time interval. The two types of encoding are concatenated and input into a recurrent neural encoder. The recurrent neural encoder processes the encoding at each time step by step and updates the internal memory state. At the end of the sequence, the internal memory state is extracted as a state latent vector.

[0013] The access frequency and dwell time of each channel in the cross-channel migration path are extracted as node attributes, and the number of migrations and time intervals between adjacent channels are extracted as edge attributes. The graph convolutional encoder constructs a feature propagation matrix based on the node attributes and the edge attributes and performs layer-by-layer convolution operation. The central node features of the final convolutional layer are concatenated with the mean features of the whole graph to generate the path latent vector.

[0014] The mutual information between the state latent vector and the path latent vector is calculated to quantify their information redundancy. An adaptive fusion strategy is designed based on the information redundancy. The adaptive fusion strategy controls the contribution ratio of the two at the input of the fully connected layer through learnable parameters. The fully connected layer outputs a comprehensive user state representation and retains the complementary information of temporal features and graph structure features.

[0015] The graph convolutional encoder constructs a feature propagation matrix based on the node attributes and edge attributes and performs layer-by-layer convolution operations. It then concatenates the center node features of the final convolutional layer with the mean features of the entire graph to generate a path latent vector, including:

[0016] Extract the time interval between adjacent channels from the edge attributes and convert it into a time decay factor. Convert the node attributes into node initial embedding vectors. Construct a feature propagation matrix based on the time decay factor and the node initial embedding vectors.

[0017] The graph convolutional encoder performs matrix operations on the feature propagation matrix and the initial node embedding vector to obtain the first layer of aggregated features. The first layer of aggregated features and the initial node embedding vector are fused together after being transformed by a learnable weight matrix to generate the first layer of output features. The first layer of output features are then further processed with the feature propagation matrix to generate the second layer of output features.

[0018] The output features of the first layer and the output features of the second layer are mapped to a unified dimension through a projection layer and then concatenated. The concatenated features are input into a fully connected layer to generate fused features. The weight parameters of the fully connected layer are initialized according to the channel switching frequency and migration path length of the target user.

[0019] The feature vector of the target user's current channel location is extracted from the fusion features and used as the central node feature. The feature vectors of all channel locations in the fusion features are weighted and summed according to the historical access frequency to obtain the mean feature of the whole graph. This mean feature is then concatenated with the central node feature to obtain the path latent vector.

[0020] A global scheduling agent and multiple channel execution agents are configured. The global scheduling agent receives the user's comprehensive state representation and outputs a channel allocation probability distribution. It then samples from this probability distribution to generate target channel identifiers for each content item and transmits them to the corresponding channel execution agents. This includes:

[0021] The global scheduling agent maintains an experience memory bank of historical allocation decisions, retrieves historical samples similar to the user's comprehensive state representation from the experience memory bank, and extracts the successful allocation records corresponding to the historical samples as prior knowledge. The user's comprehensive state representation and prior knowledge are then fused and input into the decision network to generate an initial probability vector.

[0022] The load feature matrix is ​​constructed by reading the resource utilization rate and the number of pending tasks of each channel. The load feature matrix is ​​interactively calculated with the initial probability vector through a gating mechanism, and the probability value of each channel is dynamically adjusted. The channel allocation probability distribution after load perception adjustment is output.

[0023] Entropy value detection is performed on the channel allocation probability distribution to evaluate the concentration of the channel allocation probability distribution. When the entropy value exceeds the upper limit, sharpening is performed, and when the entropy value is below the lower limit, smoothing is performed. The processed channel allocation probability distribution is used as the sampling basis.

[0024] Each content item is independently sampled from the processed channel allocation probability distribution to generate a target channel identifier. The target channel identifier is encoded as routing information and a priority mark is attached. It is then broadcast to all channel execution agents through a publish-subscribe model. Each channel execution agent selects tasks belonging to its own channel based on the routing information and triggers their execution.

[0025] Each channel's execution agent reads the local real-time load status encoding into a state vector, calculates the vector dot product with the content features to obtain the fit degree, and when the fit degree is lower than the fit threshold, generates a delay label and sends it back to the global scheduling agent, and removes the corresponding channel from the allocated probability distribution and resamples, including:

[0026] Each channel execution agent constructs a hierarchical representation structure of local load status, using real-time load indicators as the bottom layer feature, load change rate as the middle layer feature, and load saturation prediction as the top layer feature. The hierarchical representation is fused and encoded into a state vector of a unified dimension through a multilayer perceptron.

[0027] Extract the types of computing resources required for processing and the expected duration of processing from the content items to construct a content feature vector. Perform a dot product operation between the state vector and the content feature vector to obtain the fit degree and calculate the confidence interval of the fit degree. When the lower bound of the fit degree is lower than the fit threshold, it is judged as a high-risk misfit.

[0028] The channel execution agent generates a delay tag and attaches a recommended alternative channel list. The recommended alternative channel list is generated based on the load coordination relationship between channels and the historical transfer success rate. The delay tag and the recommended alternative channel list are sent back to the global scheduling agent through a priority channel.

[0029] The global scheduling agent first selects candidate channels from the recommended alternative channel list, enhances the probability values ​​of the candidate channels in the allocation probability distribution, samples from the enhanced allocation probability distribution to generate the final alternative channel identifier, and constructs a scheduling instruction containing a retry limit to send to the channel execution agent corresponding to the final alternative channel identifier.

[0030] For channels that have not received a delay flag, the execution agent performs content push notifications. The global scheduling agent parameters are updated using policy gradients based on user feedback. The parameters of each channel's execution agent are updated using temporal differential based on the deviation between the fitness value and the actual effect, including:

[0031] For channels that have not received a delay marker, the execution agent will match the content items with the user profile to generate personalized push content and send it to the user's terminal, recording the push time, content identifier and fit value in the tracking database;

[0032] Collect behavioral trajectory data generated by user terminals, integrate the behavioral trajectory data into reward values ​​through a reward shaping mechanism, retrieve the corresponding scheduling decision status from the tracking database based on content identifiers, combine the reward value and scheduling decision status to construct a feedback loop and transmit it to the global scheduling agent;

[0033] The global scheduling agent analyzes the feedback loop to extract the scheduling decision state and reward value to construct state-reward pairs. It then performs discount cumulative reward calculation on the state-reward pair sequence to construct the objective function. Finally, it updates the decision network parameters of the global scheduling agent through backpropagation of the objective function.

[0034] The channel execution agent retrieves the fit value from the tracking database based on the content identifier, maps it through the value network, outputs the value estimate of the current state as the prediction value, constructs a temporal difference target with the prediction value and the reward value, and updates the value network parameters of the channel execution agent through the temporal difference target.

[0035] A second aspect of this invention provides an AI agent-driven digital media content distribution optimization system, comprising:

[0036] The data acquisition unit is used to collect interactive behavior data of target users across multiple distribution channels and extract user behavior sequences and cross-channel migration paths.

[0037] The state encoding unit is used to input the user behavior sequence into the recurrent neural encoder to generate a state latent vector, and input the cross-channel migration path into the graph convolutional encoder to generate a path latent vector. The two are mapped to a comprehensive user state representation through a fully connected layer.

[0038] The scheduling and allocation unit is used to set up a global scheduling agent and multiple channel execution agents. The global scheduling agent receives the user's comprehensive state representation and outputs a channel allocation probability distribution. It samples from the allocation probability distribution to generate the target channel identifier for each content item and transmits it to the corresponding channel execution agent.

[0039] The channel execution unit is used for each channel execution agent to read the local real-time load status encoding into a status vector, calculate the vector dot product with the content features to obtain the fit degree, and when the fit degree is lower than the fit threshold, generate a delay mark and send it back to the global scheduling agent, and remove the corresponding channel from the allocation probability distribution and resample.

[0040] The parameter update unit is used for channel execution agents that have not received a delay flag to push content. It updates the global scheduling agent parameters through policy gradient using user feedback, and updates the channel execution agent parameters through time difference using the deviation between the fitness value and the actual effect.

[0041] A third aspect of the present invention provides an electronic device, comprising:

[0042] processor;

[0043] Memory used to store processor-executable instructions;

[0044] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0045] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0046] This method enables multi-channel, cross-domain acquisition and deep characterization of user interaction behavior, shifting digital media content distribution from static rule-driven to dynamic data-driven approaches, significantly improving the accuracy of user state understanding. The joint modeling of a recurrent neural encoder and a graph convolutional encoder fully captures the temporal dependencies of behavioral sequences and the structural relationships of cross-channel migration paths, generating highly personalized comprehensive user state representations. This supports subsequent decision-making, achieving precise matching from both macro-preference and micro-path dimensions.

[0047] The two-layer collaborative architecture of the global scheduling agent and the channel execution agent decomposes the distribution decision into two levels: macro-level channel allocation and micro-level execution adaptation. The global scheduling agent generates the channel allocation probability distribution in real time based on the user's state, avoiding the rigidity of preset rules; the channel execution agent dynamically evaluates the adaptability based on the vector dot product of local load and content features, realizing adaptive coordination between push timing and channel resources.

[0048] The fit threshold and delayed tagging feedback mechanism form a resilient error correction loop. When the channel execution agent detects insufficient fit, it proactively intercepts push notifications and triggers global scheduling reassignment, effectively avoiding content mismatch caused by channel state fluctuations. A hybrid parameter update framework combining policy gradients and temporal differences enables the global scheduling agent and each channel execution agent to independently optimize based on real user feedback. Global scheduling iteratively improves the long-term returns of channel allocation strategies through reward signals, while each channel execution agent uses the temporal difference between fit deviation and actual effect deviation to correct its own decision boundaries. This allows the overall distribution system to quickly converge to the optimal strategy space through continuous interaction, resulting in a synergistic improvement in content reach efficiency and user retention.

[0049] Furthermore, the method and system described in this invention can be implemented in a microservice architecture in high-concurrency real-time content distribution scenarios. Each intelligent agent module is deployed and scaled independently. With the help of low-pause garbage collection, distributed high-availability counting service and asynchronous message bus, it can stably support large-scale concurrent content distribution decisions under millisecond-level response constraints, and has good engineering feasibility and horizontal scalability. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the AI-driven digital media content distribution optimization method according to an embodiment of the present invention.

[0051] Figure 2 A flowchart illustrating the method for constructing a comprehensive user state representation in an embodiment of the present invention;

[0052] Figure 3 The distribution of activation values ​​for the hidden vectors of the path;

[0053] Figure 4 Distribution map for compatibility assessment of each distribution channel. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0056] Figure 1 This is a flowchart illustrating the AI ​​agent-driven digital media content distribution optimization method according to an embodiment of the present invention. The AI ​​agent-driven digital media content distribution optimization method provided by the present invention includes:

[0057] Collect interactive behavior data of target users across multiple distribution channels, and extract user behavior sequences and cross-channel migration paths;

[0058] The user behavior sequence is input into a recurrent neural encoder to generate a state latent vector, and the cross-channel migration path is input into a graph convolutional encoder to generate a path latent vector. The two are then mapped through a fully connected layer to form a comprehensive user state representation.

[0059] A global scheduling agent and multiple channel execution agents are set up. The global scheduling agent receives the user's comprehensive state representation and outputs a channel allocation probability distribution. It samples from the allocation probability distribution to generate the target channel identifier for each content item and passes it to the corresponding channel execution agent.

[0060] Each channel execution agent reads the local real-time load status encoding into a status vector, calculates the vector dot product with the content features to obtain the fit degree, and when the fit degree is lower than the fit threshold, generates a delay mark and sends it back to the global scheduling agent, and removes the corresponding channel from the allocation probability distribution and resamples.

[0061] For channels that have not received a delay flag, the execution agent will push content. The global scheduling agent parameters will be updated using policy gradients based on user feedback. The parameters of each channel's execution agent will be updated using time-series differentials based on the deviation between the fit value and the actual effect.

[0062] In a specific engineering implementation, the aforementioned AI-driven digital media content distribution optimization method and system can be deployed as a microservice-based real-time content distribution platform. This platform is divided into several independently deployable and scalable service units, including a traffic adaptation module, a status coding module, a global scheduling module, a channel execution module, a monitoring and feedback module, and a counting service module. Specifically, the traffic adaptation module is responsible for uniformly converting heterogeneous access protocols from different media channels into standardized content distribution requests; the status coding module corresponds to the status coding unit; the global scheduling module corresponds to the global scheduling agent; each channel execution module corresponds to the channel execution agent; the monitoring and feedback module is responsible for collecting user interaction feedback; and the counting service module provides real-time load and frequency status to the channel execution agents. The modules communicate with each other through a remote procedure call framework and a distributed message bus. The remote procedure call framework supports direct connection mode and cluster mode based on registry service discovery, thereby ensuring low latency while supporting dynamic scaling of the channel execution agents.

[0063] In a preferred embodiment, the access service for media channels adopts an asynchronous event-driven high-performance network framework (e.g., a non-blocking HTTP service based on Netty) to handle high-concurrency requests, and the service process uses a low-pause garbage collector (e.g., ZGC) to meet millisecond-level response constraints; the operating metrics of each service are collected by Micrometer and independently exposed through the Prometheus endpoint to observe the load level and decision quality of each agent module in real time, providing observable support for global scheduling and adaptive adjustment of channel execution.

[0064] Figure 2 This is a flowchart illustrating the method for constructing a comprehensive user state representation according to an embodiment of the present invention. The user behavior sequence is input into a recurrent neural encoder to generate a state latent vector, and the cross-channel migration path is input into a graph convolutional encoder to generate a path latent vector. Both are mapped through a fully connected layer to form a comprehensive user state representation, including:

[0065] The user behavior sequence is encoded by behavior type and time interval. The two types of encoding are concatenated and input into a recurrent neural encoder. The recurrent neural encoder processes the encoding at each time step by step and updates the internal memory state. At the end of the sequence, the internal memory state is extracted as a state latent vector.

[0066] The access frequency and dwell time of each channel in the cross-channel migration path are extracted as node attributes, and the number of migrations and time intervals between adjacent channels are extracted as edge attributes. The graph convolutional encoder constructs a feature propagation matrix based on the node attributes and the edge attributes and performs layer-by-layer convolution operation. The central node features of the final convolutional layer are concatenated with the mean features of the whole graph to generate the path latent vector.

[0067] The mutual information between the state latent vector and the path latent vector is calculated to quantify their information redundancy. An adaptive fusion strategy is designed based on the information redundancy. The adaptive fusion strategy controls the contribution ratio of the two at the input of the fully connected layer through learnable parameters. The fully connected layer outputs a comprehensive user state representation and retains the complementary information of temporal features and graph structure features.

[0068] Encoding user behavior sequences requires capturing both semantic and temporal information related to the behavior type. Behavior type encoding maps each behavior category (e.g., click, favorite, share, purchase) to a dense vector of fixed dimensions using an embedding matrix. Let the behavior type embedding vector be denoted as... Time interval encoding involves performing a logarithmic transformation on the time difference between two adjacent actions, followed by linear projection to obtain a time embedding vector. The purpose of logarithmic transformation is to compress the time difference values ​​of long-tailed distributions, making the encoding more stable. The behavior type embedding vector and the time interval embedding vector at the same time moment are concatenated along the feature dimension to obtain the joint input vector at that time moment. ,in This represents the time index in the sequence. This concatenation method preserves the independence of the two types of information, avoiding premature fusion during the encoding stage that could lead to semantic confusion.

[0069] The concatenated joint input vector sequence is fed into the recurrent neural encoder in a temporal sequence. The encoder at each time step... Receive current input Compared with the internal memory state of the previous moment Update the current memory state through a gating mechanism. The forget gate in the gating mechanism is responsible for filtering out information from historical memory that needs to be retained, the input gate controls the degree to which the current input writes the memory state, and the output gate determines the proportion of the current memory state output. This design enables the encoder to effectively alleviate the gradient vanishing problem when processing long sequences and has a strong modeling ability for both long-term user behavior preferences and short-term interest drift. After the sequence is processed, the internal memory state at the final moment is extracted at the end of the sequence. ( (total sequence length) as the hidden state vector This vector encapsulates the temporal evolution information of the user across the complete sequence of behaviors.

[0070] For modeling cross-channel migration paths, each channel is abstracted as a node in a graph, and the migration relationships between channels are abstracted as directed edges, constructing a user channel migration graph. Node attributes consist of two dimensions: the user's access frequency on that channel (i.e., the ratio of visits to total visits) and dwell time (i.e., the cumulative dwell time on that channel). Edge attributes also consist of two dimensions: the number of migrations between adjacent channels and the average time interval between migrations. When constructing the feature propagation matrix, the graph convolutional encoder normalizes the number of migrations in the edge attributes and uses it as the edge weight, while introducing the reciprocal of the time interval as a decay factor into the propagation matrix, giving higher weight to recently occurring channel migrations in information aggregation. In the layer-by-layer convolutional operation, each node aggregates the features of its neighboring nodes and combines them with its own features for linear transformation and non-linear activation. After multiple layers are stacked, the node features are gradually integrated into a larger range of graph structure information. In the final convolutional layer, the feature vector of the central node (i.e., the node corresponding to the currently active channel) is extracted. Simultaneously, the mean feature vector of the entire graph is obtained by averaging the features of all nodes. The two are concatenated to obtain the path latent vector. The central node features characterize the user's local behavior pattern in the current channel, while the global mean features reflect the global distribution of the user's cross-channel behavior. The combination of the two preserves graph structure information at both the local and global granularities.

[0071] Obtaining the hidden state vector With path latent vector Next, the mutual information between the two is calculated to quantify the degree of information redundancy. Mutual information Mutual information reflects the degree of overlap in the information carried by two vectors: high mutual information indicates a large amount of overlapping information between the two types of codes, and the contribution ratio of one should be reduced during fusion; low mutual information indicates strong complementarity between the two types of codes, and they can be fused more evenly. Since directly calculating the mutual information of high-dimensional continuous vectors is computationally complex, a neural network-based mutual information estimation method is adopted, which approximates the mutual information by training a small discriminant network. The lower bound is denoted by the estimated value. .

[0072] Adaptive fusion strategy based on Design learnable contribution weights. Specifically, introduce two learnable scalar parameters. and The contribution ratios of the state latent vector and the path latent vector at the input of the fully connected layer are controlled respectively, and constraints are applied to make... , , During the training process, As a regularization signal, it participates in the construction of the loss function: when When the redundancy threshold is exceeded, a penalty term is introduced into the loss function to prompt the model to adjust. and The ratio allocates more weight to the side with more independent information; when When the value is below the threshold, the penalty term is not activated. and It can be freely optimized under the guidance of the task objective. The fused input vector is represented as follows: ,in This indicates a weighted concatenation operation, which involves multiplying each vector by its corresponding weight before concatenating the dimensions, rather than simply summing them by weights. This allows for a controlled fusion while preserving the spatial structure of each vector's features.

[0073] Weighted concatenation vector After being input into the fully connected layer, the data undergoes several linear transformations and activation function processing to finally output the user's comprehensive state representation. The fully connected layers are designed using a layer-by-layer dimensionality reduction approach, with batch normalization and Dropout operations applied between each layer. Batch normalization stabilizes the distribution of inputs across layers, while Dropout prevents overfitting and enhances the model's generalization ability. Because... Mid-time features (from) ) and graph structure features (from The two types of heterogeneous features coexist in a concatenated manner, requiring the fully connected layer to process both types of features simultaneously during the mapping process. To preserve the complementary information of temporal features and graph structure features, the output dimension of the fully connected layer is designed to be no less than half the sum of the dimensions of the two types of latent vectors, ensuring that excessive dimensionality compression during information compression does not lead to significant loss of one type of feature. The final user comprehensive state representation is obtained as follows. It also contains the temporal evolution of user behavior sequences and graph topology information of cross-channel migration, providing high-quality input features for the subsequent channel allocation decision of the global scheduling agent.

[0074] The graph convolutional encoder constructs a feature propagation matrix based on the node attributes and edge attributes and performs layer-by-layer convolution operations. It then concatenates the center node features of the final convolutional layer with the mean features of the entire graph to generate a path latent vector, including:

[0075] Extract the time interval between adjacent channels from the edge attributes and convert it into a time decay factor. Convert the node attributes into node initial embedding vectors. Construct a feature propagation matrix based on the time decay factor and the node initial embedding vectors.

[0076] The graph convolutional encoder performs matrix operations on the feature propagation matrix and the initial node embedding vector to obtain the first layer of aggregated features. The first layer of aggregated features and the initial node embedding vector are fused together after being transformed by a learnable weight matrix to generate the first layer of output features. The first layer of output features are then further processed with the feature propagation matrix to generate the second layer of output features.

[0077] The output features of the first layer and the output features of the second layer are mapped to a unified dimension through a projection layer and then concatenated. The concatenated features are input into a fully connected layer to generate fused features. The weight parameters of the fully connected layer are initialized according to the channel switching frequency and migration path length of the target user.

[0078] The feature vector of the target user's current channel location is extracted from the fusion features and used as the central node feature. The feature vectors of all channel locations in the fusion features are weighted and summed according to the historical access frequency to obtain the mean feature of the whole graph. This mean feature is then concatenated with the central node feature to obtain the path latent vector.

[0079] In the graph structure encoding process of cross-channel migration paths, the graph convolutional encoder first preprocesses the node attributes and edge attributes of the graph. Node attributes typically include multi-dimensional discrete and continuous features such as channel type identifier, historical channel access count, and average user dwell time on that channel. A linear mapping layer uniformly maps these multi-dimensional raw features to a fixed-dimensional dense vector space, obtaining the initial embedding vector for each channel node. The most crucial information in the edge attributes is the time interval between adjacent channels, i.e., the time difference experienced by a user switching from one channel to another. A time decay factor is introduced into this time interval to characterize the gradually fading influence of inter-channel migration behavior over time. Specifically, if a channel node... arrive The time interval between them is The corresponding time decay factor Through the exponential decay function The calculation yielded, where The decay rate hyperparameter controls the impact of time intervals on edge weights. Shorter time intervals result in a decay factor closer to 1, indicating a greater contribution of the channel migration behavior to the current state; longer time intervals result in a decay factor closer to 0, indicating that the impact of the migration behavior has largely dissipated.

[0080] Based on the aforementioned time decay factor and initial node embedding vector, a feature propagation matrix is ​​constructed. The feature propagation matrix is ​​based on the original graph adjacency matrix, with a time decay factor. The original binary adjacency relationships are replaced, and self-loops are added to preserve node information. The matrix is ​​then normalized to ensure numerical stability during feature aggregation. Symmetric normalization is used, i.e. ,in This is a weighted adjacency matrix with weights based on the time decay factor. It is the identity matrix. This represents the corresponding degree matrix. This normalization operation ensures that nodes of different degrees have similar numerical magnitudes when aggregating neighborhood information, preventing nodes with higher degrees from dominating the feature propagation direction.

[0081] In the layer-by-layer operation of graph convolution, the feature propagation matrix is... With the initial embedding matrix of the nodes Perform matrix multiplication to obtain the first layer of aggregated features. This aggregated feature reflects the weighted information gathered by each node from its first-order neighborhood. The first-layer aggregated feature is then compared with the initial node embedding vector. Each through a learnable weight matrix and After transformation, the features are added and fused, then processed by a nonlinear activation function to generate the first layer of output features. ,Right now ,in The activation function is ReLU. By introducing a feature-preserving term in the form of residual connections, excessive dilution of the original node information during graph convolution operations is prevented. The output features are then used in the first layer. Based on this, continue with the feature propagation matrix Perform matrix operations and pass through a second-layer learnable weight matrix. Transformation and activation to generate second-layer output features This feature integrates second-order neighborhood information of each node, enabling it to capture longer-range inter-channel dependencies.

[0082] Because the first-layer output features and the second-layer output features are at different semantic levels, direct concatenation may lead to dimensionality mismatch or excessively large differences in feature scale. Therefore, we will separately... and Each independent projection layer maps to a unified target dimension. Then, the features are concatenated along the feature dimensions to form a multi-scale fused node feature matrix. The concatenated feature matrix is ​​fed into a fully connected layer, where it undergoes linear transformation and nonlinear activation to generate a fused feature matrix. The weights of this fully connected layer are not randomly initialized, but rather selectively initialized based on the target user's channel switching frequency and migration path length. Users with higher channel switching frequencies have more complex channel dependencies in their migration paths, and the corresponding fully connected layer weights are assigned larger initial values ​​to enhance their ability to fit complex path structures. Conversely, longer migration path lengths result in a narrower distribution range of weights during initialization to avoid gradient explosion risks on long paths. This initialization strategy allows the graph convolutional encoder to converge more quickly to a parameter space that matches user behavior patterns during the early stages of training.

[0083] From the fusion feature matrix Extract the row vector corresponding to the current channel node of the target user, and use it as the feature of the central node. This vector centrally reflects the local contextual information of the user's current channel location and the influence of surrounding channels. For the overall graph mean feature... The calculation does not use a simple arithmetic average, but instead introduces historical visit frequency as a weighting coefficient. Specifically, let the graph have a total of The first channel node, the first The historical access frequency of each channel node is The weights are obtained by normalizing the access frequency of all nodes. Then, the fused feature vectors corresponding to each node are summed according to normalized weights to obtain... Channel nodes with high historical access frequency contribute more to the overall graph mean feature. This design allows the overall graph mean feature to highlight the structural information of user preference channels more, rather than treating all channels equally.

[0084] Features of the central node Features of the whole graph mean By concatenating along the feature dimensions, the path latent vector is obtained. . It simultaneously incorporates both fine-grained local features of the user's current channel location and global channel preference distribution information, with the two complementing each other: the former captures the details of the channel state at the current moment, while the latter provides the user's macroscopic behavioral patterns across the entire channel graph. This concatenation design, along with the subsequent mapping of the state latent vector in the fully connected layer, is crucial. With path latent vector The fusion process is consistent to ensure that the path latent vector has sufficient information density to collaboratively represent the user's complete state with the output of the behavior sequence encoder.

[0085] like Figure 3 As shown in the figure, this diagram illustrates the hidden vectors of the path. The distribution of activation values ​​across 128 dimensions shows sparse fluctuations in the range of -0.6 to 0.9, with the vast majority of dimensions concentrated in a stable base range of -0.1 to 0.4. This reflects the general statistical characteristics of a large number of nodes and edges in the cross-channel migration graph. The significant positive peaks (peak values ​​exceeding 0.7) appearing at index positions approximately 20, 55, 78, and 102, and the negative dips (below -0.4) appearing at indices 35 and 90, correspond to the direct outputs of two key operations: "extracting the feature vector of the target user's current channel location from the fusion features" and "obtaining the mean feature of the entire graph by weighted summation based on historical access frequency." Because... It retains details of users' partial behavioral patterns on current channels. The global channel preference distribution is condensed, and the concatenated path latent vector has both immediate sensitivity to the current channel status and incorporates global statistical constraints on cross-channel behavior. Thus, when it is fused with the state latent vector in the future, it can effectively retain the complementary information of graph structure features and temporal features, providing a structured feature input basis for "initializing the weight parameters of the fully connected layer according to the channel switching frequency and migration path length of the target user".

[0086] In actual engineering deployments, the number of nodes in the cross-channel migration path graph changes dynamically with the amount of user historical data. To ensure the computational efficiency of graph convolution operations, an upper limit is set on the number of nodes in the graph. Any excess nodes are truncated from the oldest to the most recent based on historical access time, retaining the nodes corresponding to the most recent channel access records. This maintains the timeliness and representativeness of the graph structure under limited computing resources.

[0087] In one specific implementation, the state encoding unit and the subsequent networks used for value assessment, bidding, and allocation decisions can be implemented by an independent algorithm inference service. This algorithm inference service does not rely on a specific web container, runs as an independent process, and provides an inference interface externally through a remote procedure call framework. Internally, in addition to the recurrent neural encoder and graph convolutional encoder, it can also integrate gradient boosting tree models (e.g., XGBoost, LightGBM, CatBoost) and linear models as performance prediction and value estimation components, and can use a Bid Landscape (BLF) model to model the expected returns of channel allocation and dynamic bidding. The above models and their corresponding policy parameters are stored in a relational database with the policy identifier as the primary key, supporting hot loading and canary updates without restarting the service, thereby enabling online iteration of the parameters of the global scheduling agent decision network and the channel execution agent value network.

[0088] A global scheduling agent and multiple channel execution agents are configured. The global scheduling agent receives the user's comprehensive state representation and outputs a channel allocation probability distribution. It then samples from this probability distribution to generate target channel identifiers for each content item and transmits them to the corresponding channel execution agents. This includes:

[0089] The global scheduling agent maintains an experience memory bank of historical allocation decisions, retrieves historical samples similar to the user's comprehensive state representation from the experience memory bank, and extracts the successful allocation records corresponding to the historical samples as prior knowledge. The user's comprehensive state representation and prior knowledge are then fused and input into the decision network to generate an initial probability vector.

[0090] The load feature matrix is ​​constructed by reading the resource utilization rate and the number of pending tasks of each channel. The load feature matrix is ​​interactively calculated with the initial probability vector through a gating mechanism, and the probability value of each channel is dynamically adjusted. The channel allocation probability distribution after load perception adjustment is output.

[0091] Entropy value detection is performed on the channel allocation probability distribution to evaluate the concentration of the channel allocation probability distribution. When the entropy value exceeds the upper limit, sharpening is performed, and when the entropy value is below the lower limit, smoothing is performed. The processed channel allocation probability distribution is used as the sampling basis.

[0092] Each content item is independently sampled from the processed channel allocation probability distribution to generate a target channel identifier. The target channel identifier is encoded as routing information and a priority mark is attached. It is then broadcast to all channel execution agents through a publish-subscribe model. Each channel execution agent selects tasks belonging to its own channel based on the routing information and triggers their execution.

[0093] The global scheduling agent continuously maintains an experience memory during operation. This memory stores historical allocation decision records in key-value pairs. Each record contains a comprehensive user state representation at that time, the channel allocation scheme executed, and the final allocation performance score. When a new comprehensive user state representation... Upon arrival, with For the query vector, perform nearest neighbor search in the memory and calculate... The cosine similarity between the data and the state representations of historical samples in the database is used to select the data with the highest similarity. This record is used as a candidate historical sample. For this... The candidate samples are weighted and summed according to their effect scores to obtain the prior knowledge vector. The weights are determined by the product of normalized similarity and effect score, thus ensuring that the retrieved prior knowledge reflects both state similarity and is biased towards high-quality historical decisions. and The concatenated data is input into a decision network, which consists of a multi-layer fully connected structure. The last layer is connected to a softmax activation function, and the output is the initial probability vector for each channel. The vector dimension equals the total number of available channels. , The Middle The first component represents the allocation of content to the first component without considering real-time load. The initial tendency probability of each channel.

[0094] Read the current resource utilization rate and number of pending tasks for each channel, and construct a load feature matrix by concatenating the two metrics. Its number of rows is Each row corresponds to a real-time load status description vector for a channel. The specific implementation of the gating mechanism is as follows: The gating coefficient vector is obtained after linear transformation. And then Applying the sigmoid function maps each component to... Intervals, using element-wise multiplication With the initial probability vector Element-wise multiplication yields the intermediate adjustment vector. . The smaller the median component, the higher the load on the corresponding channel, and the higher the probability that the channel will be assigned. The corresponding price was lowered. The larger the median weight, the more abundant the corresponding channel resources, and the higher the allocation probability. After renormalization, the channel allocation probability distribution after load perception adjustment is obtained. This distribution retains prior knowledge guidance while dynamically responding to the real-time status of the channels, avoiding congestion caused by pushing a large amount of content to high-load channels.

[0095] right Entropy detection is performed to assess its concentration level. Calculation Information entropy Its expression is ,in for The Middle The probability value corresponding to each channel. When Exceeding the preset limit When the probability distribution is too uniform, the difference in the likelihood of each channel being selected is too small, indicating a lack of effective channel preference, then sharpening is performed: using a temperature coefficient... (Values ​​less than 1) After exponentiation of each component, normalization is performed to concentrate the probability quality towards higher probability channels. Below the preset lower limit This indicates that the probability distribution is too concentrated, with almost all content being allocated to a single channel, resulting in poor load balancing. In this case, smoothing processing is performed: using a temperature coefficient... (Value greater than 1) After exponentiation of each component, the distribution is renormalized to flatten the probability distribution, providing more channels with a chance of being selected. The final channel allocation probability distribution is then output after entropy detection and processing. This serves as the basis for subsequent sampling. This mechanism effectively prevents the allocation strategy from falling into two extreme states—excessive randomness leading to underutilization of high-quality channels, and excessive determinism leading to single-point overload.

[0096] based on For each content item to be distributed, perform a polynomial sampling independently to obtain the target channel identifier corresponding to that content item. Independent sampling ensures that channel selection is decoupled between different content items, preventing a batch of content from flooding into the same channel at the same time. After sampling, The code is encoded as a routing information structure, which includes a unique identifier for the content item, a target channel identifier, a task priority flag, and a timestamp. The priority flag is calculated based on a priority score derived from the timeliness of the content item, user activity, and historical click-through rate. and will The routing information structure is discretized into high, medium, and low priority levels, and high-priority content will be processed first on the channel execution agent side.

[0097] The broadcasting of routing information structures is implemented using a publish-subscribe pattern. The global scheduling agent acts as the publisher, publishing the routing information structures of all content items in batches to a unified topic channel on the message bus. Each channel execution agent acts as a subscriber, continuously listening to this topic channel. Upon receiving a message, it matches the target channel identifier in the routing information structure with its own channel identifier, retaining only tasks whose target channel identifier matches its own, and discarding the rest. This design eliminates the need for the global scheduling agent to maintain address mappings for each channel execution agent, reducing the coupling between the scheduling layer and the execution layer. Furthermore, the publish-subscribe pattern naturally supports dynamic scaling of channel execution agents; adding a new channel execution agent only requires subscribing to the same topic to join the system, without modifying any logic of the global scheduling agent.

[0098] After each channel's execution agent completes task screening, it sorts the local task queue according to priority tags, with high-priority tasks placed at the front of the queue to trigger the content push execution process first. During execution, if the channel's execution agent detects that the local real-time load has exceeded the capacity threshold, it generates a delay tag for the current task and sends it back to the global scheduling agent. Upon receiving the delay tag, the global scheduling agent then... The probability component corresponding to the affected channel is removed and renormalized. Independent sampling is then performed again to reallocate the target channel for the affected content items, thereby achieving dynamic self-healing of the allocation decision. The entire scheduling process works collaboratively through three levels: prior fusion driven by experience memory, gating adjustment based on load awareness, and probability correction constrained by entropy value. This ensures that the channel allocation probability distribution remains reasonable across three dimensions: historical experience guidance, real-time status awareness, and distribution pattern constraints. Finally, a publish-subscribe broadcast mechanism is used to efficiently complete multi-channel task routing.

[0099] In one specific implementation, the experience memory maintained by the global scheduling agent can be hosted by a high-performance cache (e.g., Kvrocks) compatible with the Redis protocol to support low-latency nearest neighbor retrieval of massive historical allocation decision records. The publish-subscribe broadcast mechanism can be implemented collaboratively by a distributed message queue (e.g., Kafka) and the remote procedure call framework. The global scheduling agent publishes the routing information structure to a unified topic, and each channel execution agent, as a subscriber, filters tasks belonging to its own channel according to the target channel identifier. Channel execution agents can dynamically register and remove themselves by relying on the service discovery capabilities of the registry center. New channel execution agents can join the system by subscribing to the same topic without modifying any logic of the global scheduling module, thereby achieving smooth scaling up and down when business traffic fluctuates.

[0100] Each channel's execution agent reads the local real-time load status encoding into a state vector, calculates the vector dot product with the content features to obtain the fit degree, and when the fit degree is lower than the fit threshold, generates a delay label and sends it back to the global scheduling agent, and removes the corresponding channel from the allocated probability distribution and resamples, including:

[0101] Each channel execution agent constructs a hierarchical representation structure of local load status, using real-time load indicators as the bottom layer feature, load change rate as the middle layer feature, and load saturation prediction as the top layer feature. The hierarchical representation is fused and encoded into a state vector of a unified dimension through a multilayer perceptron.

[0102] Extract the types of computing resources required for processing and the expected duration of processing from the content items to construct a content feature vector. Perform a dot product operation between the state vector and the content feature vector to obtain the fit degree and calculate the confidence interval of the fit degree. When the lower bound of the fit degree is lower than the fit threshold, it is judged as a high-risk misfit.

[0103] The channel execution agent generates a delay tag and attaches a recommended alternative channel list. The recommended alternative channel list is generated based on the load coordination relationship between channels and the historical transfer success rate. The delay tag and the recommended alternative channel list are sent back to the global scheduling agent through a priority channel.

[0104] The global scheduling agent first selects candidate channels from the recommended alternative channel list, enhances the probability values ​​of the candidate channels in the allocation probability distribution, samples from the enhanced allocation probability distribution to generate the final alternative channel identifier, and constructs a scheduling instruction containing a retry limit to send to the channel execution agent corresponding to the final alternative channel identifier.

[0105] Upon receiving content allocation instructions from the global scheduling agent, each channel execution agent first performs hierarchical modeling of its local real-time load status. The hierarchical representation structure consists of three levels: the bottom layer features are composed of immediate load indicators, including directly sampleable real-time metrics such as current CPU utilization, memory usage, and bandwidth consumption; the middle layer features are composed of load change rates, which statistically analyze the first-order differences of the bottom-level immediate indicators within a recent time window, reflecting the dynamic trend of load increase or decrease; the top layer features are composed of load saturation prediction values, which, based on the current load level and its changing trend, predict whether the channel will reach its processing capacity limit within a short future time window, outputting a saturation probability estimate. The three layers of features are flattened and concatenated, then input into a multilayer perceptron (MLP) for fusion encoding. The MLP's output layer is set to a fixed dimension, thereby compressing the heterogeneous, multi-granular hierarchical load information into a unified-dimensional state vector. This is used for subsequent dot product calculations.

[0106] The construction of content feature vectors focuses on the resource requirements of content items for distribution channels. Two key attributes are extracted from the metadata and processing flow descriptions of the content items: computational resource type (e.g., whether specific processing capabilities such as video transcoding, image compression, and text indexing are required) and expected processing time (i.e., the estimated processing time required to complete the distribution of the content item). The computational resource type is encoded as a multi-hot vector, and the expected processing time is normalized to a continuous value. The two are concatenated and mapped to the state vector via a linear projection layer. The same dimensions yield content feature vectors. Execution state vector With content feature vector The vector dot product operation yields the fitness scalar. ,Right now Adaptability The higher the value, the better the match between the current channel load status and the resource requirements of the content items.

[0107] like Figure 4 As shown in the figure, the adaptation evaluation mechanism is calculated across five core distribution channels. The adaptation values ​​for each channel (Email 0.81, App 0.72, Video 0.55, Web 0.48, Social Media 0.23) are the direct output of the dot product operation. The adaptation threshold marked by the red dashed line is 0.65, corresponding to the judgment rule that "when the lower bound of the adaptation is lower than the adaptation threshold, it is judged as a high-risk mismatch". Specifically, the lower bound of the adaptation for the social media channel (0.23) and the Web channel (0.48) (after combining confidence interval estimation) is much lower than the threshold, triggering the process of "generating a delay label and sending it back to the global scheduling agent"; while the Email and App channels are higher than the threshold and can execute push normally. This distribution chart verifies the necessity of "calculating the confidence interval of fit"—if judged solely by the mean of a single dot product, the difference between the video channel (0.55) and the web channel (0.48) might be misjudged. However, by combining the lower bound value, both are included in the candidate range for delay labeling, effectively avoiding the risk of misdistribution due to instantaneous load fluctuations. After the delay label is returned, the global scheduling agent "prioritizes the selection of candidate channels from the recommended alternative channel list and enhances the probability values ​​of candidate channels in the allocation probability distribution," thereby achieving dynamic redistribution from unsuitable channels to suitable channels, ensuring the robustness and accuracy of content distribution under real-time load changes.

[0108] Judging fitness solely based on a single dot product scalar is uncertain; therefore, a confidence interval estimation mechanism is introduced. For the state vector... With content feature vector The mean fitness of the dot product results under multiple sampling or perturbation conditions is calculated by statistical analysis. with standard deviation This allows us to determine the lower bound of the confidence interval. ,in This is the confidence coefficient, and its value is set according to the confidence level (e.g., 95% confidence level corresponds to...). ). Current boundary value Below the preset adaptation threshold At that time, it was determined that the current channel had a high risk of mismatch with the content item, even if the average value was low. If the value exceeds the threshold, it may trigger the subsequent delayed tag generation process. This conservative judgment strategy effectively avoids the risk of distribution failure due to load fluctuations.

[0109] After a channel is identified as high-risk and incompatible, the channel's intelligent agent generates a delay marker and simultaneously constructs a recommended list of alternative channels, which is then appended to the delay marker. The generation of the alternative channel list is based on two types of information: the load synergy between channels and historical transfer success rates. Load synergy describes the complementarity of load levels between different channels within the same time period. If channel A is under high load while channel B is typically under low load, then the two have strong load synergy and can be considered priority alternative candidates. Historical transfer success rates record the proportion of content successfully distributed after being transferred from the current channel to various candidate channels in the past; channels with higher success rates rank higher in the list. Let the current channel be... The set of candidate alternative channels is For each candidate channel in the set Calculate the overall recommendation score ,in For load coordination score, To improve the success rate of historical transfers, Hyperparameters to balance the contributions of both. The top few channels, sorted in descending order, form a recommended alternative channel list. Delay flags and the alternative channel list are transmitted back via a priority channel, which enjoys higher scheduling priority in the message queue, ensuring the global scheduling agent can respond promptly and avoiding accumulated content distribution delays due to excessive waiting time.

[0110] After receiving the delay flag, the global scheduling agent selects candidate channels from the attached list of recommended alternative channels, rather than directly resampling from the entire channel space. This involves considering the probability distribution of candidate channels in the current allocation. The corresponding probability value is then enhanced by multiplying the probability value of the candidate channel by an enhancement coefficient. ( At the same time, previously unsuitable channels that have been removed will be... The corresponding probability value is set to zero, and then the entire probability distribution is renormalized to obtain the enhanced allocation probability distribution. .from Mid-sampling generates the final alternative channel identifier .

[0111] When generating scheduling instructions, a scheduling instruction structure containing a retry limit is constructed. This structure includes a content item identifier and a final alternative channel identifier. Maximum number of retries allowed And the sequence number of this scheduling. Maximum number of retries. This setting prevents an infinite loop when multiple channels experience consecutive high-risk incompatibilities. Once the retries reach the limit, a fallback strategy is triggered (such as downgrading to the default channel or temporarily storing the data in the scheduling queue). Scheduling instructions are transmitted via the standard distribution channel. The corresponding channel execution agent receives the data and re-executes the suitability assessment process. The closed-loop mechanism of delayed tag return and resampling ensures the robustness and adaptive scheduling capability of content distribution under scenarios with dynamic changes in channel load. At the same time, by guiding the replacement of the channel list with recommendations, the search space for resampling is effectively shortened, and scheduling efficiency is improved.

[0112] As one specific implementation, the local real-time load status read by each channel execution agent can be provided by a highly available counting service. This counting service adopts a dual-node peer-to-peer replication architecture and persists data to disk using an embedded persistent storage engine (such as RocksDB). It maintains real-time counts and occupancy according to multiple business dimensions such as advertisers, orders, plans, and transactions. Based on this, the channel execution agent constructs a hierarchical representation of real-time load indicators, load change rate, and load saturation prediction. Combined with the above counting results, it performs frequency control to avoid over-distribution of the same content to the same user or the same channel in a short period of time. When the real-time occupancy reported by the counting service approaches the capacity threshold, the delay marker generation and alternative channel reallocation process are triggered, realizing real-time linkage between load status and distribution decisions.

[0113] For channels that have not received a delay flag, the execution agent performs content push notifications. The global scheduling agent parameters are updated using policy gradients based on user feedback. The parameters of each channel's execution agent are updated using temporal differential based on the deviation between the fitness value and the actual effect, including:

[0114] For channels that have not received a delay marker, the execution agent will match the content items with the user profile to generate personalized push content and send it to the user's terminal, recording the push time, content identifier and fit value in the tracking database;

[0115] Collect behavioral trajectory data generated by user terminals, integrate the behavioral trajectory data into reward values ​​through a reward shaping mechanism, retrieve the corresponding scheduling decision status from the tracking database based on content identifiers, combine the reward value and scheduling decision status to construct a feedback loop and transmit it to the global scheduling agent;

[0116] The global scheduling agent analyzes the feedback loop to extract the scheduling decision state and reward value to construct state-reward pairs. It then performs discount cumulative reward calculation on the state-reward pair sequence to construct the objective function. Finally, it updates the decision network parameters of the global scheduling agent through backpropagation of the objective function.

[0117] The channel execution agent retrieves the fit value from the tracking database based on the content identifier, maps it through the value network, outputs the value estimate of the current state as the prediction value, constructs a temporal difference target with the prediction value and the reward value, and updates the value network parameters of the channel execution agent through the temporal difference target.

[0118] Channel execution agents that haven't received a delay marker enter the content push execution phase after confirming that their current load status meets the adaptation requirements. During execution, the structured description of the content item is matched with the target user's profile data. The user profile covers multiple dimensions such as interest tags, historical preference weights, and active time period distribution. Through feature alignment and similarity ranking, personalized push content that best matches the user's current state is selected from the candidate content pool. After matching, the push content is packaged into a standardized message format and sent to the user's terminal. Simultaneously, the push time, content identifier, and adaptation scalar value obtained during the adaptation evaluation are written into the tracking database, forming a traceable push event record. The tracking database is indexed using the content identifier as the primary key, facilitating quick retrieval of the corresponding scheduling context information during subsequent reward calculation and parameter update phases.

[0119] After receiving push notifications, the user terminal continuously collects user behavior data, including various interaction types such as clicks, browsing duration, forwarding, saving, exiting, and explicit ratings. Different behavior types correspond to different instant reward signal strengths. Through a reward shaping mechanism, the above heterogeneous behavior data is integrated into a unified reward value. The core of the reward shaping mechanism lies in assigning differentiated reward weights to different behaviors. For example, the reward weight for deep browsing is higher than that for shallow clicking, and the reward weight for forwarding is higher than that for simply saving, thus enabling the reward signal to more accurately reflect the actual effect of content distribution. In the time dimension, the reward value also incorporates a time discount factor, appropriately attenuating delayed feedback generated within a longer time window after push notification, to balance the impact of immediate and delayed feedback on parameter updates. After reward fusion is completed, the scheduling decision state corresponding to the content item is retrieved from the tracking database based on the content identifier. The scheduling decision state and reward value are combined to construct a feedback loop, and the feedback loop data structure is transmitted to the global scheduling agent, providing it with the experiential signals needed for closed-loop learning.

[0120] After receiving the feedback loop, the global scheduling agent parses the scheduling decision state and reward value contained therein, and combines the two into a state-reward pair. ,in This represents the comprehensive user state representation that the global scheduling agent relies on when making this channel allocation decision. This corresponds to the fusion reward value. The accumulated state rewards within a certain time window are used to calculate the discounted cumulative reward for the sequence. The calculation method is to apply a discount factor to the reward value at each time step in the sequence. Perform a weighted summation, that is ,in The sequence length within the current time window. For the first The corresponding reward value for each step This is the discount factor, and its value range is... This is used to control the degree to which future rewards influence current decisions. It accumulates rewards using discounts. A policy gradient objective function is constructed, aiming to maximize the expected cumulative reward. Backpropagation is performed on the parameters of the global scheduling agent's decision network using this objective function. The gradient of each layer's parameters is calculated using the chain rule, and an adaptive learning rate optimizer is employed to update the parameters. To prevent excessively large policy update steps from causing training instability, a gradient pruning mechanism is introduced during gradient calculation, limiting the gradient norm to a preset threshold to ensure smooth convergence of the decision network parameters in each update.

[0121] The parameter updates of the channel execution agent employ a temporal difference method, performed in parallel with the policy gradient updates of the global scheduling agent. Based on the content identifier, the adaptation scalar value recorded during the push execution phase for each content item is retrieved from the tracking database and input into the value network. The value network then outputs a value estimate for the current state. The predicted value and corresponding fusion reward value Constructing a temporal difference objective The calculation method is as follows ,in For the value estimation corresponding to the state at the next time step, under the single-step temporal difference framework, if the next time step is the terminating state, then... Take zero. Timing difference error. Defined as The temporal difference error is used as the loss function of the value network. The gradients of the parameters of each layer of the value network are calculated through backpropagation and then updated. Temporal difference updates have the advantage of online learning, which can incrementally correct the parameters of the value network without waiting for the complete trajectory to end, enabling the channel execution agent to quickly respond to dynamic changes in the load environment and user behavior patterns.

[0122] To ensure consistent parameter updates between the global scheduling agent and the execution agents of each channel, an experience replay buffer is set up to store and randomly sample historical state reward pairs. This breaks the temporal correlation between samples and reduces gradient estimation bias caused by high correlation between sample sequences. Within each parameter update cycle, a fixed number of historical samples are uniformly distributed from the experience replay buffer and mixed with fresh samples within the current time window to participate in gradient calculation, balancing sample diversity and data freshness. The decision network parameters of the global scheduling agent and the value network parameters of the channel execution agents maintain independent target networks. The target network parameters are periodically updated to converge towards the main network parameters using a soft update method. By controlling the update rate of the target network to be close to zero, the stability of the target network is maintained, thus providing a relatively stable reference benchmark for the calculation of the temporal difference objective and avoiding training oscillations caused by frequent changes in the target network. Through the above-mentioned collaborative update mechanism of policy gradient and temporal difference, the entire multi-agent distribution system can achieve end-to-end adaptive optimization driven by continuous user feedback, gradually improving the accuracy of content distribution and the utilization efficiency of channel resources.

[0123] In one specific implementation, the tracking database and user feedback collection can be achieved collaboratively by independent monitoring and analysis services. The monitoring service receives interactive events such as exposures and clicks in an asynchronous event-driven manner, and writes them to a local log, a distributed message queue, and a persistent counting service in a three-way fan-out manner, balancing traceability, real-time performance, and high throughput. The analysis service, as a consumer of the message queue, reduces the event stream and completes the writing of frequency control data and the construction of retrieved data, providing a data source for the reward shaping mechanism and feedback loop. The above-mentioned monitoring and analysis link, together with the aforementioned counting service, cache, and relational database, constitutes an end-to-end closed-loop learning data path, enabling the policy gradient update of the global scheduling agent and the temporal differential update of the execution agents of each channel to be continuously performed online based on real user feedback.

[0124] A second aspect of this invention provides an AI agent-driven digital media content distribution optimization system, comprising:

[0125] The data acquisition unit is used to collect interactive behavior data of target users across multiple distribution channels and extract user behavior sequences and cross-channel migration paths.

[0126] The state encoding unit is used to input the user behavior sequence into the recurrent neural encoder to generate a state latent vector, and input the cross-channel migration path into the graph convolutional encoder to generate a path latent vector. The two are mapped to a comprehensive user state representation through a fully connected layer.

[0127] The scheduling and allocation unit is used to set up a global scheduling agent and multiple channel execution agents. The global scheduling agent receives the user's comprehensive state representation and outputs a channel allocation probability distribution. It samples from the allocation probability distribution to generate the target channel identifier for each content item and transmits it to the corresponding channel execution agent.

[0128] The channel execution unit is used for each channel execution agent to read the local real-time load status encoding into a status vector, calculate the vector dot product with the content features to obtain the fit degree, and when the fit degree is lower than the fit threshold, generate a delay mark and send it back to the global scheduling agent, and remove the corresponding channel from the allocation probability distribution and resample.

[0129] The parameter update unit is used for channel execution agents that have not received a delay flag to push content. It updates the global scheduling agent parameters through policy gradient using user feedback, and updates the channel execution agent parameters through time difference using the deviation between the fitness value and the actual effect.

[0130] A third aspect of the present invention provides an electronic device, comprising:

[0131] processor;

[0132] Memory used to store processor-executable instructions;

[0133] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0134] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0135] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An AI-driven digital media content distribution optimization method, characterized in that, include: Collect interactive behavior data of target users across multiple distribution channels, and extract user behavior sequences and cross-channel migration paths; The user behavior sequence is input into a recurrent neural encoder to generate a state latent vector, and the cross-channel migration path is input into a graph convolutional encoder to generate a path latent vector. The two are then mapped through a fully connected layer to form a comprehensive user state representation. A global scheduling agent and multiple channel execution agents are set up. The global scheduling agent receives the user's comprehensive state representation and outputs a channel allocation probability distribution. It samples from the allocation probability distribution to generate the target channel identifier for each content item and passes it to the corresponding channel execution agent. Each channel execution agent reads the local real-time load status encoding into a status vector, calculates the vector dot product with the content features to obtain the fit degree, and when the fit degree is lower than the fit threshold, generates a delay mark and sends it back to the global scheduling agent, and removes the corresponding channel from the allocation probability distribution and resamples. For channels that have not received a delay flag, the execution agent will push content. The global scheduling agent parameters will be updated using policy gradients based on user feedback. The parameters of each channel's execution agent will be updated using time-series differentials based on the deviation between the fit value and the actual effect.

2. The method according to claim 1, characterized in that, The user behavior sequence is input into a recurrent neural encoder to generate a state latent vector, and the cross-channel migration path is input into a graph convolutional encoder to generate a path latent vector. Both are then mapped through a fully connected layer to form a comprehensive user state representation, including: The user behavior sequence is encoded by behavior type and time interval. The two types of encoding are concatenated and input into a recurrent neural encoder. The recurrent neural encoder processes the encoding at each time step by step and updates the internal memory state. At the end of the sequence, the internal memory state is extracted as a state latent vector. The access frequency and dwell time of each channel in the cross-channel migration path are extracted as node attributes, and the number of migrations and time intervals between adjacent channels are extracted as edge attributes. The graph convolutional encoder constructs a feature propagation matrix based on the node attributes and the edge attributes and performs layer-by-layer convolution operation. The central node features of the final convolutional layer are concatenated with the mean features of the whole graph to generate the path latent vector. The mutual information between the state latent vector and the path latent vector is calculated to quantify their information redundancy. An adaptive fusion strategy is designed based on the information redundancy. The adaptive fusion strategy controls the contribution ratio of the two at the input of the fully connected layer through learnable parameters. The fully connected layer outputs a comprehensive user state representation and retains the complementary information of temporal features and graph structure features.

3. The method according to claim 2, characterized in that, The graph convolutional encoder constructs a feature propagation matrix based on the node attributes and edge attributes and performs layer-by-layer convolution operations. It then concatenates the center node features of the final convolutional layer with the mean features of the entire graph to generate a path latent vector, including: Extract the time interval between adjacent channels from the edge attributes and convert it into a time decay factor. Convert the node attributes into node initial embedding vectors. Construct a feature propagation matrix based on the time decay factor and the node initial embedding vectors. The graph convolutional encoder performs matrix operations on the feature propagation matrix and the initial node embedding vector to obtain the first layer of aggregated features. The first layer of aggregated features and the initial node embedding vector are fused together after being transformed by a learnable weight matrix to generate the first layer of output features. The first layer of output features are then further processed with the feature propagation matrix to generate the second layer of output features. The output features of the first layer and the output features of the second layer are mapped to a unified dimension through a projection layer and then concatenated. The concatenated features are input into a fully connected layer to generate fused features. The weight parameters of the fully connected layer are initialized according to the channel switching frequency and migration path length of the target user. The feature vector of the target user's current channel location is extracted from the fusion features and used as the central node feature. The feature vectors of all channel locations in the fusion features are weighted and summed according to the historical access frequency to obtain the mean feature of the whole graph. This mean feature is then concatenated with the central node feature to obtain the path latent vector.

4. The method according to claim 1, characterized in that, A global scheduling agent and multiple channel execution agents are configured. The global scheduling agent receives the user's comprehensive state representation and outputs a channel allocation probability distribution. It then samples from this probability distribution to generate target channel identifiers for each content item and transmits them to the corresponding channel execution agents. This includes: The global scheduling agent maintains an experience memory bank of historical allocation decisions, retrieves historical samples similar to the user's comprehensive state representation from the experience memory bank, and extracts the successful allocation records corresponding to the historical samples as prior knowledge. The user's comprehensive state representation and prior knowledge are then fused and input into the decision network to generate an initial probability vector. The load feature matrix is ​​constructed by reading the resource utilization rate and the number of pending tasks of each channel. The load feature matrix is ​​interactively calculated with the initial probability vector through a gating mechanism, and the probability value of each channel is dynamically adjusted. The channel allocation probability distribution after load perception adjustment is output. Entropy value detection is performed on the channel allocation probability distribution to evaluate the concentration of the channel allocation probability distribution. When the entropy value exceeds the upper limit, sharpening is performed, and when the entropy value is below the lower limit, smoothing is performed. The processed channel allocation probability distribution is used as the sampling basis. Each content item is independently sampled from the processed channel allocation probability distribution to generate a target channel identifier. The target channel identifier is encoded as routing information and a priority mark is attached. It is then broadcast to all channel execution agents through a publish-subscribe model. Each channel execution agent selects tasks belonging to its own channel based on the routing information and triggers their execution.

5. The method according to claim 1, characterized in that, Each channel's execution agent reads the local real-time load status encoding into a state vector, calculates the vector dot product with the content features to obtain the fit degree, and when the fit degree is lower than the fit threshold, generates a delay label and sends it back to the global scheduling agent, and removes the corresponding channel from the allocated probability distribution and resamples, including: Each channel execution agent constructs a hierarchical representation structure of local load status, using real-time load indicators as the bottom layer feature, load change rate as the middle layer feature, and load saturation prediction as the top layer feature. The hierarchical representation is fused and encoded into a state vector of a unified dimension through a multilayer perceptron. Extract the types of computing resources required for processing and the expected duration of processing from the content items to construct a content feature vector. Perform a dot product operation between the state vector and the content feature vector to obtain the fit degree and calculate the confidence interval of the fit degree. When the lower bound of the fit degree is lower than the fit threshold, it is judged as a high-risk misfit. The channel execution agent generates a delay tag and attaches a recommended alternative channel list. The recommended alternative channel list is generated based on the load coordination relationship between channels and the historical transfer success rate. The delay tag and the recommended alternative channel list are sent back to the global scheduling agent through a priority channel. The global scheduling agent first selects candidate channels from the recommended alternative channel list, enhances the probability values ​​of the candidate channels in the allocation probability distribution, samples from the enhanced allocation probability distribution to generate the final alternative channel identifier, and constructs a scheduling instruction containing a retry limit to send to the channel execution agent corresponding to the final alternative channel identifier.

6. The method according to claim 1, characterized in that, For channels that have not received a delay flag, the execution agent performs content push notifications. The global scheduling agent parameters are updated using policy gradients based on user feedback. The parameters of each channel's execution agent are updated using temporal differential based on the deviation between the fitness value and the actual effect, including: For channels that have not received a delay marker, the execution agent will match the content items with the user profile to generate personalized push content and send it to the user's terminal, recording the push time, content identifier and fit value in the tracking database; Collect behavioral trajectory data generated by user terminals, integrate the behavioral trajectory data into reward values ​​through a reward shaping mechanism, retrieve the corresponding scheduling decision status from the tracking database based on content identifiers, combine the reward value and scheduling decision status to construct a feedback loop and transmit it to the global scheduling agent; The global scheduling agent analyzes the feedback loop to extract the scheduling decision state and reward value to construct state-reward pairs. It then performs discount cumulative reward calculation on the state-reward pair sequence to construct the objective function. Finally, it updates the decision network parameters of the global scheduling agent through backpropagation of the objective function. The channel execution agent retrieves the fit value from the tracking database based on the content identifier, maps it through the value network, outputs the value estimate of the current state as the prediction value, constructs a temporal difference target with the prediction value and the reward value, and updates the value network parameters of the channel execution agent through the temporal difference target.

7. An AI-driven digital media content distribution optimization system, used to implement the method as described in any one of claims 1-6, characterized in that, include: The data acquisition unit is used to collect interactive behavior data of target users across multiple distribution channels and extract user behavior sequences and cross-channel migration paths. The state encoding unit is used to input the user behavior sequence into the recurrent neural encoder to generate a state latent vector, and input the cross-channel migration path into the graph convolutional encoder to generate a path latent vector. The two are mapped to a comprehensive user state representation through a fully connected layer. The scheduling and allocation unit is used to set up a global scheduling agent and multiple channel execution agents. The global scheduling agent receives the user's comprehensive state representation and outputs a channel allocation probability distribution. It samples from the allocation probability distribution to generate the target channel identifier for each content item and transmits it to the corresponding channel execution agent. The channel execution unit is used for each channel execution agent to read the local real-time load status encoding into a status vector, calculate the vector dot product with the content features to obtain the fit degree, and when the fit degree is lower than the fit threshold, generate a delay mark and send it back to the global scheduling agent, and remove the corresponding channel from the allocation probability distribution and resample. The parameter update unit is used for channel execution agents that have not received a delay flag to push content. It updates the global scheduling agent parameters through policy gradient using user feedback, and updates the channel execution agent parameters through time difference using the deviation between the fitness value and the actual effect.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images