Digital media content pushing method and device, and storage medium

Abstract

The embodiment of the application provides a digital media content pushing method, device and storage medium, which are applied to the scene of digital media content pushing. The digital media content pushing method provided by the embodiment of the application determines a task queue corresponding to to-be-pushed content according to a resource type and a pushing address of the to-be-pushed content; in the case that the to-be-pushed content corresponds to a first task queue, the to-be-pushed content is pushed; if the pushing fails, a resource collaborative scheduling model based on deep reinforcement learning is used to dynamically allocate resources for the to-be-pushed content, so that the to-be-pushed content retries the pushing and succeeds. The embodiment of the application classifies the to-be-pushed content according to a queue, and introduces a deep reinforcement learning model trained based on multi-dimensional features of Internet of Things, so as to dynamically schedule resources for the to-be-pushed content of different task queues, and the real-time performance and accuracy of digital media content pushing are considered in the case that resources of the Internet of Things platform are limited.

Description

Technical Field

[0001] This application relates to the field of Internet technology, specifically to a method, apparatus, and storage medium for pushing digital media content. Background Technology

[0002] An IoT platform is an integrated system that connects and manages various IoT devices, collects, processes, and analyzes the data generated by these devices, and provides device management, data storage, and application development services. Therefore, IoT platforms often handle massive numbers of terminal devices accessing and interacting with data. These devices are frequently deployed in mobile or edge environments with complex and ever-changing network conditions, resulting in limited platform resources and difficulties in ensuring network connectivity stability.

[0003] In digital media content delivery scenarios on IoT platforms, the stability of network connections is difficult to guarantee, and the end-to-end link from the device to the business server is more likely to fail, leading to delivery failures. Delivery failures trigger the IoT platform's retry mechanism. Repeated deliveries further consume the IoT platform's resources, causing invalid messages to accumulate and occupy resources needed for normal business operations, resulting in a vicious cycle of decreased overall system throughput and response latency. Summary of the Invention

[0004] This application provides a digital media content push method, apparatus, and storage medium. The content to be pushed is classified into queues and a deep reinforcement learning model trained based on multi-dimensional features of the Internet of Things is introduced to dynamically schedule resources for the content to be pushed in different task queues. This approach balances the real-time performance and accuracy of digital media content push even when the resources of the Internet of Things platform are limited.

[0005] The digital media content push method provided in this application includes:

[0006] The task queue corresponding to the content to be pushed is determined based on the resource type and push address of the content to be pushed.

[0007] If the content to be pushed corresponds to the first task queue, then the content to be pushed will be pushed.

[0008] If the push fails, a resource collaborative scheduling model based on deep reinforcement learning dynamically allocates resources to the content to be pushed so that the content to be pushed can be successfully retried. The resource collaborative scheduling model is trained based on the environmental interference characteristics, terminal device characteristics, node operation characteristics of the IoT platform and the task profile of the content to be pushed.

[0009] In some implementations, the step of training the resource collaborative scheduling model includes:

[0010] Construct a simulation environment for the Internet of Things (IoT) platform;

[0011] The environmental interference characteristics, terminal device characteristics, node operation characteristics, and task profile of the content to be pushed to the IoT platform are obtained from the historical database.

[0012] Based on the environmental interference characteristics, terminal device characteristics, node operation characteristics of the IoT platform, and the task profile of the content to be pushed, a state space matrix is ​​constructed.

[0013] The state space matrix is ​​input into the resource collaborative scheduling model to output a simulated scheduling strategy.

[0014] The simulation scheduling strategy is executed in the simulation environment of the Internet of Things platform to obtain the updated status of the simulation scheduling strategy. The updated status of the simulation scheduling strategy includes the updated values ​​of the environmental interference characteristics, terminal device characteristics, node operation characteristics and task profile of the content to be pushed by the Internet of Things platform.

[0015] Based on the updated state of the simulated scheduling strategy, the value network and policy network in the resource collaborative scheduling model are trained.

[0016] In some implementations, training the value network and policy network in the resource collaborative scheduling model based on the updated state of the simulated scheduling policy includes:

[0017] The update status of the simulated scheduling strategy is evaluated based on the target reward function to obtain the reward value of the simulated scheduling strategy.

[0018] Based on the reward value of the simulated scheduling strategy, a first loss function is determined to train the model of the strategy network;

[0019] A second loss function is determined based on the reward value of the simulated scheduling strategy and the evaluation value of the value network on the simulated scheduling strategy, in order to train the model of the value network.

[0020] In some implementations, if the push fails, a resource collaborative scheduling model based on deep reinforcement learning is used to dynamically allocate resources to the content to be pushed, so that the content to be pushed can be successfully retried and pushed. This includes:

[0021] Based on the success rate and time of the push of the content to be pushed, a first optimization target is determined. The first optimization target is positively correlated with the success rate of the push of the content to be pushed and negatively correlated with the time of the push of the content to be pushed.

[0022] A resource collaborative scheduling model based on deep reinforcement learning is used to determine a first scheduling strategy corresponding to the first optimization objective.

[0023] Resources are dynamically allocated to the content to be pushed according to the first scheduling strategy.

[0024] In some embodiments, the digital media content delivery method further includes:

[0025] When the content to be pushed corresponds to the second task queue, the second resource allocation strategy for the content to be pushed is determined based on the policy network of the resource collaborative scheduling model.

[0026] Based on the value network of the resource collaborative scheduling model, the second resource allocation strategy is evaluated to determine the predicted result of the second resource allocation strategy being applied to the content to be pushed;

[0027] If the content to be pushed is successfully pushed and the semantic accuracy reaches the preset accuracy in the prediction results, resources are allocated to the content to be pushed according to the second resource allocation strategy.

[0028] The content to be pushed is pushed to ensure that the accuracy of the push result reaches the preset accuracy.

[0029] In some implementations, when the content to be pushed corresponds to a second task queue, determining the resource allocation strategy for the content to be pushed based on the policy network of the resource collaborative scheduling model includes:

[0030] Based on the success rate and semantic accuracy of the content to be pushed, a second optimization objective is determined, wherein the second optimization objective is positively correlated with the success rate and semantic accuracy of the content to be pushed.

[0031] A policy network based on a resource collaborative scheduling model using deep reinforcement learning determines a second scheduling policy corresponding to the second optimization objective as the resource allocation policy for the content to be pushed.

[0032] In some embodiments, the digital media content delivery method further includes:

[0033] When the content to be pushed corresponds to a third task queue, a third resource allocation strategy for the content to be pushed is determined based on the strategy network of the resource collaborative scheduling model.

[0034] If the resources required by the third resource allocation strategy are not invoked by the first resource allocation strategy and the second resource allocation strategy, resources are allocated to the content to be pushed according to the third resource allocation strategy and the content to be pushed is pushed.

[0035] If the resources required by the third resource allocation strategy are invoked by the first resource allocation strategy or the second resource allocation strategy, the push of the content to be pushed shall be stopped.

[0036] In some implementations, when the content to be pushed corresponds to a third task queue, determining the third resource allocation strategy for the content to be pushed based on the policy network of the resource collaborative scheduling model includes:

[0037] Based on the success rate of the push of the content to be pushed, the resource idle rate and energy consumption index of the IoT platform, a third optimization target is determined. The third optimization target is positively correlated with the success rate of the push of the content to be pushed, and negatively correlated with the resource idle rate and energy consumption index of the IoT platform.

[0038] A policy network based on a resource collaborative scheduling model using deep reinforcement learning determines a third scheduling policy corresponding to the third optimization objective as the resource allocation policy for the content to be pushed.

[0039] On the other hand, this application implements a computer device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the digital media content push method as described in any of the above embodiments by calling the computer program stored in the memory.

[0040] On the other hand, this application implements a computer-readable storage medium storing a computer program adapted for loading by a processor to execute the digital media content delivery method as described in any of the above embodiments.

[0041] This application's implementation determines the task queue corresponding to the content to be pushed based on its resource type and push address; if the content to be pushed corresponds to a first task queue, the content to be pushed is pushed; if the push fails, a resource collaborative scheduling model based on deep reinforcement learning dynamically allocates resources to the content to be pushed so that the content to be pushed can be successfully retried and pushed. The resource collaborative scheduling model is trained based on the environmental interference characteristics, terminal device characteristics, node operation characteristics of the IoT platform, and the task profile of the content to be pushed. The content to be pushed is categorized into queues, and a deep reinforcement learning model trained based on multi-dimensional IoT features is introduced to dynamically schedule resources for the content to be pushed in different task queues, balancing the real-time nature and accuracy of digital media content push even with limited resources on the IoT platform. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating the digital media content push method provided in an embodiment of this application.

[0044] Figure 2 This is a schematic diagram illustrating the content push of the first task queue provided in an embodiment of this application.

[0045] Figure 3 This is a schematic diagram illustrating the training of the resource collaborative scheduling model provided in the embodiments of this application.

[0046] Figure 4 This is a schematic diagram illustrating the content push of the second task queue provided in an embodiment of this application.

[0047] Figure 5 This is a schematic diagram illustrating the content push of the third task queue provided in an embodiment of this application.

[0048] Figure 6 A schematic structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0049] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] This application provides a method, apparatus, and storage medium for pushing digital media content. This application relates to the field of Internet technology and the scenario of pushing digital media content on an Internet of Things (IoT) platform.

[0051] First, the background technology involved in the embodiments of this application will be described:

[0052] An IoT platform is an integrated system that connects and manages various IoT devices, collects, processes, and analyzes the data generated by these devices, and provides device management, data storage, and application development services. Therefore, IoT platforms often handle massive numbers of terminal devices accessing and interacting with data. These devices are frequently deployed in mobile or edge environments with complex and ever-changing network conditions, resulting in limited platform resources and difficulties in ensuring network connectivity stability.

[0053] In digital media content delivery scenarios on IoT platforms, the stability of network connections is difficult to guarantee, and the end-to-end link from the device to the business server is more likely to fail, leading to delivery failures. Delivery failures trigger the IoT platform's retry mechanism. Repeated deliveries further consume the IoT platform's resources, causing invalid messages to accumulate and occupy resources needed for normal business operations, resulting in a vicious cycle of decreased overall system throughput and response latency.

[0054] To address the aforementioned technical issues, the digital media content push method provided in this application embodiment can classify the content to be pushed into queues and introduce a deep reinforcement learning model trained based on multi-dimensional features of the Internet of Things to dynamically schedule resources for the content to be pushed in different task queues, thus balancing the real-time performance and accuracy of digital media content push even when the resources of the Internet of Things platform are limited.

[0055] The digital media content push method provided in this application is specifically illustrated through the following embodiments. These embodiments are described in detail below. It should be noted that the order of description of the following embodiments is not intended to limit the priority of the embodiments.

[0056] It is understood that in the specific implementation of this application, user object data, context data and other related data are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0057] Figure 1 This is a flowchart illustrating the digital media content push method provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the digital media content push method includes:

[0058] Step 11: Determine the task queue corresponding to the content to be pushed based on the resource type and push address of the content to be pushed;

[0059] Step 12: If the content to be pushed corresponds to the first task queue, push the content to be pushed;

[0060] Step 13: If the push fails, the resource collaborative scheduling model based on deep reinforcement learning dynamically allocates resources to the content to be pushed so that the content to be pushed can be successfully retried. The resource collaborative scheduling model is trained based on the environmental interference characteristics, terminal device characteristics, node operation characteristics and task profile of the content to be pushed on the IoT platform.

[0061] Specifically, based on the resource type of the content to be pushed (such as media streams, text data, etc.) and the push address (such as the location of the target device or network node), intelligent analysis is used to allocate it to different task queues to achieve subsequent differentiated scheduling and resource management.

[0062] When the resource type is a high-priority, real-time media stream (such as a real-time monitoring stream or a timed advertisement), the system will directly assign it to the first task queue. Similarly, if the target address is an important address that requires priority push, the system will also directly assign it to the first task queue. The first task queue enjoys higher bandwidth and computing resource guarantees to ensure rapid response for critical business operations.

[0063] The target device and target address are preset. When the resource type of the content to be pushed is identified as a high-priority, real-time target resource type, or the push address is a target address that needs to be pushed first, the content to be pushed can be assigned to the first task queue.

[0064] Figure 2 This is a schematic diagram illustrating the content push of the first task queue provided in an embodiment of this application.

[0065] like Figure 2 As shown, for the highest priority first queue task, an immediate push strategy is adopted, which ensures that critical business can get the fastest response on the first attempt and avoids delays in emergency tasks that may be caused by unified queuing.

[0066] When the content push of the first task queue fails, the system does not simply and blindly repeat the retry. Instead, it activates a resource collaborative scheduling model based on deep reinforcement learning (DRL) to dynamically allocate the resources required for the retry. Given the limited resources of the IoT platform, it prioritizes ensuring that the content to be pushed in the first task queue can be pushed in a timely and successful manner.

[0067] The training data for the resource collaborative scheduling model comes from real historical data from the IoT platform, including environmental interference characteristics such as network latency, packet loss rate and signal strength, terminal device characteristics such as remaining battery power, computing power and connection status, node operation characteristics such as current server load, bandwidth utilization and queue length, and task profiles of the content to be pushed, such as content size, priority and business deadline.

[0068] The resource collaborative scheduling model can accurately perceive whether the failure was caused by environmental interference, busy receiving devices, or saturated platform processing capacity, and make precise interventions accordingly to ensure that the content to be pushed in the first queue of tasks can be pushed in a timely and successful manner.

[0069] Thus, the implementation method of this application can improve the success rate of repeated push of content to be pushed in the first queue task, avoid the vicious cycle of resource contention and system throughput reduction caused by invalid retries, and take into account the real-time performance and accuracy of digital media content push under the condition of limited IoT platform resources.

[0070] Figure 3 This is a schematic diagram illustrating the training of the resource collaborative scheduling model provided in the embodiments of this application.

[0071] like Figure 3 As shown, in some implementations, the steps of training the resource collaborative scheduling model include:

[0072] Step 01: Construct a simulation environment for the IoT platform;

[0073] Step 02: Obtain the environmental interference characteristics, terminal device characteristics, node operation characteristics, and task profile of the content to be pushed from the historical database of the IoT platform;

[0074] Step 03: Construct a state space matrix based on the environmental interference characteristics, terminal device characteristics, node operation characteristics, and task profile of the content to be pushed to the IoT platform;

[0075] Step 04: Input the state space matrix into the resource collaborative scheduling model and output the simulated scheduling strategy;

[0076] Step 05: Execute the simulation scheduling strategy in the simulation environment of the IoT platform to obtain the updated status of the simulation scheduling strategy. The updated status of the simulation scheduling strategy includes the updated values ​​of the environmental interference characteristics of the IoT platform, terminal device characteristics, node operation characteristics, and task profile of the content to be pushed.

[0077] Step 06: Based on the updated state of the simulated scheduling strategy, train the value network and policy network in the resource collaborative scheduling model.

[0078] Specifically, conducting DRL trial-and-error training in real production systems is too costly and risky. Therefore, it is necessary to build a highly realistic IoT push digital twin simulation environment as the data foundation and execution stage for subsequent training.

[0079] Historical data can be derived from multimodal logs. Environmental interference characteristics can be derived from network probe data, including historical network latency, jitter, packet loss rate records, and base station / router load information. Node operation characteristics can be derived from platform monitoring logs, including server memory utilization, bandwidth utilization, and queue depth time series. Terminal device characteristics can be derived from device reporting logs, including periodically reported battery levels, network signal strength (RSRP / RSRQ), screen status, etc. Task profiles of content to be pushed can be derived from task execution logs, including detailed records of each push task, such as task ID, content size, target device, priority, initiation time, success / failure status, and number of retries.

[0080] The original historical data can be transformed into a unified continuous state vector that can be directly processed by the DRL model, and a state space matrix can be constructed in the perception layer.

[0081] For the event that task T fails at time t, extract network data, device data, and overall platform resource data related to the target device of task T within the time window of [t-τ, t].

[0082] The environmental interference feature vector (S_env) can include network latency, which represents the average round-trip time (milliseconds) within the window; network jitter, which represents the standard deviation of the latency within the window; packet loss, which represents the average percentage of packets lost within the window; and cell load, which represents the average utilization of the base stations to which the device is connected.

[0083] The terminal device feature vector (S_dev) can include device battery level (battery_level), representing the current battery percentage; signal strength (signal_strength), represented as RSRP value (dBm); screen status (screen_on), represented as a binary variable (1 for screen on, 0 for screen off); and device performance level (perf_tier), a static performance score mapped according to the device model (e.g., 0-1).

[0084] The platform's runtime feature vector (S_plat) can include CPU utilization (cpu_util), representing the average CPU utilization of the core processing server; memory utilization (mem_util), representing the average memory utilization; egress bandwidth utilization (bw_util), representing the network egress bandwidth utilization; and high-priority queue depth (high_prio_queue_len), representing the number of waiting tasks in the first task queue.

[0085] The task profile feature vector (S_task) can include task priority; content data size (data_size_MB); number of failures (failure_count), which is the cumulative number of failures for the task so far; and content type (content_type), which is represented using one-hot encoding, such as [1,0,0] representing video, [0,1,0] representing image, etc.

[0086] Minimize-maximum normalize each continuous parameter in the environmental disturbance feature vector (S_env) and scale it to the [0,1] interval to eliminate the influence of dimensions and accelerate the convergence of the neural network. The final result is S_env = [Norm(latency),Norm(jitter), Norm(packet_loss), Norm(cell_load)].

[0087] Similarly, by normalizing the continuous values ​​of the terminal device feature vector (S_dev), platform operation feature vector (S_plat), and task profile feature vector (S_task), we can obtain:

[0088] S_dev=[Norm(battery_level),Norm(signal_strength),screen_on,perf_tier];

[0089] S_plat=[Norm(cpu_util),Norm(mem_util),Norm(bw_util),Norm(high_prio_queue_len)];

[0090] S_task = [priority, Norm(data_size_MB), Norm(failure_count), one_hot(content_type)].

[0091] The state space matrix can be represented as S_t = CONCAT(S_env, S_dev, S_plat, S_task).

[0092] The constructed state space matrix is ​​input into the initialized resource collaborative scheduling model (i.e., DRL agent). The model calculates through its internal policy network and outputs a simulated scheduling policy (action).

[0093] In the IoT platform simulation environment, the model outputs a simulation scheduling policy A_t. The environment is updated according to the received action instructions to obtain an updated state S_{t+1}, which includes the updated values ​​of S_env, S_dev, S_plat, and S_task.

[0094] Resource collaborative scheduling models can include policy networks (Actors) and value networks (Critics).

[0095] The role of the policy network is to directly generate specific resource scheduling actions based on the currently observed complex system state. It receives a high-dimensional state vector S_t, which is processed by a deep neural network to output a specific scheduling action A_t. The training objective of the policy network is to learn so that its output action can maximize the long-term cumulative push success rate and system efficiency.

[0096] The role of a value network is to provide a forward-looking value assessment of the decisions (state-action pairs) made by the policy network. It typically receives a state S_t and an action A_t suggested by the Actor, and outputs V_t representing the predicted outcome of executing action A_t in state S_t (or following the current policy in state S_t). The training objective of the value network is to learn so that its output predictions provide stable, low-variance directional guidance for the Actor's policy updates.

[0097] In some implementations, step 06: Based on the updated state of the simulated scheduling policy, model training is performed on the value network and policy network in the resource collaborative scheduling model, including:

[0098] Step 061: Evaluate the updated state of the simulated scheduling strategy according to the target reward function, and obtain the reward value of the simulated scheduling strategy;

[0099] Step 062: Determine the first loss function based on the reward value of the simulated scheduling policy to train the policy network model;

[0100] Step 063: Determine the second loss function based on the reward value of the simulated scheduling strategy and the evaluation value of the value network on the simulated scheduling strategy, so as to train the model of the value network.

[0101] Specifically, such as Figure 3 As shown, the reward function (Reward) calculates the immediate reward R_t, thus obtaining a complete training sample (S_t, A_t, R_t, S_{t+1}).

[0102] The role of the reward function is to transform complex and abstract business objectives (such as "ensuring successful and efficient push notifications") into a computable scalar signal, thereby quantifying the immediate quality of each scheduling action of the agent and guiding its strategy towards long-term optimality. Therefore, different reward functions can be executed for training based on the content to be pushed in different task queues.

[0103] For example, for the content to be pushed in the first task queue, priority should be given to ensuring the success rate and timeliness of the push. Successful and timely push of digital media content can be used as a reward for success, and failed and timed push of digital media content can be used as a penalty for failure.

[0104] In this way, a set of training samples (S_t, A_t, R_t, S_{t+1}) can be obtained to train the policy network (Actor) and the value network (Critic).

[0105] R_t can directly evaluate the execution effect of action A_t. The larger the value of R_t, the better the execution effect of action A_t. The difference between the value of R_t and its theoretical maximum value is used as the first loss function L1 to train the policy network (Actor) to ensure that the policy network (Actor) outputs the evaluation of the execution effect of action A_t.

[0106] V_t represents the predicted effect that can be obtained by performing action A_t in state S_t (or following the current policy in state S_t). R_t can directly evaluate the actual performance of action A_t. The closer the predicted effect represented by V_t is to the actual performance represented by R_t, the better the predicted effect represented by V_t. The difference between R_t and A_t is used as the second loss function L2 to train the value network (Critic) to ensure that the predicted effect represented by the evaluation action V_t output by the value network (Critic) is optimized.

[0107] In some implementations, step 13: If the push fails, a resource collaborative scheduling model based on deep reinforcement learning dynamically allocates resources to the content to be pushed so that the content to be pushed can be successfully retried, including:

[0108] Step 131: Based on the success rate and push time of the content to be pushed, determine the first optimization target. The first optimization target is positively correlated with the success rate of the content to be pushed and negatively correlated with the push time of the content to be pushed.

[0109] Step 132: Based on the resource collaborative scheduling model of deep reinforcement learning, determine the first scheduling strategy corresponding to the first optimization objective;

[0110] Step 133: Dynamically allocate resources for the content to be pushed according to the first scheduling strategy.

[0111] Specifically, when making intelligent retry decisions after a push fails, the core optimization direction to follow, namely the first optimization goal, is to balance two key business metrics: push success rate and push time. The goal is to maximize the success rate of task delivery in the shortest possible time.

[0112] During the training of the resource collaborative scheduling model, the training samples include the task profiles of the content to be pushed. For the training samples located in the first task queue, a first reward function can be set for training, which can be: R_t=w1*R_success+w2*R_efficiency.

[0113] For example, set the weights as follows: w1=0.5, w2=-0.1, R_success=expected success of push, R_efficiency=α*average task delay+expected failure of push, and α is 0.01.

[0114] If the average latency after executing action A_t is 200ms, and 98 out of 100 tasks with a priority of 1.0 (the highest) successfully push the data, then R_success=0.98, R_efficiency=0.02+200*0.01=2.02.

[0115] R_t=0.5*0.98-0.1*2.02=0.248.

[0116] During training, ensure that R_t after executing action A_t is as close as possible to the maximum value, that is, 100 out of 100 tasks with priority of 1.0 (highest) are successfully pushed without timeout, with R_success=1, R_efficiency=0, and R_t=0.5.

[0117] Thus, when the push of content to be pushed in the first task queue fails, the executed A_t can ensure successful and timely push the next time when making intelligent retry decisions.

[0118] Figure 4 This is a schematic diagram illustrating the content push of the second task queue provided in an embodiment of this application.

[0119] like Figure 4 As shown, in some embodiments, the digital media content delivery method further includes:

[0120] Step 21: When the content to be pushed corresponds to the second task queue, determine the second resource allocation strategy for the content to be pushed based on the strategy network of the resource collaborative scheduling model.

[0121] Step 22: Based on the value network of the resource collaborative scheduling model, evaluate the second resource allocation strategy to determine the predicted results of the second resource allocation strategy for the content to be pushed;

[0122] Step 23: If the content to be pushed is successfully pushed and the semantic accuracy reaches the preset accuracy in the prediction results, allocate resources to the content to be pushed according to the second resource allocation strategy.

[0123] Step 24: Push the content to be pushed to ensure that the accuracy of the push results reaches the preset accuracy.

[0124] Specifically, when the resource type is a media stream requiring high accuracy (such as news broadcasts), the system will allocate it to the second task queue. Similarly, if the target address requires accurate push, the system will also directly allocate it to the second task queue. The content to be pushed in the second task queue must ensure semantic integrity, meaning that the information received by the receiving end must be free of distortion and loss.

[0125] When the content to be pushed is identified as belonging to the second queue, the system does not push it immediately. Instead, it first invokes the policy network in the resource collaborative scheduling model. Based on learning from historical best policies, the policy network comprehensively analyzes the current state and generates a second resource allocation policy specifically tailored for this task. The second resource allocation policy is a specific pre-execution scheme whose goal is to proactively optimize conditions before pushing the content.

[0126] The resource allocation strategy proposed by the policy network is fed into the value network for simulation and evaluation. The core function of the value network is to predict whether the push will be successful if resources are allocated and executed strictly according to the strategy, and what level of "semantic accuracy" the delivered content will achieve.

[0127] The "semantic accuracy" is directly related to the "semantic anomaly perception" ability, which is to judge whether the core information of the content (such as audio streams or text of news broadcasts) is complete and error-free after experiencing a potentially unstable transmission link.

[0128] The value network calculates and outputs a quantitative prediction value, which represents a preliminary judgment on the quality of the push under the proposed strategy, thus realizing the prediction of risk.

[0129] The system determines that a resource allocation strategy is reliable enough to guarantee high-quality push notifications only when the predicted value reaches or exceeds the threshold, and then approves its execution. Subsequently, based on the strategy details, the system allocates the planned computing resources, network bandwidth, and storage space to the content to be pushed. If the predicted value fails to meet the target, the strategy will be rejected. The system may require the strategy network to regenerate the plan or activate a higher-level compensation mechanism, thereby filtering out inefficient or ineffective plans before actual resource allocation, thus preventing resource waste and disruption to normal system operations from the outset.

[0130] After resources are successfully allocated and ready according to the optimization strategy, the system officially executes the push operation for the second queue. Since the entire push process is based on intelligent planning and predictive evaluation, its goal is clearly to ensure that the accuracy of the push results reaches the preset accuracy.

[0131] In some implementations, step 21: when the content to be pushed corresponds to a second task queue, a resource allocation strategy for the content to be pushed is determined based on the policy network of the resource collaborative scheduling model, including:

[0132] Step 211: Based on the success rate and semantic accuracy of the content to be pushed, determine the second optimization objective. The second optimization objective is positively correlated with the success rate and semantic accuracy of the content to be pushed.

[0133] Step 212: The policy network of the resource collaborative scheduling model based on deep reinforcement learning determines the second scheduling policy corresponding to the second optimization objective as the resource allocation policy for the content to be pushed.

[0134] Specifically, the second optimization goal aims to balance two key business metrics: push success rate and semantic accuracy, with the goal of increasing the semantic accuracy of push content while maximizing the push success rate.

[0135] During the training of the resource collaborative scheduling model, the training samples include the task profiles of the content to be pushed. For the training samples located in the second task queue, a second reward function can be set for training, specifically: R_t=w1*R_success+w2*R_efficiency.

[0136] For example, set the weights as follows: w1=0.5, w2=-0.4, R_success=expected success of push, R_efficiency=average percentage of semantically abnormal regions.

[0137] If, after executing action A_t, 98 out of 100 tasks with a priority of 2.0 successfully push the message, and the average percentage of semantically abnormal regions is 25%, then R_success=0.98 and R_efficiency=0.25.

[0138] R_t=0.5*0.98-0.4*0.25=0.39.

[0139] During training, ensure that R_t after executing action A_t is as close as possible to the maximum value, that is, 100 out of 100 tasks with priority of 2.0 are successfully pushed, and none of them have semantic anomalies, i.e., R_success=1, R_efficiency=0, R_t=0.5.

[0140] Thus, A_t executed according to the second scheduling strategy can ensure successful push and semantic accuracy of the pushed content.

[0141] Figure 5 This is a schematic diagram illustrating the content push of the third task queue provided in an embodiment of this application.

[0142] like Figure 5 As shown, in some embodiments, the digital media content delivery method further includes:

[0143] Step 31: When the content to be pushed corresponds to the third task queue, determine the third resource allocation strategy for the content to be pushed based on the strategy network of the resource collaborative scheduling model.

[0144] Step 32: If the resources required by the third resource allocation strategy are not called by the first and second resource allocation strategies, allocate resources for the content to be pushed according to the third resource allocation strategy and push the content to be pushed.

[0145] Step 33: If the resources required by the third resource allocation strategy are called by the first or second resource allocation strategy, stop pushing the content to be pushed.

[0146] Specifically, when the content to be pushed is identified by the system and categorized into the third task queue, it indicates that the task may belong to a category with lower real-time requirements or relatively general business priority. Based on learning from historical best strategies, the policy network comprehensively analyzes the current state and generates a third resource allocation strategy specifically customized for this task.

[0147] The system checks whether the necessary resources specified in the third resource allocation strategy (such as bandwidth of a specific frequency band, computing units of edge nodes, or storage space) are being used or have already been used by the corresponding strategies of the higher-priority first queue task (urgent push) or second queue task (high-accuracy guaranteed push). If the check result is no, that is, the required resources are idle or available for allocation, the system will approve the strategy and, according to its plan, actually allocate the corresponding computing, storage, and network resources for the content to be pushed, and then immediately execute the push operation.

[0148] If the system detects that the resources required by the third resource allocation strategy have been occupied by the strategy of the first or second queue, i.e. there is a resource conflict, the system will make the decision to "stop pushing the content to be pushed".

[0149] "Stop pushing content to be pushed" may manifest as putting the task back into the waiting queue, canceling the current scheduling attempt, or directly discarding the push request. The purpose is to unconditionally ensure the resource exclusivity and execution success rate of high-priority tasks, and prevent system congestion or degradation of critical business quality caused by excessive resource competition.

[0150] In some implementations, step 31: when the content to be pushed corresponds to a third task queue, a third resource allocation strategy for the content to be pushed is determined based on the policy network of the resource collaborative scheduling model, including:

[0151] Step 311: Based on the success rate of the push of the content to be pushed, the resource idle rate and energy consumption index of the IoT platform, determine the third optimization target. The third optimization target is positively correlated with the success rate of the push of the content to be pushed, and negatively correlated with the resource idle rate and energy consumption index of the IoT platform.

[0152] Step 312: Based on the policy network of the resource collaborative scheduling model of deep reinforcement learning, determine the third scheduling policy corresponding to the third optimization objective as the resource allocation policy for the content to be pushed.

[0153] Specifically, the third optimization objective aims to balance two key business metrics: push success rate and semantic accuracy, with the goal of minimizing the resource idle rate and energy consumption of the IoT platform while maximizing the push success rate.

[0154] During the training of the resource collaborative scheduling model, the training samples include the task profile of the content to be pushed. For the training samples located in the third task queue, a third reward function can be set for training, which can be: R_t=w1*R_success+w2*R_efficiency.

[0155] For example, set the weights: w1=0.5, w2=-0.3, R_success=expected success of push, R_efficiency=k1*resource idle rate of IoT platform +k2*energy consumption of IoT platform, k1 can be 0.5, k2 can be 0.5, and the energy consumption of IoT platform is represented as 0 to 1. When the energy consumption of IoT platform is represented as 0, no resources are allocated to any content to be pushed in the third task queue. When the energy consumption of IoT platform is represented as 1, the energy consumption of IoT platform reaches the upper limit.

[0156] If, after executing action A_t, 98 out of 100 tasks with a priority of 3.0 are successfully pushed, and the resource idle rate of the IoT platform is 30%, and the energy consumption of the IoT platform is expressed as 0.4, then R_success=0.98, R_efficiency=0.3*0.5+0.4*0.5=0.35.

[0157] R_t=0.5*0.98-0.3*0.35=0.385.

[0158] During training, ensure that R_t after executing action A_t is as close to the maximum value as possible, that is, 100 out of 100 tasks with priority of 3.0 are successfully pushed, and the additional energy consumed by the content to be pushed in the third task queue is close to zero, and there is no idle resource situation.

[0159] Thus, A_t, executed according to the third scheduling strategy, can ensure that the resource idle rate and energy consumption of the IoT platform are minimized while maximizing the push success rate.

[0160] All of the above technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0161] To facilitate better implementation of the digital media content push method of the embodiments of this application, the embodiments of this application also provide a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0162] Figure 6 A schematic structural diagram of the computer device provided in the embodiments of this application, such as Figure 6 As shown, the computer device may include: a communication interface, a memory, a processor, and a communication bus. The communication interface, memory, and processor communicate with each other via the communication bus. The communication interface is used for data communication between the device and external devices. The memory can be used to store software programs and modules, and the processor runs the software programs and modules stored in the memory, such as the software programs for the corresponding operations in the foregoing method embodiments.

[0163] In some embodiments, the processor may invoke software programs and modules stored in memory to execute the digital media content push method described above.

[0164] In some embodiments, the computer device may be integrated into a terminal or server that has storage and a processor and thus computing power, or the computer device may be the terminal or server.

[0165] This application also provides a computer-readable storage medium for storing a computer program. This computer-readable storage medium can be applied to a computer device, and the computer program causes the computer device to execute the corresponding processes in the methods described above in the embodiments of this application; for brevity, further details are omitted here.

[0166] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0167] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0168] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0169] In addition, the functional units in the embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0170] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer or a server) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0171] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for pushing digital media content, characterized in that, include: The task queue corresponding to the content to be pushed is determined based on the resource type and push address of the content to be pushed. If the content to be pushed corresponds to the first task queue, then the content to be pushed will be pushed. If the push fails, a resource collaborative scheduling model based on deep reinforcement learning dynamically allocates resources to the content to be pushed so that the content to be pushed can be successfully retried. The resource collaborative scheduling model is trained based on the environmental interference characteristics, terminal device characteristics, node operation characteristics of the Internet of Things platform and the task profile of the content to be pushed. When the content to be pushed corresponds to the second task queue, the second resource allocation strategy for the content to be pushed is determined based on the policy network of the resource collaborative scheduling model. Based on the value network of the resource collaborative scheduling model, the second resource allocation strategy is evaluated to determine the predicted result of the second resource allocation strategy being applied to the content to be pushed; If the content to be pushed is successfully pushed and the semantic accuracy reaches the preset accuracy in the prediction results, resources are allocated to the content to be pushed according to the second resource allocation strategy. The content to be pushed is pushed to ensure that the accuracy of the push result reaches the preset accuracy.

2. The digital media content push method according to claim 1, characterized in that, The steps for training the resource collaborative scheduling model include: Construct a simulation environment for the Internet of Things (IoT) platform; The environmental interference characteristics, terminal device characteristics, node operation characteristics, and task profile of the content to be pushed to the IoT platform are obtained from the historical database. Based on the environmental interference characteristics, terminal device characteristics, node operation characteristics of the IoT platform, and the task profile of the content to be pushed, a state space matrix is ​​constructed. The state space matrix is ​​input into the resource collaborative scheduling model to output a simulated scheduling strategy. The simulation scheduling strategy is executed in the simulation environment of the Internet of Things platform to obtain the updated status of the simulation scheduling strategy. The updated status of the simulation scheduling strategy includes the updated values ​​of the environmental interference characteristics, terminal device characteristics, node operation characteristics and task profile of the content to be pushed by the Internet of Things platform. Based on the updated state of the simulated scheduling strategy, the value network and policy network in the resource collaborative scheduling model are trained.

3. The digital media content push method according to claim 2, characterized in that, The step of training the value network and policy network in the resource collaborative scheduling model based on the updated state of the simulated scheduling policy includes: The update status of the simulated scheduling strategy is evaluated based on the target reward function to obtain the reward value of the simulated scheduling strategy. Based on the reward value of the simulated scheduling strategy, a first loss function is determined to train the model of the strategy network; A second loss function is determined based on the reward value of the simulated scheduling strategy and the evaluation value of the value network on the simulated scheduling strategy, in order to train the model of the value network.

4. The digital media content delivery method according to any one of claims 1-3, characterized in that, If the push fails, a resource collaborative scheduling model based on deep reinforcement learning is used to dynamically allocate resources to the content to be pushed, so that the content to be pushed can be successfully retried and pushed, including: Based on the success rate and time of the push of the content to be pushed, a first optimization target is determined. The first optimization target is positively correlated with the success rate of the push of the content to be pushed and negatively correlated with the time of the push of the content to be pushed. A resource collaborative scheduling model based on deep reinforcement learning is used to determine a first scheduling strategy corresponding to the first optimization objective. Resources are dynamically allocated to the content to be pushed according to the first scheduling strategy.

5. The digital media content push method according to claim 4, characterized in that, When the content to be pushed corresponds to the second task queue, the resource allocation strategy for the content to be pushed is determined based on the policy network of the resource collaborative scheduling model, including: Based on the success rate and semantic accuracy of the content to be pushed, a second optimization objective is determined, wherein the second optimization objective is positively correlated with the success rate and semantic accuracy of the content to be pushed. A policy network based on a resource collaborative scheduling model using deep reinforcement learning determines a second scheduling policy corresponding to the second optimization objective as the resource allocation policy for the content to be pushed.

6. The digital media content push method according to claim 5, characterized in that, The digital media content delivery method also includes: When the content to be pushed corresponds to a third task queue, a third resource allocation strategy for the content to be pushed is determined based on the strategy network of the resource collaborative scheduling model. If the resources required by the third resource allocation strategy are not invoked by the first resource allocation strategy and the second resource allocation strategy corresponding to the first scheduling strategy, resources are allocated to the content to be pushed according to the third resource allocation strategy and the content to be pushed is pushed. If the resources required by the third resource allocation strategy are invoked by the first resource allocation strategy or the second resource allocation strategy, the push of the content to be pushed shall be stopped.

7. The digital media content push method according to claim 6, characterized in that, When the content to be pushed corresponds to a third task queue, the third resource allocation strategy for the content to be pushed is determined based on the policy network of the resource collaborative scheduling model, including: Based on the success rate of the push of the content to be pushed, the resource idle rate and energy consumption index of the IoT platform, a third optimization target is determined. The third optimization target is positively correlated with the success rate of the push of the content to be pushed, and negatively correlated with the resource idle rate and energy consumption index of the IoT platform. A policy network based on a resource collaborative scheduling model using deep reinforcement learning determines a third scheduling policy corresponding to the third optimization objective as the resource allocation policy for the content to be pushed.

8. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program, and the processor executing the digital media content push method according to any one of claims 1-7 by calling the computer program stored in the memory.

9. A computer program product, characterized in that, It includes computer instructions, which, when executed by a processor, implement the digital media content push method according to any one of claims 1-7.

Images