An immersive media resource allocation method based on auction bidding

Abstract

This invention discloses an immersive media resource allocation method based on auction bidding, belonging to the field of satellite network resource allocation. Specifically, it involves: First, constructing an immersive media resource allocation system composed of multiple users, a space-ground integrated network, and multimedia servers; then, for the current immersive media resource, users initiate requests, bidding based on their budget and an estimated value of the resource; next, ground station nodes receive resource request and bidding information from all users and sort the bids in descending order; a second-price auction method is used to determine the successful bidder; and using a Dijkstra algorithm and load balancing, each user's request information is sent to different multimedia servers. Finally, each multimedia server releases its corresponding resources, providing immersive media services, and returns the current resource utilization and task execution results to the ground station nodes. This invention improves resource utilization and user satisfaction.

Description

Technical Field

[0001] This invention belongs to the field of satellite network resource allocation and relates to auction design methods, specifically an immersive media resource allocation method based on auction bidding. Background Technology

[0002] With the rapid development of satellite communications, the convergence of satellite and terrestrial networks has become a trend. Integrated space-ground networks are robust and reliable, offering advantages such as wide coverage and 24 / 7 connectivity, making them suitable for various applications and industries. The bandwidth demands of immersive media such as 4K / 8K video and virtual reality are growing rapidly. This explosive growth in multimedia traffic poses challenges to networks. Due to unstable network connections in urban areas and high costs of broadband infrastructure construction in rural areas, it is difficult for people to freely enjoy real-time, low-cost immersive media services. Furthermore, while immersive media emphasizes personal experience and multi-user interaction, end-to-end latency and low transmission bandwidth result in poor service quality.

[0003] Integrated terrestrial and satellite networks typically provide best-effort delivery for multimedia services, focusing on Quality of Service (QoS) performance rather than the end-user's personal experience. For example, 5G-ALLSTAR, part of the EU's Horizon 2020 project, offers a virtual reality-based tennis match scene that viewers from Europe and South Korea can stream online. IM applications provide entertainment and communication services; however, these services involve large amounts of data, require demanding network environments, and can be costly. Unstable inter-satellite links in satellite networks also impact user satisfaction with high-quality IM services.

[0004] Existing technologies regarding video stream allocation resources in terrestrial networks include the following literature:

[0005] Reference [1] considers a machine learning model to predict the number of users in the cloud and then actively optimizes the allocation of resources near the users to achieve a trade-off between access latency and cost.

[0006] Reference [2] designed an edge computing-assisted video traffic offloading scenario to ensure user experience and improve network resource efficiency.

[0007] Reference [3] proposes an online intelligent auction algorithm for resources on satellite networks. These resource allocation methods are difficult to meet the requirements of IM applications for real-time performance, bandwidth, and latency.

[0008] Therefore, there is a need for a bidding-based resource allocation method that can provide users with personalized, immersive media services.

[0009] Related literature:

[0010] [1] F.Haouari, E.Baccour, A.Erbad, A.Mohamed, and M.Guizani, “Qoe-awareresource allocation for crowdsourced live streaming: a machine learning approach,” in Proc.IEEE Int.Conf.Commun.(ICC), Shanghai, China, May 2019, pp.1–6.

[0011] [2] S. Kumar, N. Wang, Y. Rahulan, and B. Evans, "QoE-aware video streaming over integrated space and terrestrial 5G networks," IEEE Netw., vol.35, no.4, pp.95–101, Jul.2021.

[0012] [3] A method for allocating MEC-assisted satellite network resources based on online intelligent auction, Publication No.: CN115689228A. Summary of the Invention

[0013] To address the aforementioned problems, this invention provides an immersive media resource allocation method based on auction bidding. It introduces a multi-agent reinforcement learning (MARL) algorithm, treating each user as an agent that learns and adapts its behavior according to the environment. Through learning and game theory, network utility is optimized, achieving efficient resource allocation in practice. The method proposed in this invention is significant in improving resource utilization and user satisfaction.

[0014] The specific steps of the immersive media resource allocation method based on auction bidding are as follows:

[0015] Step 1: Construct an immersive media resource allocation system consisting of one user, an integrated terrestrial-satellite network, and a multimedia server;

[0016] The nodes of the space-ground integrated network are represented by the graph G=(V,E), where V represents satellite and ground station nodes, and E represents inter-satellite links and satellite-ground links.

[0017] Each multimedia server has the same capacity and offers M types of immersive media services. For each service type, n services are available. Service types include low QoS and high QoS. Different service types occupy different resource units.

[0018] Step 2: For the current immersive media resources, each user initiates their own request based on their needs;

[0019] The request information from all users is composed of a set B = {b1, b2, ..., b} i ,...,b I}; Each request message b i =(t i ,r i ,d i ) contains the type t of the requested service. i Required resource unit r i and the longest latency d that the user can tolerate i .

[0020] Step 3: Each user submits a bid based on their request information, budget, and estimated value of the current immersive media resources; and observes the previous bids from other users to decide whether to continue bidding or withdraw their bid.

[0021] When users bid, each user is treated as an agent. Based on a multi-agent reinforcement learning algorithm, the agent's action strategy is updated according to the current state, and the system utility is calculated. The agent's strategy is then adjusted based on the calculation results to make the bidding more reasonable.

[0022] System utility includes two parts: the utility of users participating in the auction and the resource utilization of the multimedia server.

[0023] The utility of a user participating in an auction is expressed as: u = y[(vp)x + R(1-x)] + (1-y)D;

[0024] Where y represents whether the user withdraws from the auction, y=1 indicates withdrawal; if the user withdraws from the auction, a fixed withdrawal cost D needs to be paid; if y=0, the user continues the auction, but may be rejected; x=0 indicates that the auction is unsuccessful and a fixed rejection cost R needs to be paid; x=1 indicates that the auction is successful, p is the user's bid price, v represents the value of the goods, and vp represents the reward the user can receive after the auction is successful.

[0025] The resource utilization rate of a multimedia server represents the current resource usage, denoted by α.

[0026] Therefore, the system utility is expressed as U = Σ i∈I Σ m∈M u i,m +K(1-α); K is a constant;

[0027] u i,m This represents the utility of the i-th user for the m-th type of immersive media service;

[0028] The adjustment rules for the agent's strategy are as follows: based on reinforcement learning algorithms, the Actor-Critic algorithm is used to learn user behavior; the specific process is as follows:

[0029] s301: Define the state space of the agent as S(t) = [b i U m,i [(t-1), P(t-1), α], which are the request information b of the i-th user respectively. i The utility U of the i-th user for the m-th type of immersive media service at the previous time step m,i (t-1), the user's bid information P(t-1) at the previous time step, and the system's resource utilization rate α.

[0030] The utility of the i-th user for the m-th type of immersive media service is calculated as follows:

[0031] U m,i =y m,i C m,i +(1-y m,i )D m,i

[0032] y m,i Indicates whether the i-th user withdraws their bid for the m-th type of immersive media service; D m,i C represents the withdrawal cost incurred when the i-th user withdraws their bid for the m-th type of immersive media service; m,i The cost for the i-th user to participate in the auction for the m-th type of immersive media service is calculated as follows:

[0033] C m,i =x m,i (p m,i -v m,i )+(1-x m,i )R m,i

[0034] p m,i v represents the bid price of the i-th user for the m-th type of immersive media service. m,i R represents the value of the m-th type of immersive media business; m,i This represents the cost of the i-th user's bid for the m-th type of immersive media service being rejected;

[0035] s302: Design the action space of the intelligent agent A(t) = [p i (t),y i [(t)], which represent the bid p of the i-th user. i (t) and whether to withdraw this round of auction y i (t).

[0036] s303: Design the agent's reward function R(t) to maximize user utility and resource utilization α:

[0037]

[0038] S304: Use the Actor-critic algorithm to learn user behavior and generate the optimal result.

[0039] Step 4: For the current immersive media resources, the ground station nodes of the integrated space-ground network accept resource request information and bidding information from all users, and sort the users' bids in descending order; a second-price auction method is used to determine the users who win the auction and the users who reject it.

[0040] The number of users who successfully bid will remain the same as the number of existing resources, and the price they need to pay will be the second highest bid.

[0041] Step 5: For all users who successfully bid, the request information of each user is sent to different multimedia servers using the Dijkstra algorithm and load balancing.

[0042] Step Six: After receiving the user's request information, each multimedia server releases the resources corresponding to the request information to provide immersive media services; and returns the current resource utilization and task execution results to the ground station node.

[0043] The advantages of this invention are:

[0044] 1) An immersive media resource allocation method based on auction bidding allows users to bid according to their own needs for immersive media resources. As a result, users will give real and positive feedback on their needs for the business, which improves user participation and interactivity. This also helps to stimulate users' demand for business-related resources.

[0045] 2) An immersive media resource allocation method based on auction bidding, in which a multi-agent reinforcement learning algorithm can dynamically adjust the agent's action strategy according to the current state and network utility, thereby improving the user's personal reward and the system resource utilization rate.

[0046] 3) An immersive media resource allocation method based on auction bidding, which addresses the problem of focusing more on network transmission performance than user experience in integrated space-ground networks. It uses an auction method to allocate limited immersive resources to users, providing them with different types of immersive media services, and achieving flexible allocation and efficient utilization of resources. Attached Figure Description

[0047] Figure 1This is a flowchart of an immersive media resource allocation method based on auction bidding according to the present invention;

[0048] Figure 2 This is a schematic diagram of the immersive media resource allocation system constructed according to the present invention. Detailed Implementation

[0049] The embodiments of the present invention will now be described in detail and clearly with reference to the accompanying drawings.

[0050] This invention provides an immersive media resource allocation method based on auction bidding, specifically including: constructing an immersive media resource allocation system composed of multiple satellites and multiple users; introducing a service satisfaction index to represent the service relationship between satellites and users, which represents the overall service satisfaction, including: the satisfaction of satellites with the service benefits to users and the satisfaction of users with satellite services; and using an online multi-round auction resource allocation algorithm that maximizes overall service satisfaction and the Vickrey-Clerke-Groves payment mechanism to determine the service matching relationship between satellites and users and the final payment price of the traded resources, so as to maximize overall service satisfaction.

[0051] This invention's auction bidding allows users to bid based on their own demand for immersive media resources and the bidding rules of other users. Simultaneously, to ensure network profitability and efficiency, this invention introduces the concept of network utility. Specifically, network utility is composed of user bidding and resource allocation results, aiming to achieve high resource profitability and maximum user benefit. Furthermore, this invention introduces a multi-agent reinforcement learning algorithm (MARL), where each user is treated as an agent, learning and changing their behavior according to the environment. Through learning and game theory, network utility is optimized, achieving efficient resource allocation in practice. The method proposed in this invention is significant in improving resource utilization and user satisfaction.

[0052] The aforementioned immersive media resource allocation method based on auction bidding, such as Figure 1 As shown, the specific steps are as follows:

[0053] Step 1: Construct an immersive media resource allocation system consisting of one user, an integrated terrestrial-satellite network, and a multimedia server;

[0054] Immersive media resource allocation system such as Figure 2 As shown, the nodes of the space-ground integrated network are represented by the graph G=(V,E), where V represents satellite and ground station nodes, and E represents inter-satellite links and satellite-to-ground links.

[0055] Multimedia servers are used to provide immersive media resources, including virtual games, 360-degree videos, 4K or 8K videos, etc. Each multimedia server has the same capacity and offers M types of immersive media services. For each service type, n services are available. Service types include low QoS and high QoS. Different service types occupy different resource units.

[0056] Step 2: For the current immersive media resources, each user initiates their own request based on their needs;

[0057] The request information from all users is composed of a set B = {b1, b2, ..., b} i ,...,b I}; Each request message b i =(t i ,r i ,d i ) contains the type t of the requested service. i Required resource unit r i and the longest latency d that the user can tolerate i .

[0058] Step 3: Each user submits a bid based on their request information, budget, and estimated value of the current immersive media resources; and observes the previous bids from other users to decide whether to continue bidding or withdraw their bid.

[0059] If other users bid too high, the auction will be withdrawn, but there will be a withdrawal cost D. Alternatively, bidding may continue, but this could be rejected, incurring a rejection cost R. Both D and R are constants.

[0060] When users bid, each user is treated as an agent. Based on a multi-agent reinforcement learning algorithm, the agent's action strategy is updated according to the current state, and the system utility is calculated. The agent's strategy is adjusted according to the calculation results, and the auction rules and agent strategies are adjusted to better adapt to the current bidding environment, so as to further improve network utility, meet the needs of network profit and efficiency, and make the bidding more rational.

[0061] System utility includes two parts: the utility of users participating in the auction and the resource utilization of the multimedia server.

[0062] The utility of a user participating in an auction is expressed as: u = y[(vp)x + R(1-x)] + (1-y)D;

[0063] Where y represents whether the user withdraws from the auction, y=1 indicates withdrawal; if the user withdraws from the auction, a fixed withdrawal cost D needs to be paid; if y=0, the user continues the auction, but may be rejected; x=0 indicates that the auction is unsuccessful and a fixed rejection cost R needs to be paid; x=1 indicates that the auction is successful, p is the user's bid price, v represents the value of the goods, and vp represents the reward the user can receive after the auction is successful.

[0064] The resource utilization rate of a multimedia server represents the current resource usage, denoted by α.

[0065] Therefore, the system utility is expressed as U = ∑ i∈I ∑ m∈M u i,m +K(1-α); K is a constant;

[0066] u i,m This represents the utility of the i-th user for the m-th type of immersive media service;

[0067] The adjustment rules for the agent's strategy are as follows: based on reinforcement learning algorithms, the Actor-Critic algorithm is used to learn user behavior; the specific process is as follows:

[0068] s301: Define the state space of the agent as S(t) = [b i U m,i [(t-1), P(t-1), α], which are the request information b of the i-th user respectively. i The utility U of the i-th user for the m-th type of immersive media service at the previous time step m,i (t-1), the user's bid information P(t-1) at the previous time step, and the system's resource utilization rate α.

[0069] The utility of the i-th user for the m-th type of immersive media service is calculated as follows:

[0070] U m,i =y m,i C m,i +(1-y m,i )D m,i

[0071] y mi Indicates whether the i-th user withdraws their bid for the m-th type of immersive media service; D mi C represents the withdrawal cost incurred when the i-th user withdraws their bid for the m-th type of immersive media service; m,i The cost for the i-th user to participate in the auction for the m-th type of immersive media service is calculated as follows:

[0072] C m,i =xm,i (p m,i -v m,i )+(1-x m,i )R m,i ]

[0073] p m,i v represents the bid price of the i-th user for the m-th type of immersive media service. m,i R represents the value of the m-th type of immersive media business; m,i This represents the cost of the i-th user's bid for the m-th type of immersive media service being rejected;

[0074] s302: Design the action space of the intelligent agent A(t) = [p i (t),y i [(t)], which represent the bid p of the i-th user. i (t) and whether to withdraw this round of auction y i (t).

[0075] s303: Design the agent's reward function R(t) to maximize user utility and resource utilization α:

[0076]

[0077] S304: Use the Actor-critic algorithm to learn user behavior and generate the optimal result.

[0078] Step 4: For the current immersive media resources, the ground station nodes of the integrated space-ground network accept resource request information and bidding information from all users, and sort the users' bids in descending order; a second-price auction method is used to determine the users who win the auction and the users who reject it.

[0079] The number of users who successfully bid will remain the same as the number of existing resources, and the price they need to pay will be the second highest bid.

[0080] Step 5: For all users who successfully bid, the request information of each user is sent to different multimedia servers using the Dijkstra algorithm and load balancing.

[0081] Dijkstra's algorithm is used to find the shortest path between a ground station and a multimedia server under the current network conditions. Since there may be multiple multimedia servers, not all request information will be forwarded to a single server. Load balancing is employed to send user requests to the server.

[0082] The specific steps are as follows:

[0083] S501: Calculate the shortest path between nodes: Use Dijkstra's algorithm to find the shortest path for transmission between the ground station and multiple multimedia servers under the current network conditions.

[0084] S502: Node Load: Calculate the node load α based on the resource utilization of each multimedia server. For example, multimedia server 1 has a resource utilization of 60%, multimedia server 2 has a resource utilization of 70%, and multimedia server 3 has a resource utilization of 50%.

[0085] S503: Determine the load balancing strategy: To achieve dynamic load balancing, we can choose to perform load balancing based on the resource utilization α of each multimedia server. For example, when the resource utilization of a node exceeds a threshold (e.g., 80%), we will transfer requests to other nodes.

[0086] S504: Dynamically Adjust Load Balancing: Based on real-time monitoring of the load on each node, dynamically adjust the load balancing as needed. For example, when the resource utilization of a node decreases, we can reallocate requests to that node.

[0087] Step Six: After receiving the user's request information, each multimedia server releases the resources corresponding to the request information to provide immersive media services; and returns the current resource utilization and task execution results to the ground station node.

[0088] Example:

[0089] First, each user initiates a request, bidding based on their own need for immersive media resources and the behavior of other users. These bids are then submitted to the auction market. In each auction, users attempt to maximize their chances of success by either increasing the bid or withdrawing their bid.

[0090] Then, the network controls and forwards user requests: the ground station is the permission control center that receives user auction information and sorts all users' bidding information. If the auction is permitted, the requested resource is forwarded through the network.

[0091] The successful bidder will be determined according to the auction rules, and the immersive media resources will be allocated to the successful bidder.

[0092] Finally, the shortest path for transmission between the ground station and the multimedia server is found, and the user's request information is sent to the server using load balancing. The server receives the user request information forwarded from the network. If there are too many requests, they need to be queued. The server releases the corresponding resources according to the request information and provides immersive media services. The server then returns the current resource utilization and task execution results to the ground station.

Claims

1. A method for allocating immersive media resources based on auction bidding, characterized in that, The specific steps are as follows: Step 1: Construct an immersive media resource allocation system consisting of one user, an integrated terrestrial-satellite network, and a multimedia server; Step 2: For the current immersive media resources, each user initiates their own request based on their needs; bids are made based on their budget and estimated value of the current immersive media resources; and other users' bids from the previous round are observed to decide whether to continue bidding or withdraw their bids. When users bid, each user is treated as an intelligent agent. Based on a multi-agent reinforcement learning algorithm, the agent's action strategy is updated according to the current state, and the system utility is calculated. And adjust the agent's strategy based on the calculation results to make the bidding more rational; System utility includes two parts: the utility of users participating in the auction and the resource utilization of the multimedia server. The utility of a user participating in an auction is expressed as: u = y[(vp)x + R(1-x)] + (1-y)D; Where y represents whether the user withdraws from the auction, y=1 indicates withdrawal; if the user withdraws from the auction, a fixed withdrawal cost D needs to be paid; if y=0, the user continues the auction, but may be rejected; x=0 indicates that the auction is unsuccessful and a fixed rejection cost R needs to be paid; x=1 indicates that the auction is successful, p is the user's bid price, v represents the value of the goods, and vp represents the reward the user can receive after the auction is successful. The resource utilization rate of a multimedia server represents the current resource usage, denoted by α. Therefore, the system utility is expressed as U = Σ i∈I Σ m∈M u i,m +K(1-α); K is a constant; u i,m This represents the utility of the i-th user for the m-th type of immersive media service; Step 3: For the current immersive media resources, the ground station nodes of the integrated space-ground network receive resource request information and bidding information from all users, and sort the users' bids in descending order; use the second highest bid auction method to determine the users who win the auction and the users who reject it. The number of users who successfully bid will remain the same as the number of existing resources, and the price they need to pay will be the second highest bid. Step 4: For all users who successfully bid, the Dijkstra algorithm and load balancing method are used to send each user's request information to different multimedia servers. Step 5: After receiving the user's request information, each multimedia server releases the resources corresponding to the request information to provide immersive media services; and returns the current resource utilization and task execution results to the ground station node.

2. The immersive media resource allocation method based on auction bidding as described in claim 1, characterized in that, The nodes of the integrated space-ground network are represented by the graph G = (V, E), where V represents satellite and ground station nodes, and E represents inter-satellite links and satellite-to-ground links; Each multimedia server has the same capacity and offers M types of immersive media services. For each service type, n services are available. Service types include low QoS and high QoS. Different service types occupy different resource units.

3. The immersive media resource allocation method based on auction bidding as described in claim 1, characterized in that, The request information from the I users forms a set B = {b1, b2, ..., b} i ,...,b I }; Each request message b i =(t i ,r i ,d i ) contains the type t of the requested service. i Required resource unit r i and the longest latency d that the user can tolerate i .

4. The immersive media resource allocation method based on auction bidding as described in claim 1, characterized in that, In step two, the adjustment rule for adjusting the agent strategy is as follows: based on the reinforcement learning algorithm, the Actor-Critic algorithm is used to learn user behavior; The specific process is as follows: s301: Define the state space of the agent as S(t) = [b i U m,i [(t-1), P(t-1), α], which are the request information b of the i-th user respectively. i The utility U of the i-th user for the m-th type of immersive media service at the previous time step m,i (t-1), the user's bid information P(t-1) at the previous time step, and the system's resource utilization rate α; The utility of the i-th user for the m-th type of immersive media service is calculated as follows: U m,i =y m,i C m,i +(1-y m,i )D m,i y m,i Indicates whether the i-th user withdraws their bid for the m-th type of immersive media service; D m,i C represents the withdrawal cost incurred when the i-th user withdraws their bid for the m-th type of immersive media service; m,i The cost for the i-th user to participate in the auction for the m-th type of immersive media service is calculated as follows: C m,i =x m,i (v m,i -p m,i )+(1-x m,i )R m,i p m,i v represents the bid price of the i-th user for the m-th type of immersive media service. m,i R represents the value of the m-th type of immersive media business; m,i This represents the cost of the i-th user's bid for the m-th type of immersive media service being rejected; s302: Design the action space of the intelligent agent A(t) = [p i (t),y i [(t)], which represent the bid p of the i-th user. i (t) and whether to withdraw this round of auction y i (t); s303: Design the agent's reward function R(t) to maximize user utility and resource utilization α: S304: Use the Actor-critic algorithm to learn user behavior and generate the optimal result.

Images