Space-air-ground scene on-demand service strategy system based on user demand analysis

Abstract

The invention discloses an air-space-ground scene on-demand service strategy system based on user demand analysis, and solves the problems that in the prior art, deep analysis of personalized demands of users is lacked, and accurate resource allocation cannot be realized according to dynamically changing demands. According to the scheme, a user side comprises a data acquisition module, a data processing module and a demand uploading module which are connected in sequence; the user side is used for data acquisition, data preprocessing and demand uploading; the network end comprises a network state acquisition module, a demand analysis module, a strategy calculation module and a scheduling module which are connected in sequence; the network state acquisition module is used for acquiring current network resources in real time; the network end analyzes the demand and obtains a network resource allocation strategy according to an analysis result; and efficient data transmission and dynamic resource configuration are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to an air-space-ground-land scene on-demand service strategy system based on user demand analysis. Background Art

[0002] With the development of integrated space-air-ground networks, the diversification of Internet of Things (IoT) devices and user needs is increasing. These devices and user needs are distributed in different spatial environments and show different service requirements at different times. Existing resource management mechanisms mostly use static resource configuration, which is difficult to adapt to the changing needs and resource status in space-air-ground scenarios in real time.

[0003] With the rapid development of communication networks and information processing technologies, users' demand for services is becoming more diversified and personalized. In order to meet users' dynamic needs under limited network resources, on-demand service strategies have become the key to improving user experience and resource utilization. However, the existing architecture lacks an effective demand parsing mechanism and dynamic resource allocation algorithm, resulting in low efficiency and accuracy in resource allocation. Summary of the invention

[0004] The present invention provides an on-demand service strategy system for air-space-ground scenarios based on user demand analysis, thereby solving the problem of lack of in-depth analysis of user personalized needs in the prior art and inability to achieve accurate resource allocation according to dynamically changing needs, thereby achieving efficient data transmission and dynamic resource configuration.

[0005] The present invention provides an on-demand service strategy system for air-space-ground scenes based on user demand analysis, the system comprising: a user end and a network end, the user end and the network end are connected via a data processing module and a demand analysis module;

[0006] The user end includes a data acquisition module, a data processing module and a demand upload module connected in sequence; the data acquisition module is used to obtain environmental data collected in real time by sensors on each user device, the data processing module is used to preprocess the environmental data to obtain preprocessed environmental data; the demand upload module is used to propose business requirements based on the preprocessed environmental data to obtain user device demand data;

[0007] The network end includes a network status acquisition module, a demand analysis module, a strategy calculation module and a scheduling module which are connected in sequence; the network status acquisition module is used to obtain the current network resources in real time; the demand analysis module performs demand analysis on the user equipment demand data to obtain a analysis result; the strategy calculation module is used to calculate the network resource allocation strategy based on the analysis result and the pre-processed environmental data; the scheduling module is used to schedule the air-space-ground network resources according to the network resource allocation strategy; wherein the network resources include: land-based network, air-based network and space-based network.

[0008] In a possible implementation, the data acquisition module performs data preprocessing on the environmental data to obtain preprocessed environmental data, including: performing data screening, time redundancy elimination and space redundancy elimination on the environmental data to obtain preprocessed environmental data.

[0009] In a possible implementation, the demand parsing module performs demand parsing on the user equipment demand data, including: performing data classification and data priority allocation on the user equipment demand data by utilizing a layered architecture.

[0010] In a possible implementation, the strategy calculation module calculates the network resource allocation strategy according to the analysis result and the pre-processed environment data, including:

[0011] respectively determining a first data transmission rate between the user terminal and the air-based network and a second data transmission rate between the user terminal and the land-based network;

[0012] Determine a third interference plus noise ratio of a signal between the land-based network and the user terminal, and calculate a third data transmission rate according to the third interference plus noise ratio;

[0013] Determine a fourth interference plus noise ratio of a signal between the air-based network and the user terminal, and calculate a fourth data transmission rate according to the fourth interference plus noise ratio;

[0014] Determine a fifth interference plus noise ratio of a signal between the space-based network and the user terminal, and calculate a fifth data transmission rate according to the fifth interference plus noise ratio;

[0015] Calculate the actual consumption time of the parsing result according to the first data transmission rate, the second data transmission rate, the third data transmission rate, the fourth data transmission rate, and the fifth data transmission rate;

[0016] Determine a temporary reward function for each user device according to the actual consumption time, and determine a long-term reward function for each user device according to the temporary reward function;

[0017] Determine a first network environment state at time slot t according to an access capacity constraint function of the land-based network and the air-based network and a storage capacity constraint function of the land-based network and the air-based network;

[0018] Optimizing the long-term reward function according to the first network environment state at time slot t to obtain an optimized reward function;

[0019] Determine the optimal strategy for the current user equipment and the optimal strategies for the remaining user equipments according to the first network environment state at time slot t and the second network environment state at time slot t+1;

[0020] The optimization reward function is optimized according to the optimal strategy of the current user equipment and the optimal strategies corresponding to the remaining user equipment to obtain a final reward function, and the final reward function is solved to obtain a network resource allocation strategy for each user equipment.

[0021] In a possible implementation, the temporary reward function is expressed as:

[0022]

[0023] in, Indicates the maximum delay that a user device can accept when requesting a service; T i (t) represents the time consumed by the actual task; θ1 represents the cost weight; θ2 represents the weight of the overall service rate Rate; C B represents the cost of using the land-based network; C V represents the cost of using the air-based network; Rate represents the overall service rate; Indicates; N V express; Indicates; N B express.

[0024] In a possible implementation, the long-term reward function is expressed as:

[0025]

[0026] Where λ represents the first constant; τ represents the time slot index of the time step; r i (t+τ+1) represents; t represents the time step; v i (t) represents the long-term reward of the i-th user device in time slot t.

[0027] In a possible implementation, the optimization reward function is expressed as:

[0028]

[0029] Where, Ε[·] represents the expected operation; s i Indicates the current environment status; v i (s i ,π) indicates that in s i The optimal reward of taking action π in the state; λ represents the first constant; τ represents the time slot index of the time step; t represents the time step; s i (t) represents the state function; r i (t+τ+1) represents the temporary reward at time slot t+τ+1.

[0030] In a possible implementation, the final reward function is expressed as:

[0031]

[0032] Wherein, Ε[·] represents the expected operation; λ represents the first constant; r i (·) represents the temporary reward corresponding to the i-th user device; τ represents the time slot index of the time step; t represents the time step; s i (t+τ) represents; π i Indicates the policy of the current user device; π -i Indicates the policies for other user devices; i (t) represents the state function.

[0033] In a possible implementation manner, the network end further includes: a feedback module, which feeds back the network resource allocation strategy to the strategy calculation module through a feedback mechanism.

[0034] One or more technical solutions provided in the present invention have at least the following technical effects or advantages:

[0035] (1) The present invention adopts a collaborative architecture design of user-side modules and network-side modules. The user-side is responsible for efficient data preprocessing and optimization, while the network-side performs in-depth demand analysis and resource allocation. The two form an end-to-end closed-loop system through close interaction, achieving efficient data transmission and dynamic resource allocation;

[0036] (2) The user-end module of the present invention has a built-in multi-level data preprocessing algorithm library, which can realize preprocessing functions such as time redundancy elimination and space redundancy optimization. By optimizing before data transmission, the amount of transmitted data is significantly reduced and the network burden is reduced;

[0037] (3) The present invention deploys a demand analysis module on the network side, and uses data mining and machine learning algorithms to accurately identify demand based on user data and historical demand information. The on-demand service submodule can dynamically adjust the resource allocation strategy according to the demand analysis results to achieve efficient coordination of cross-domain (air, ground, and space-based) resources;

[0038] (4) The on-demand service submodule of the present invention integrates a dynamic link selection algorithm and a resource allocation algorithm library, supports real-time calculation of the best link and dynamic adjustment based on the availability of network resources. This design ensures that the system's service quality and efficiency can be maximized even when resources are tight. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of an on-demand service strategy system for air-ground-space scenarios based on user demand analysis provided by an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of the physical network-virtual network-scheduling mechanism of the present invention provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] An on-demand service strategy system for air-space-ground scenarios based on user demand analysis, such as Figure 1 As shown, the system includes: a user end and a network end, and the user end and the network end are connected through a data processing module and a demand parsing module.

[0043] The user end includes a data acquisition module, a data processing module and a demand uploading module which are connected in sequence.

[0044] The data acquisition module is used to obtain environmental data collected in real time by sensors on each user device;

[0045] Exemplarily, the user-side module is deployed on user devices, such as vehicle-mounted terminals, smart phones, etc. The module is initialized when it starts, including hardware resource detection and network connection configuration, to ensure that the device has the ability to collect and preprocess data. The network-side module consists of multiple functional sub-modules and is deployed on edge computing nodes or cloud platforms.

[0046] By calling the local sensor interface, the user-side module collects a variety of user data, such as location information, environmental images, device status, etc. This module dynamically adjusts the intensity of data collection based on the preset collection frequency and data volume control strategy to ensure energy saving of the device.

[0047] User devices (such as smart vehicles, mobile terminals, IoT devices, etc.) collect environmental data in real time through built-in sensors, such as location, speed, road conditions, neighboring vehicle information, etc. For example, mobile terminals (such as smart phones, tablets, etc.) use built-in sensors (GPS, accelerometers, cameras, etc.) to collect user location, motion status, network connection status, application usage and other data in real time. Smart vehicles collect road conditions, traffic flow, vehicle speed, location and other data through on-board sensors (such as radar, cameras, LIDAR, etc.), and evaluate the vehicle's perception ability and network connection status.

[0048] The data processing module is used to perform data preprocessing on the environmental data to obtain preprocessed environmental data; here, data preprocessing is performed on the environmental data to obtain preprocessed environmental data, including: data screening, time redundancy elimination and space redundancy elimination of the environmental data to obtain preprocessed environmental data.

[0049] Exemplarily, the built-in data preprocessing mechanism is optimized at the architecture level, including data screening, redundancy elimination, and data compression. This design simplifies the data processing process and ensures the structural optimization of data before transmission through modular preprocessing to reduce the use of communication bandwidth. The user-side module establishes a secure communication channel with the network-side module and uses a lightweight protocol for data transmission. The architecture design ensures that user data can be quickly and stably transmitted to the network-side module under different network environments, reducing transmission delays.

[0050] The demand upload module is used to propose business requirements based on the pre-processed environmental data and obtain user equipment demand data;

[0051] For example, the user equipment proposes its own requirements for network services based on the data collected in real time, including bandwidth for data transmission, accuracy of perceived data, delay tolerance, continuity of service, etc.

[0052] In a specific embodiment provided by the present invention, in the smart city system, each intelligent connected vehicle user Ui is regarded as an intelligent entity, and base stations, drones, and satellites are called service providers, corresponding to land-based networks, air-based networks, and space-based networks, respectively. When a user requests a service from a service provider, the user device demand data and pre-processed environment data need to be uploaded to the service provider.

[0053] The network end includes a network status acquisition module, a demand analysis module, a policy calculation module, and a scheduling module, which are connected in sequence. The network end module consists of multiple functional sub-modules and is deployed on edge computing nodes or cloud platforms. This architecture design manages computing resources in a distributed manner and automatically selects the optimal service node based on user location and data traffic, thereby improving overall service efficiency. The network end module integrates the state perception function of air, ground, and space-based resources, and collects resource availability and performance data in real time. This function is achieved through a distributed monitoring architecture, which can dynamically perceive the load and remaining capacity of resources and provide real-time basis for resource allocation.

[0054] The network status acquisition module is used to obtain current network resources in real time.

[0055] The demand analysis module performs demand analysis on the user equipment demand data to obtain an analysis result; here, the demand analysis on the user equipment demand data includes: using a layered architecture to perform data classification and data priority allocation on the user equipment demand data.

[0056] Here, the demand analysis module analyzes user needs based on a hierarchical architecture. The first layer performs preliminary data classification, and the second layer performs detailed analysis and demand priority allocation. Through this hierarchical architecture design, the system can quickly respond to high-priority needs while ensuring that low-priority tasks are handled reasonably.

[0057] Depending on the characteristics of different devices, requirements are divided into multiple categories, such as:

[0058] (1) For mobile terminals, requirements include high-definition video streaming, real-time navigation, emergency calls, etc.

[0059] (2) For IoT devices, requirements include low-latency device status upload, real-time environmental monitoring, etc.

[0060] (3) For intelligent vehicles, requirements include high-precision environmental perception, real-time communication, etc.

[0061] Assume that there are intelligent connected vehicle users U, the i-th user U i In the jth time slot, the network requests to watch a high-definition movie. After the user demand analysis submodule, the user U i The demand is expressed as: Among them, β i,j Indicates the amount of data required for the requested service, T i,j Indicates the actual time cost required to complete the task. Indicates U i The maximum acceptable delay when requesting this service.

[0062] The strategy calculation module is used to calculate the network resource allocation strategy based on the analysis results and the pre-processed environmental data. In view of the high cost of satellite communication and the long service delay, when base stations and drones cannot meet the needs of users, they will seek the intervention of space-based networks.

[0063] Here, the network resource allocation strategy is calculated based on the analysis results and the pre-processed environment data, such as Figure 2 As shown, the process includes the following steps S1 to S9.

[0064] S1, respectively determining a first data transmission rate between a user terminal and an air-based network and a second data transmission rate between a user terminal and a land-based network;

[0065] Here, the first data transmission rate is expressed as:

[0066]

[0067] w i UE is the bandwidth of Ui, p i UE is the transmit power of Ui, PL i,V is the path loss between Ui and the UAV, is the white noise power

[0068] The second data transmission rate is expressed as:

[0069]

[0070] Among them, PL i,B is the path loss between Ui and the UAV, is the Gaussian white noise power in the network.

[0071] S2, determining a third interference plus noise ratio of a signal between the land-based network and the user terminal, and calculating a third data transmission rate according to the third interference plus noise ratio; determining a fourth interference plus noise ratio of a signal between the air-based network and the user terminal, and calculating a fourth data transmission rate according to the fourth interference plus noise ratio; determining a fifth interference plus noise ratio of a signal between the space-based network and the user terminal, and calculating a fifth data transmission rate according to the fifth interference plus noise ratio;

[0072] Here, the interference plus noise ratio of the signal between the land-based network and the user end is expressed as:

[0073]

[0074] Among them, p B represents the transmission power of the base station, χ B,i represents the distance between the base station and the user, θ B,i Indicates the base station and Ui The path loss exponent of the link between Represents additive white Gaussian noise in the network.

[0075] Therefore, the third data transmission rate between the land-based network and the user equipment can be obtained:

[0076]

[0077] Among them, N B Indicates the number of users accessing the base station in the same time slot. It means that the bandwidth obtained by users accessing the base station is related to the number of accesses, and an average distribution strategy is implemented.

[0078] When the resources of land-based networks and air-based networks cannot meet the needs of ground users, land-based networks and air-based networks need to seek help from satellite networks. L,B The fourth data transmission rate between the space-based network and the ground-based network data is expressed as:

[0079]

[0080] Among them, W L represents the communication bandwidth of the satellite, P L represents the satellite's transmission power, h represents the channel gain, Represents additive white Gaussian noise in the network.

[0081] Similarly, the interference plus noise ratio and the fifth data transmission rate of the signal between the space-based network and the user end are expressed as:

[0082]

[0083] Where PL' represents the path loss, represents the white noise power.

[0084] S3, calculating the actual consumption time of the parsing result according to the first data transmission rate, the second data transmission rate, the third data transmission rate, the fourth data transmission rate, the fourth data transmission rate and the fifth data transmission rate;

[0085] S4, determining a temporary reward function for each user device according to the actual consumption time, and determining a long-term reward function for each user device according to the temporary reward function;

[0086] S5, determining a first network environment state at time slot t according to an access capacity constraint function of the land-based network and the air-based network and a storage capacity constraint function of the land-based network and the air-based network;

[0087] S6, optimizing the long-term reward function according to the first network environment state at time slot t to obtain an optimized reward function;

[0088] S7, determining an optimal strategy for the current user equipment and optimal strategies corresponding to other user equipments according to the first network environment state at time slot t and the second network environment state at time slot t+1;

[0089] S8, optimizing the optimization reward function according to the optimal strategy of the current user device and the optimal strategies corresponding to the remaining user devices to obtain a final reward function, solving the final reward function to obtain a network resource allocation strategy for each user device.

[0090] Exemplarily, in order to model the uncertainty of a random environment, the problem of a vehicle user selecting a resource provider is formulated as a random game.

[0091] In view of the diverse business needs of user Ui in the Internet of Vehicles system, the goal of the present invention is to select a suitable service provider in the integrated air-ground-space network, and provide services to as many users as possible under the constraints of latency and economic cost, so as to achieve the highest service rate of the overall system. In the method provided by the present invention, the overall service rate of the system is the most critical issue.

[0092] User Ui selects its own service provider at time slot t based on the current state of the environment, combined with its own business needs and latency requirements, and then receives a reward to evaluate the performance of the selected action. Therefore, the design of the reward function will directly guide the learning process. In the method, the temporary reward function of user Ui is defined as:

[0093]

[0094] in, Indicates the maximum delay that a user device can accept when requesting a service; T i (t) represents the time consumed by the actual task; θ1 represents the cost weight; θ2 represents the weight of the overall service rate Rate; C B represents the cost of using the land-based network; C V represents the cost of using the air-based network; Rate represents the overall service rate; represents the probability of the space-based network providing service, or N V Indicates the number of air-based network accesses; represents the probability of providing service through the land-based network, or N B represents the number of land-based network accesses; i represents the i-th user equipment.

[0095] The overall service rate Rate of the system is expressed as:

[0096]

[0097] Among them, T i (t) represents the actual time consumed by the service provider when the user requests the current service.

[0098] The immediate reward of user Ui in any time slot t mainly depends on the information he observes.

[0099] 1) Observed information: the type of service M currently requested, the action A taken (the selected service provider), the number of Ui accesses in a time slot on the base station / drone, the storage capacity of the base station / drone, and the current system service rate.

[0100] 2) Unobserved information: actions taken by other users Ui in the network and their benefits.

[0101] Next, consider maximizing the long-term reward v by choosing the appropriate action at each time slot i (t).

[0102] Specifically, at a certain time slot in the process, the discounted reward is the sum of its rewards in the current time slot, plus the sum of future rewards multiplied by a constant factor. Therefore, the user's long-term reward function is expressed as:

[0103]

[0104] Where λ represents the first constant; τ represents the time slot index of the time step; r i (t+τ+1) represents the temporary reward of time slot t+τ+1; t represents the time step; v i (t) represents the long-term reward of the i-th user device in time slot t.

[0105] Specifically, the value of λ reflects the impact of future rewards on the optimal decision: if λ is close to 0, it means that the decision emphasizes short-term gains; in contrast, if λ is close to 1, more weight is given to future rewards, and the decision is said to be far-sighted.

[0106] Each user's action space A i ∈{0,1,2} respectively represent: Ui abandons the service request, selects the land-based network for service, and selects the air-based network for service. At any time slot t, the goal of each user is to take the best action a * i (t)∈A i , so that the long-term reward in formula (9) is maximized. Therefore, the optimal matching problem for the final user Ui, i∈I can be formulated as:

[0107]

[0108] Among them, a i represents the action corresponding to the i-th user device; v i (t) represents the long-term reward of the i-th user device; t represents the time step; a * i (t) represents the best action corresponding to the i-th user equipment; A i Represents a collection of actions.

[0109] Note that the resource on-demand matching problem considered consists of I sub-problems, which correspond to I different users.

[0110] In addition, each user has no information about other users, such as actions taken, rewards obtained, and in the considered network, it is assumed that all users Ui are selfish and rational. Therefore, at any time slot t, all users observe the current environment state s i (t)∈S i , non-cooperatively choose their actions a i (t)∈A i , to maximize the long-term reward in (9).

[0111] Stochastic games are a generalization of Markov decision processes in the multi-agent setting, also called Markov games, represented by the tuple:<I,S,A,P,R> I is the number of users; S is the state set including each user's state; A is the action set A = A1×A2×…×A U , A i is the action set of user Ui; P:S×A×S∈[0,1] is the state transition probability function; R={R1,R2,…,R U}Contains rewards for all users.

[0112] Since user Ui accesses the base station or drone, the service provider's resources are evenly distributed, and the resources obtained by each user are related to the number of accessed users and the capacity limit of the service provider itself. Changes in the environmental state will affect the choice of user Ui. Since there is no cooperation between competing users Ui, the environmental state is defined based on the local observation of each user Ui. At time slot t, the environmental state observed by user Ui is given by the following formula:

[0113] s i (t)=(M i (t),Ρ(t),Q(t)) (11)

[0114] Among them, M i(t)={0,1,2} represents the service type requested by Ui in time slot t, which respectively means that Ui requests data upload service, computing service, and storage service.

[0115] Since the resource provider implements a resource sharing strategy, the computing resources and bandwidth resources allocated to each user are related to the number of Ui accesses. Too many Ui choosing the same resource provider will result in a reduction in the resources obtained, and the computing and communication time will be greatly extended, thus affecting the overall efficiency. Therefore, it is necessary to set the maximum number of accesses allowed by the base station / drone. Ρ(t)∈{0,1,2} indicates whether the number of Ui accesses to the base station or drone has reached the peak, expressed as:

[0116]

[0117] Among them, N B max ,N V max Respectively represent the maximum number of users allowed by the base station and the drone. P(t) = 0 means that the base station and drone capacity have not reached the access peak, P(t) = 1 means that the base station access capacity is full, and P(t) = 2 means that the drone access capacity is full.

[0118] Q(t)∈{0,1,2} represents the storage capacity limit of the base station and the drone:

[0119]

[0120] Among them, β i (t) represents the amount of data uploaded by user Ui in time slot t, λ B ,λ V They represent the cache capacity of the base station and the UAV respectively.

[0121] π i (s i ,a i ) is the mapping from state to action, π i (s i ,a i )=Pr(a t =a|s t =s)∈[0,1] gives the i Take action i In other words, for every state s i ∈S i Ui has many different strategy choices, which can be expressed as: π i (s i )={π i (s i ,a i )|ai ∈A i In a random game, the joint strategy of I players is defined as the strategy vector π = (π1(s1), π2(s2), …, π I (s I )). Based on the above discussion, in the formulated stochastic game, the optimization goal of each user Ui is to maximize its expected return over time. Therefore, the optimization goal in formula (7) can be restated as:

[0122]

[0123] Where, Ε[·] represents the expected operation; s i Indicates the current environment status; v i (s i ,π) indicates that in s i The optimal reward of taking action π in the state; λ represents the first constant; τ represents the time slot index of the time step; t represents the time step; s i (t) represents the state function; r i (t+τ+1) represents the temporary reward at time slot t+τ+1.

[0124] From the environmental state i (t) to the new state s i The state transition at (t+1) is determined by the joint strategy of all users. In addition, in the non-cooperative game, at each time slot t, each user is in state s i (t) independently chooses its strategy to maximize the optimization reward v i (s i ,π i ), and then receives its current personal reward based on the joint policy (action set) π. Therefore, the user device cannot be simply expected to maximize the corresponding expected reward.

[0125] Here, the goal of each user is to i ∈S i Learn the optimal strategy π i * , the optimal strategy of other users is learned as π -i * =(π1 * ,π2 * ,…,π i * ,…,π U * ), then the expected return can be restated as formula (15). A solution of the stochastic game is described by Nash equilibrium, which is defined by Definition 1.

[0126] The final reward function is expressed as:

[0127]

[0128] Order i (t) = s i ,s i (t+1)=s i ', formula (15) can be split and transformed into:

[0129]

[0130] Wherein, Ε[·] represents the expected operation; λ represents the first constant; r i (·) represents the temporary reward corresponding to the i-th user device; τ represents the time slot index of the time step; t represents the time step; s i (t+τ) represents; π i Indicates the policy of the current user device; π -i Indicates the policies for other user devices; i (t) represents the state function.

[0131] A Nash equilibrium is a set of I optimal strategies, (π1 * ,π2 * ,…,π I * ), where no user can obtain any higher reward by changing only their own strategy. That is, for each user Ui∈I, in each state s i ∈S i ,have:

[0132]

[0133] Among them, i It is a set of strategies that user Ui can adopt.

[0134] That is to say, there is always a NE strategy in the formulated on-demand matching game problem. In the network, each user gets his own optimal strategy, and no end user can get a better strategy by changing his strategy. Therefore, in this method, the purpose of each user device Ui is to i Find a NE optimal strategy.

[0135] Each user observes its local information o i (t) and get your own reward r i(t). Since the future state S(t+1) depends only on the current state S(t) and the action A(t) taken, this dynamic multi-agent reinforcement learning process has the Markov property, so it is formulated as a stochastic game. Specifically, the stochastic game of a single user is modeled as a Markov decision (MDP) process.

[0136] Q-learning is applied to solve the independent MDP problem, and a MA-Q (Multi-agent q-learning) learning algorithm based on independent learning is proposed to solve the problem of on-demand resource matching for multi-user and multi-service in SAGIN network. In the proposed algorithm, each user runs an independent Q-learning algorithm and learns a separate optimal strategy for their MDP at the same time. Specifically, the selection of the optimal action depends on the Q function. According to the definition of Q-learning, the Q value is updated according to formula (18):

[0137]

[0138] Where α is the step size (learning rate), 0<α≤1; s i ' is the next state observed after the current state executes the behavior strategy.

[0139] In order to ensure the convergence of Q learning, the learning rate is set to:

[0140]

[0141] Among them, α begin , α end are the initial and final values ​​of the given α, and iterations is the maximum number of iterations of the learning algorithm.

[0142] The scheduling module is used to schedule air-space-ground network resources according to the network resource allocation strategy; wherein the network resources include: land-based network, air-based network and space-based network.

[0143] The network end also includes: a feedback module, which feeds back the network resource allocation strategy to the strategy calculation module through a feedback mechanism.

[0144] During system operation, the feedback module will continuously optimize the resource allocation strategy through the feedback mechanism to ensure task priority adjustment and maximum resource utilization. If improper resource utilization or execution failure is found, adjustments will be made through the feedback mechanism. The resource allocation strategy is dynamically adjusted based on real-time feedback information. For example, if the load of a resource is too high, some tasks will be adjusted to other resources to optimize system performance.

[0145] The method provided by the present invention realizes an on-demand service architecture of air-ground collaboration through multi-level demand analysis and resource allocation strategies. The system can not only analyze user needs in real time and quantify them into specific KPIs, but also perform intelligent resource allocation through advanced algorithms to ensure efficient use of network bandwidth, computing resources and equipment endurance, thereby improving the overall traffic management system.

[0146] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. All or part of the present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, mobile communication terminals, multi-processor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0147] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the present invention.

Claims

1. An on-demand service strategy system for air-ground scenes based on user demand analysis, characterized in that: It includes: a user end and a network end, wherein the user end and the network end are connected via a data processing module and a demand analysis module; The user end includes a data acquisition module, a data processing module and a demand upload module connected in sequence; the data acquisition module is used to obtain environmental data collected in real time by sensors on each user device, and the data processing module is used to perform data preprocessing on the environmental data to obtain preprocessed environmental data; The demand uploading module is used to propose business requirements according to the pre-processed environment data to obtain user equipment demand data; The network end includes a network status acquisition module, a demand analysis module, a strategy calculation module and a scheduling module which are connected in sequence; The network status acquisition module is used to obtain current network resources in real time; The demand analysis module is used to analyze the demand data of the user equipment to obtain an analysis result; the strategy calculation module is used to calculate the network resource allocation strategy according to the analysis result and the pre-processed environment data; The scheduling module is used to schedule air-space-ground-space network resources according to the network resource allocation strategy; wherein the network resources include: land-based network, air-based network and space-based network.

2. The air-ground scene on-demand service strategy system based on user demand analysis according to claim 1 is characterized in that: The data acquisition module performs data preprocessing on the environmental data to obtain preprocessed environmental data, including: performing data screening, time redundancy elimination and space redundancy elimination on the environmental data to obtain preprocessed environmental data.

3. The on-demand service strategy system for air-space-ground scenarios based on user demand analysis according to claim 1 is characterized in that: The demand analysis module performs demand analysis on the user equipment demand data, including: utilizing a layered architecture to perform data classification and data priority allocation on the user equipment demand data.

4. The air-ground scene on-demand service strategy system based on user demand analysis according to claim 1 is characterized in that: The strategy calculation module calculates the network resource allocation strategy according to the analysis result and the pre-processed environment data, including: respectively determining a first data transmission rate between the user terminal and the air-based network and a second data transmission rate between the user terminal and the land-based network; Determine a third interference plus noise ratio of a signal between the land-based network and the user terminal, and calculate a third data transmission rate according to the third interference plus noise ratio; Determine a fourth interference plus noise ratio of a signal between the air-based network and the user terminal, and calculate a fourth data transmission rate according to the fourth interference plus noise ratio; Determine a fifth interference plus noise ratio of a signal between the space-based network and the user terminal, and calculate a fifth data transmission rate according to the fifth interference plus noise ratio; Calculate the actual consumption time of the parsing result according to the first data transmission rate, the second data transmission rate, the third data transmission rate, the fourth data transmission rate and the fifth data transmission rate; Determine a temporary reward function for each user device according to the actual consumption time, and determine a long-term reward function for each user device according to the temporary reward function; Determine a first network environment state at time slot t according to an access capacity constraint function of the land-based network and the air-based network and a storage capacity constraint function of the land-based network and the air-based network; Optimizing the long-term reward function according to the first network environment state at time slot t to obtain an optimized reward function; Determine the optimal strategy for the current user equipment and the optimal strategies for the remaining user equipments according to the first network environment state at time slot t and the second network environment state at time slot t+1; The optimization reward function is optimized according to the optimal strategy of the current user equipment and the optimal strategies corresponding to the remaining user equipment to obtain a final reward function, and the final reward function is solved to obtain a network resource allocation strategy for each user equipment.

5. The air-ground scene on-demand service strategy system based on user demand analysis according to claim 4 is characterized in that: The temporary reward function is expressed as: in, Indicates the maximum delay that a user device can accept when requesting a service; T i (t) represents the time consumed by the actual task; θ1 represents the cost weight; θ2 represents the weight of the overall service rate Rate; C B represents the cost of using the land-based network; C V represents the cost of using the air-based network; Rate represents the overall service rate; N represents the probability of providing service through air-based network; V Indicates the number of air-based network accesses; represents the probability of providing service through land-based network; N B represents the number of land-based network accesses; i represents the i-th user equipment.

6. The air-ground scene on-demand service strategy system based on user demand analysis according to claim 5 is characterized in that: The long-term reward function is expressed as: Where λ represents the first constant; τ represents the time slot index of the time step; r i (t+τ+1) represents the temporary reward of time slot t+τ+1; t represents the time step; v i (t) represents the long-term reward of the i-th user device in time slot t.

7. The air-ground scene on-demand service strategy system based on user demand analysis according to claim 4 is characterized in that: The optimization reward function is expressed as: Where, Ε[·] represents the expected operation; s i Indicates the current environment status; v i (s i ,π) indicates that in s i The optimal reward of taking action π in the state; λ represents the first constant; τ represents the time slot index of the time step; t represents the time step; s i (t) represents the state function; r i (t+τ+1) represents the temporary reward at time slot t+τ+1.

8. The air-ground scene on-demand service strategy system based on user demand analysis according to claim 4 is characterized in that: The final reward function is expressed as: Wherein, Ε[·] represents the expected operation; λ represents the first constant; r i (·) represents the temporary reward corresponding to the i-th user device; τ represents the time slot index of the time step; t represents the time step; s i (t+τ) represents; π i Indicates the policy of the current user device; π -i Indicates the policies for other user devices; i (t) represents the state function.

9. The air-ground scene on-demand service strategy system based on user demand analysis according to claim 1 is characterized in that: The network end further includes: a feedback module, which feeds back the network resource allocation strategy to the strategy calculation module through a feedback mechanism.

Images