An intelligent resource allocation method and system for power terminals in an ISAC heterogeneous service scenario

Abstract

The application discloses a kind of ISAC heterogeneous service scene under power terminal intelligent resource allocation method and system, the method includes according to CT link channel model calculation CT user transmission rate, using finite block length channel coding scheme to solve ST service can be realized transmission rate, analysis ST delay demand, design ST outage probability constraint, introduce proportion fair algorithm, build CT and ST service collaborative resource allocation model, set corresponding Markov model parameter, the Markov model parameter includes state space, action space and reward function, utilize the double-delay deep deterministic policy gradient algorithm based on proportion fair, solve optimal resource allocation scheme;The application is a kind of highly reliable scheduling mechanism, in the case where resource is limited, different service heterogeneous demand can be effectively met, while satisfying the low delay demand of ST service, the data rate of CT service is maximized.

Description

TECHNICAL FIELD

[0001] The application relates to a resource allocation method and system, in particular to an intelligent resource allocation method and system for a power terminal in an ISAC heterogeneous service scenario. BACKGROUND

[0002] With the advent of 6G technology, the smart grid will usher in a new era of deep integration of perception, artificial intelligence and communication, including the combination of artificial intelligence and communication, integrated sensing and communication, and ubiquitous connectivity. In this pattern of interconnection, human connection and intelligent connection, ISAC unifies sensing and communication functions in a more efficient way by using a single waveform and a single device, greatly enhancing the 6G network capability through integrated sensing and communication, and providing strong support for scenarios such as optimized operation, fault detection and energy scheduling of the smart grid.

[0003] In the smart grid, ISAC technology can improve the real-time perception capability of the power grid to the environment and the state of the equipment, and enhance the reliability of the communication network. There are different types of service traffic in the ISAC network, and the CT task includes data interaction in distributed energy management and transmission of household power load information. This type of service requires high throughput and high bandwidth network capability. The ST task includes power grid fault positioning and power equipment state monitoring, and this type of service has low delay and high reliability requirements.

[0004] Existing researches mostly focus on the optimization of a single task, and lack of mechanisms for fair allocation of multiple communication tasks, which cannot balance the fairness of heterogeneous tasks. In particular, in the case of limited 6G network resources, how to balance the high throughput and high bandwidth requirements of communication tasks and the low delay and high reliability requirements of sensing tasks has become a key challenge. In addition, the fairness of users between different communication tasks also needs to be fully considered to avoid significant decline in the quality of service of some users. SUMMARY

[0005] The application aims to provide an intelligent resource allocation method for a power terminal in an ISAC heterogeneous service scenario to balance the low delay of ST and the reliability of CT while maximizing the average data rate of CT services. On the other hand, the application aims to provide an intelligent resource allocation system for a power terminal in an ISAC heterogeneous service scenario.

[0006] Technical scheme: The intelligent resource allocation method for a power terminal in an ISAC heterogeneous service scenario comprises the following steps:

[0007] (1) calculating the CT user transmission rate according to the CT link channel model;

[0008] (2) solving the ST service achievable transmission rate by using a finite block length channel coding scheme;

[0009] (3) Analyze the ST delay requirements and design ST interruption probability constraints;

[0010] (4) Introduce the proportional fairness algorithm to construct a resource allocation model for CT and ST business collaboration;

[0011] (5) Set the corresponding Markov model parameters, which include the state space, action space and reward function;

[0012] (6) Solve the optimal resource allocation scheme using the gradient algorithm of dual-delay deep deterministic strategy based on proportional fairness.

[0013] Preferably, the CT link channel model formula in step 1 is as follows:

[0014]

[0015] in, ω represents the transmission rate of the i-th CT user on resource block b in time slot t. i,b (t) represents the total number of perforated mini-time slots. ω i,b (t)∈{0,1,…,K}, where K represents the total number of mini-slots, and η i,b,k (t) represents the perforation status of the CT user in the k-th mini-time slot resource block b in time slot t, η i,b,k (t) = 1 indicates that the hole has been pierced, and 0 indicates otherwise;

[0016]

[0017] in, σ represents the signal-to-noise ratio of CT user i in resource block b in time slot t. 2 Indicates noise power. This represents the downlink transmission power of the i-th CT user on resource block b in time slot t. Indicates the transmission channel gain. and This indicates interference from other CT and ST users;

[0018]

[0019] in, x represents the downlink transmission rate of the i-th CT user across all resource blocks after punching. i,b (t) = 1 indicates that resource block b has been allocated to the i-th CT user, and 0 otherwise.

[0020] Preferably, the formula for the finite block length channel coding scheme in step 2 is as follows:

[0021]

[0022] where, Rj(t) denotes achievable transmission rate of the jth ST user at time slot t, Sj(t) denotes signal-to-noise ratio of the ST user, Q denotes the number of symbols in a mini-slot, -1 (θ) denotes the inverse Gaussian function, θ denotes the transmission error probability, denotes channel dispersion, represents the randomness of the user channel;

[0023]

[0024] Preferably, the ST outage probability constraint in step 3 is as follows:

[0025]

[0026] where, V(t) denotes the total number of ST packets arrived in time slot t, V(t) =∑ k∈K V k (t) is calculated, V k (t) denotes the number of ST packets arrived in mini-slot k, s denotes the size of ST packet.

[0027] Preferably, the proportional fair algorithm formula in step 4 is as follows:

[0028]

[0029] where, Rj(t) denotes the instantaneous data rate of the jth CT user in time slot t, denotes the historical average throughput at time slot t, which is updated as follows:

[0030]

[0031] where, T denotes the time window length for calculating the average throughput.

[0032] Preferably, the resource allocation model formula in step 4 is as follows:

[0033]

[0034] where, P max denotes the maximum transmission power of the base station, x denotes the RB allocation matrix, p denotes the power allocation vector, η denotes the mini-slot puncturing number matrix, and the objective of the optimization problem is to find the optimal values of the above three parameters x * , p * and η * which maximize the average CT transmission rate considering the fairness of CT users;

[0035] denotes that one resource block is only associated with one single user;

[0036] denotes that the sum of transmission power is guaranteed not to exceed the maximum transmission power of the base station;

[0037] denotes that the reliability of ST service is guaranteed, ω i,b (t)∈{0,1,…,K} denotes the range of the number of puncturing time slots of resource block b.

[0038] Preferably, the state space and action space in step 5 are as follows:

[0039] State space: s(t) = {g e (t), g u (t), λ, I, J}, where g e (t) and g u (t) represent the channel conditions of ST and CT users respectively, λ represents the Poisson arrival rate of ST traffic, and I and J represent the number of CT and ST users respectively;

[0040] Action space: a(r) = {x, p, η}, where x represents resource block allocation, p represents transmission power allocation, and η represents ST scheduling strategy.

[0041] Preferably, the reward function in step 5 is as follows:

[0042] R(t) represents the sum of the weights of R e (t) and R u (t);

[0043]

[0044] where R e (t) represents the demand of CT users, and R u (t) represents the reliability demand of ST.

[0045] Preferably, the specific steps of solving the optimal resource allocation scheme in step 6 are as follows:

[0046] (61) Initialize the state space s t , the training space θ1, θ2, φ, the target network θ1, θ2, φ, the experience replay pool R, and the delay update parameter d;

[0047] (62) Select action a t according to the policy network plus noise ε ~ N(0.6) at each time step;

[0048] (63) According to action a tReceive reward r t and the next state s t+1 , will (s t ,a t ,r t ,s t+1 Store it in the experience replay pool R;

[0049] (64) Randomly sample N data points (s) from the experience replay pool R. t ,a t ,r t ,s t+1 );

[0050] (65) Use the Bellman equation to calculate the objective value y = r + γmin i＝1,2 Q θi′ (s′,a′), where γ represents the discount factor;

[0051] (66) Use the loss function loss=E[(yQ) θi (s,a)) 2 Update the critic network;

[0052] (67) Calculate the gradient every d time steps. Update the actor network;

[0053] (68) Update the target network parameters using a soft update method: θ′ i =τθ i +(1-τ)θ′ i , i=1,2 and φ'=τφ+(1-τ)φ', τ<<1;

[0054] (69) Repeat steps 62 to 69 until the maximum number of training steps T is reached.

[0055] The present invention discloses an intelligent resource allocation system for power terminals in an ISAC heterogeneous service scenario, comprising:

[0056] Channel monitoring and data acquisition module: Composed of various types of sensors deployed in the power terminal, it is responsible for real-time acquisition of various physical parameters of CT link channel, transmitting the data to CT link channel model calculation unit, and monitoring and collecting network status indicators related to ST service, providing data support for subsequent ST service analysis;

[0057] CT Link Channel Model Calculation Unit: It has a built-in CT link channel model, receives data from the channel monitoring and data acquisition module, calculates the CT user transmission rate using an algorithm, and outputs the result to the resource allocation and coordination control module.

[0058] Limited block length channel coding solving unit: configure the limited block length channel coding scheme optimized according to the pre-stored characteristics of ST services, receive part of the ST service related data from the channel monitoring and data acquisition module, solve the achievable transmission rate of ST services by using the coding algorithm, and feed back the result to the resource allocation collaborative control module;

[0059] ST service analysis and constraint design unit: analyze the ST delay demand by using big data analysis and real-time monitoring technology, design the ST interruption probability constraint, and deliver the constraint condition and strategy information to the resource allocation collaborative control module;

[0060] Proportional fair algorithm and collaborative model construction module: built-in proportional fair algorithm, receive service data and parameters from the above-mentioned units, construct a CT and ST service collaborative resource allocation model, determine the objective function, and output the model and function information to the double-delay deep deterministic policy gradient algorithm solving unit;

[0061] Markov model parameter configuration unit: responsible for setting the state space, action space and reward function of the Markov model according to the real-time service state of the power terminal, and delivering the configured parameters to the double-delay deep deterministic policy gradient algorithm solving unit;

[0062] Double-delay deep deterministic policy gradient algorithm solving unit: according to the feedback of the Markov model and the objective function constructed by the proportional fair algorithm, use the double-delay deep deterministic policy gradient algorithm to solve the optimal resource allocation scheme, and real-time issue the scheme to the resource scheduling execution module of the power terminal;

[0063] Resource scheduling execution module: receive the optimal resource allocation scheme, responsible for accurately allocating resources to CT users and ST services according to the scheme, and has the ability to monitor the resource allocation execution in real time and dynamically adjust the allocation strategy to cope with sudden situations.

[0064] Beneficial effects: compared with the prior art, the present application has the following advantages: by constructing a resource allocation model, introducing a proportional fair algorithm, and combining deep reinforcement learning to dynamically allocate resources for two kinds of services through interaction with the environment, the CT average data rate is maximized under the premise of considering the ST delay demand and CT user reliability. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 It is an ISAC heterogeneous service multiplexing scenario model of the present application;

[0066] Figure 2 It is a double-delay deep deterministic policy gradient algorithm diagram of the present application. DETAILED DESCRIPTION

[0067] The technical solutions of the present application will be described in detail below with reference to the drawings.

[0068] This embodiment considers a base station serving I CT users and J ST users, denoted by i and j respectively as the ith CT user and the jth ST user, i∈{1,2,…,I}, considering the corresponding downlink CT requests and ST link requests of these users, the two kinds of service requests adopt OFDMA multiplexing mode, and the network resources are divided into B resource blocks RB, wherein a resource block b contains 12 subcarriers, b∈{1,2,…,B}.

[0069] This embodiment adopts a puncturing method to schedule ST traffic to meet the low latency requirement of ST service, and by adopting a short TTI to schedule ST data packets, when the ST data packets arrive, the frequency resources of the CT service being transmitted will be preempted, thereby realizing the low latency requirement of the ST service. However, this method inevitably affects the transmission quality of the CT, so the low latency requirement of the ST service should also be considered when considering the transmission quality of the CT service.

[0070] (1) Calculate the CT user transmission rate according to the CT link channel model.

[0071] The data rate of the CT user can be calculated by the Shannon formula, and the signal-to-noise ratio of the CT user i on the resource block b at the time slot t is as follows:

[0072]

[0073] wherein, P (t) represents the downlink transmission power of the ith CT user on the resource block b at the time slot t, and the unit is Hz, H and and

[0074] respectively represent the interference generated by other CT users and ST users, and σ 2 represents the noise power;

[0075] The puncturing effect of ST traffic on CT service is considered, and η i,b,k (t) represents the puncturing situation of the CT user on the resource block b of the kth mini time slot at the time slot t, and η i,b,k (t)=1 represents being punctured, and vice versa;

[0076]

[0077] wherein, ω i,b(t)∈{0,1,2,…,K}, K represents the total number of mini-slots, the downlink transmission rate of the i-th user on all resource blocks after puncturing can be represented as:

[0078]

[0079] where x i,b (t) = 1 indicates that the resource block b is allocated to the i-th CT user, and 0 otherwise.

[0080] (2) Calculate the CT user transmission rate according to the CT link channel model.

[0081] Since the size of the ST data packet is usually much smaller than that of the CT data packet, it cannot be solved by using the Shannon formula as the CT service, and a finite block length channel coding scheme can be used to solve it. The achievable transmission rate of the j-th ST user at time slot t can be represented as:

[0082]

[0083] where, SINR of ST user, Q -1 (θ) represents the inverse Gaussian function, and θ represents the transmission error probability, Q

[0084]

[0085] (3) Analyze the ST delay requirement and design the ST outage probability constraint.

[0086] The reliability of ST is realized by ensuring that the outage probability is less than a specified threshold. When the size of the ST data packet arriving in time slot t is greater than the transmission rate that the ST user can achieve, it is considered that transmission outage occurs. Use ε to represent the maximum outage threshold that can be tolerated, and the outage probability constraint of ST can be represented as:

[0087]

[0088] where V(t) represents the total number of ST data packets arriving in time slot t, and V(t) = ∑ k∈K V k (t) is calculated, V k (t) represents the number of ST data packets arriving in mini-slot k, and s represents the size of the ST data packet.

[0089] (4) Introduce the proportional fair algorithm to build a CT and ST service collaborative resource allocation model.

[0090] Considering that the puncturing of ST service will affect the quality of service of CT service, and the ST puncturing position is uncertain, which may cause the CT users with low transmission rate, such as users at the edge of the cell with poor channel conditions, to suffer more loss caused by puncturing, therefore, in order to avoid the fairness between users being greatly affected, a priority strategy needs to be developed to make a trade-off between the total throughput and the fairness of user satisfaction, the present scheme adopts the proportional fairness algorithm, and the priority is calculated by introducing the historical average throughput of the user, and the objective function of the PF scheduler is:

[0091]

[0092] wherein, represents the instantaneous data rate of the CT user in the time slot t, represents the historical average throughput at the time slot t, and is updated according to the following formula:

[0093]

[0094] wherein, T represents the time window length for calculating the average throughput, and the PF scheduler can establish proportional fairness between users over time.

[0095] The present embodiment describes the resource allocation problem of CT and ST as an optimization problem, and the target is to maximize the throughput of CT users under the premise of meeting the QoS constraint of ST service. Unlike the traditional strategy of maximizing the average transmission rate of all CT users, the PF algorithm is introduced to make a trade-off between the throughput and the fairness of CT user satisfaction, so as to avoid large differences in the quality of service of CT users. The following is the formula of the CT and ST service cooperative resource allocation model:

[0096]

[0097]

[0098] wherein, P max represents the maximum transmission power of the base station, x represents the RB allocation matrix, p represents the power allocation vector, and η represents the mini-slot puncturing number matrix. The objective of the optimization problem is to find the optimal values of the above three parameters x * , p * and η * , which maximizes the average CT transmission rate considering the fairness of CT users; represents the resource block allocation constraint, which means that a resource block is only associated with a single user; represents the guarantee that the sum of transmission power does not exceed the maximum transmission power of the base station; represents the guarantee of the reliability of ST service, and the constraint ω i,b(t) e {0, 1, …, K} represents the value range of the puncturing time slot number of the resource block b.

[0099] (5) Set the corresponding Markov model parameters, including state space, action space and reward function.

[0100] In the embodiment, the Markov decision process is used to represent the wireless resource scheduling problem, wherein the state s(t) represents the current situation of the system, the action a(t) represents the resource allocation strategy, and the reward R(t) represents the optimization target of the system performance;

[0101] In the state space, s(t) = {g e (t), g u (t), λ, I, J}, g e (t) and g u (t) represent the channel conditions of the ST and CT users respectively, λ represents the Poisson arrival rate of the ST traffic, and I and J represent the number of CT and ST users respectively;

[0102] The action set of the agent is defined as a(t) = {x, p, η}, wherein x, p, and η represent the resource block allocation, the transmission power allocation, and the ST scheduling strategy respectively;

[0103] The reward at time t is represented by R(t), the demand of the CT and ST users is defined in the reward function, and then the action with higher reward is selected by the agent, and in the present scheme, the reward function is represented according to the following two equations:

[0104]

[0105] wherein R e (t) represents the demand of the CT user, including the CT data rate and reliability, and R u (t) represents the ST reliability demand, and the business demand of the CT and ST is comprehensively considered, and is represented as the sum of the weights of and:

[0106]

[0107] wherein the value of the parameter is adjusted in the training process, θ(t) represents the estimated interruption probability, and ε represents the maximum interruption probability that can be tolerated.

[0108] The state s t (t) is evaluated, and the action value function of the next action a t (t) is Q π (s t (t), a t (t)), and the goal of the reinforcement learning is to find the optimal strategy π * that maximizes Qπ (s t ,a t ), wherein Q π (s t ,a t ) is calculated according to the Bellman equation, and the calculation formula is:

[0109] Q π (s t ,a t ) = E[R(s t ,a t )+Q π (s t+1 ,a t+1 )].

[0110] (6) The optimal resource allocation scheme is solved by using the double-delay deep deterministic policy gradient algorithm based on proportional fairness.

[0111] The double-delay deep deterministic policy gradient algorithm adopts a double network architecture, target policy smoothing regularization and delay update technology, solves the overestimation problem in wireless resource scheduling, and TD3 involves six neural networks in total. The training network includes actor network μ φ , critic1 network Q θ1 and critic2 network Q θ2 , and the target network includes target network μ' φ , target critic1 network Q' θ1 and target critic2 network Q' θ2 .

[0112] The critic network is updated by minimizing the error between the evaluation value and the target value. The target actor network calculates the action a' under the state s', and selects the smaller Q value as the target Q value under the current state:

[0113]

[0114] According to the Q value update rule, the resource allocation strategy is adjusted, so that the scheduling decision under each state can maximize the overall performance of the system. The loss function of the critic network is:

[0115] loss = E[(y-Q θi (s,a)) 2 ];

[0116] Wherein, i = 1, 2 corresponds to two critic networks, and the parameters of the critic network are updated by minimizing the two loss functions;

[0117] The action a = μφ (s), the evaluation value of state-action pair (s, a) calculated using critic1 or critic2 is Q θ1 (s, a). The gradient ascent algorithm is used to maximize Q θ1 (s, a), the update of the actor network is completed:

[0118]

[0119] TD3 uses a soft update method to update the target network parameters, that is, a learning rate τ is introduced, and the old target network parameters and the new corresponding network parameters are weighted and averaged to assign the target network, which is expressed by the following formula:

[0120] θ′ i =τθ i +(1-τ)θ′ i , i = 1, 2;

[0121] φ′ = τφ + (1-σ)φ′, τ << 1;

[0122] Based on the DRL process and TD3 architecture, a proportional fairness-based double-delay deep deterministic policy gradient algorithm is proposed. The algorithm selects action a(t) = {x, p, η} according to the state space s t , and updates the policy according to the obtained reward r t , stores (s t , a t , r t , s t+1 ) into the experience replay pool, and samples data from the experience replay pool every time step, and iteratively updates the actor network, critic network and target network according to the data;

[0123] Wherein, the actor network represents the scheduling policy, and the critic network evaluates the performance of the current policy. In order to ensure the stability of the network, the actor and target network are updated every d iterations, and through this process, the PF-TD3A algorithm can obtain the optimal solution x * , p * and η * of RB resource block, power allocation and ST scheduling, so as to improve the overall performance of the system.

Claims

1. A method for intelligent resource allocation of power terminals in an ISAC heterogeneous service scenario, characterized in that, Includes the following steps: (1) Collect physical parameter data of CT link channel, transmit the data to CT link channel model, and calculate CT user transmission rate; (2) Collect data related to ST service and calculate the achievable transmission rate of ST service using a finite block long channel coding scheme; (3) Analyze the ST delay requirements and design ST interruption probability constraints; (4) Based on the results of steps (1) to (3), the proportional fairness algorithm is introduced to construct a resource allocation model for CT and ST business collaboration and determine the first objective function; (5) Based on the CT and ST business collaborative resource allocation model described in step (3), set the corresponding Markov model parameters and determine the second objective function. The parameters include the state space, action space and reward function. (6) Based on the first objective function and the second objective function, the optimal resource allocation scheme is calculated using the dual-delay deep deterministic strategy gradient algorithm, and the optimal resource allocation scheme is accurately allocated to CT users and ST services.

2. The intelligent resource allocation method for power terminals according to claim 1, characterized in that, The formula for the CT link channel model described in step 1 is as follows: ； in, Let B represent the transmission rate of the i-th CT user on resource block b in time slot t, where B represents the total number of resource blocks. , This represents the total number of mini-time slots that have been perforated. , K represents the total number of mini-slots. This indicates the perforation status of the CT user in the k-th mini-time slot resource block b of time slot t. A value of 0 indicates that the hole has been pierced, otherwise it is 0. ； in, This represents the signal-to-noise ratio of CT user i in resource block b in time slot t. Indicates noise power. This represents the downlink transmission power of the i-th CT user on resource block b in time slot t. Indicates the transmission channel gain. and This indicates interference from other CT and ST users; ； in, After punching, the downlink transmission rate of the i-th CT user across all resource blocks is... This indicates that resource block b has been allocated to the i-th CT user, otherwise it is 0.

3. The intelligent resource allocation method for power terminals according to claim 1, characterized in that, The formula for the finite block long channel coding scheme described in step 2 is as follows: ； Where I represents the number of CT users, This represents the achievable transmission rate of the j-th ST user in time slot t. Indicates the signal-to-noise ratio of ST users. This represents the number of symbols in a mini-slot. Represents the inverse Gaussian function. Indicates the probability of transmission error. This indicates channel dispersion, representing the randomness of the user channel; ； ； in, This represents the downlink transmission power of the j-th ST user on resource block b in time slot t. This represents the transmission channel gain of the j-th ST user in resource block b of time slot t. Indicates noise power. and This indicates interference from other ST and CT users.

4. The intelligent resource allocation method for power terminals according to claim 1, characterized in that, The ST interruption probability constraint mentioned in step 3 is as follows: ； Where V(t) represents the total number of data packets arriving at ST within time slot t, expressed as... Calculate, V k (t) represents the number of ST packets arriving within mini-slot k, s represents the size of the ST packets, and J represents the number of ST users. Let represent the achievable transmission rate of the j-th ST user at time slot t, ɛ represent the maximum tolerable interruption threshold, and K represent the total number of mini-time slots.

5. The intelligent resource allocation method for power terminals according to claim 1, characterized in that, The proportional fairness algorithm formula described in step 4 is as follows: ； in, This represents the instantaneous data rate of the CT user within time slot t. The historical average throughput at time slot t is updated according to the following formula: ； Where T represents the length of the time window for calculating the average throughput.

6. The intelligent resource allocation method for power terminals according to claim 1, characterized in that, The resource allocation model formula described in step 4 is as follows: ； ； ； ； ； ； ； Among them, P max Let x represent the maximum transmission power of the base station, p represent the power allocation vector, and η represent the mini-slot punch number matrix. The goal of the optimization problem is to find the optimal values ​​of these three parameters, x*. 、 p* and η* maximize the average CT transmission rate that takes into account fairness among CT users; I represents the number of CT users, and T represents the length of the time window for calculating the average throughput. This represents the instantaneous data rate of the CT user within time slot t. This indicates that resource block b was allocated to the i-th CT user, otherwise it is 0; B represents the total number of resource blocks. This represents the total number of mini-time slots that have been perforated. This represents the downlink transmission power of the i-th CT user on resource block b in time slot t; This indicates a resource block allocation constraint, meaning that a resource block is associated with only one single user; This means that the sum of the transmission power will not exceed the maximum transmission power of the base station; This indicates a guarantee of the reliability of ST business. This indicates the range of values ​​for the number of perforated time slots in resource block b.

7. The intelligent resource allocation method for power terminals according to claim 1, characterized in that, The state space and action space described in step 5 are as follows: State space: ,in and Let represent the channel conditions for ST and CT users respectively, λ represent the Poisson arrival rate of ST traffic, and I and J represent the number of CT and ST users respectively; Action space: , where x represents resource block allocation, p represents transmission power allocation, and η represents ST scheduling policy.

8. The intelligent resource allocation method for power terminals according to claim 1, characterized in that, The reward function described in step 5 is as follows: R(t) represents and The sum of weights, parameters The value is adjusted during training; ； ； in, This reflects the needs of CT users. Indicates ST's reliability requirements, This represents the downlink transmission rate of the i-th user across all resource blocks after punching. This represents the historical average throughput at time slot t. Let represent the achievable transmission rate of the j-th ST user in time slot t, s represent the size of the ST data packet, V(T) represent the total number of ST data packets arriving in time slot t, and β is the weighting coefficient.

9. The intelligent resource allocation method for power terminals according to claim 1, characterized in that, The specific steps for solving the optimal resource allocation scheme in step 6 are as follows: (61) Initialize the state space s t Train network parameters θ1, θ2, Target network parameters θ'1, θ'2, Experience replay pool R and delayed update parameter d; (62) Based on the policy network, add noise ε ~ N(0.6) and select action a at each time step. t ; (63) Obtain a reward based on the action. t and the next state s t+1 , will (s t a t r t s t+1 Store it in the experience replay pool R; (64) Randomly sample N data points (s) from the experience replay pool R. t a t r t s t+1 ) ; (65) Calculate the target value using the Bellman equation. , where γ represents the discount factor; (66) Using the loss function Update the critic network; (67) Calculate the gradient every d time steps. Update the actor network; (68) Update the target network parameters using a soft update method: and ; (69) Repeat steps 62 to 69 until the maximum number of training steps T is reached; Where r represents the reward, γ represents the discount factor, and Q θi Let Q represent the training network. θi 'Target network, Q θi (s,a) represents the evaluation value of the state-action pair (s,a), i=1,2 corresponds to two critic networks, and τ represents the learning rate. This indicates that the actor network calculates action a in state s. This represents an actor network.

10. A smart resource allocation system for power terminals in an ISAC heterogeneous service scenario, characterized in that, include: Channel monitoring and data acquisition module: Composed of various types of sensors deployed in the power terminal, it is responsible for real-time acquisition of various physical parameters of CT link channel, transmitting the data to CT link channel model calculation unit, and monitoring and collecting network status indicators related to ST service, providing data support for subsequent ST service analysis; CT Link Channel Model Calculation Unit: It has a built-in CT link channel model, receives data from the channel monitoring and data acquisition module, calculates the CT user transmission rate using an algorithm, and outputs the result to the resource allocation and coordination control module. Finite block length channel coding solution unit: Configures and pre-stores optimized finite block length channel coding schemes based on ST service characteristics, receives some ST service-related data from the channel monitoring and data acquisition module, uses coding algorithms to solve for the transmission rate achievable by the ST service, and feeds the results back to the resource allocation and coordination control module; ST Business Analysis and Constraint Design Unit: Utilizes big data analysis and real-time monitoring technology to analyze ST latency requirements, design ST interruption probability constraints, and transmit constraint conditions and strategy information to the resource allocation and collaborative control module; Proportional Fairness Algorithm and Collaborative Model Construction Module: It has a built-in proportional fairness algorithm, receives business data and parameters from the above units, constructs a collaborative resource allocation model for CT and ST businesses, determines the objective function, and outputs the model and function information to the dual-delay deep deterministic policy gradient algorithm solution unit. Markov Model Parameter Configuration Unit: Responsible for setting the state space, action space and reward function of the Markov model according to the real-time business status of the power terminal, and passing the configured parameters to the dual-delay deep deterministic policy gradient algorithm solution unit; The dual-delay deep deterministic policy gradient algorithm solution unit: Based on the objective function constructed by Markov model feedback and proportional fairness algorithm, the dual-delay deep deterministic policy gradient algorithm is used to solve the optimal resource allocation scheme, and the scheme is sent to the resource scheduling execution module of the power terminal in real time; Resource scheduling and execution module: Receives the optimal resource allocation plan and is responsible for accurately allocating resources to CT users and ST services according to the plan. It has the ability to monitor the resource allocation execution status in real time and dynamically adjust the allocation strategy to deal with emergencies.

Images