A method and device for edge intelligent scheduling of cognitive wireless network

Abstract

The present application discloses an edge intelligent scheduling method and device for a cognitive wireless network, which relates to the field of cognitive radio communication technology. The method includes: constructing a cognitive wireless network system; based on various preset constraints, with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, using the Lyapunov optimization method to construct an independent time slot optimization model, wherein the independent time slot optimization model includes various parameters to be optimized; using a preset multi-agent Markov decision process model to reconstruct the independent time slot optimization model to obtain a target optimization model; based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to optimize the target optimization model, and obtain the target parameter values ​​corresponding to each parameter to be optimized. The present application realizes the joint scheduling of tasks and resources in a dynamic network environment, and reduces the long-term average system overhead.

Description

Technical Field

[0001] The present invention relates to the technical field of cognitive radio communications, and in particular to an edge intelligent scheduling method and device for a cognitive wireless network. Background Art

[0002] With the rapid development of 5G / 6G technology, a large number of heterogeneous devices are interconnected to support multiple task types, such as enhanced mobile broadband, large-scale machine-type communications, and ultra-reliable low-latency communications. More importantly, with the enhancement of Multi-access Edge Computing (MEC), more and more tasks are cached and processed on MEC servers to ensure real-time and reliability. However, task offloading also consumes communication resources and energy resources, and if tasks are not well scheduled between terminal devices and MEC servers, resource competition may be intensified.

[0003] Existing joint optimization of task offloading and resource allocation comprehensively explores the optimization of task partitioning, offloading decisions, communication bandwidth, transmission power, computing frequency, and energy consumption to minimize latency / energy / cost or maximize utility. However, only a few studies have considered the fundamental problem of spectrum scarcity, which limits the realization of task offloading. Without sufficient spectrum, tasks cannot be exchanged between terminal devices and MEC servers. More importantly, due to the increasing demand for communication, spectrum is always scarce, which is why spectrum is continuously developed and utilized from the millimeter wave band in the 5G era to the terahertz band in the 6G era. Summary of the invention

[0004] In view of this, the present invention provides an edge intelligent scheduling method and device for a cognitive wireless network, the main purpose of which is to solve the problem of unreasonable task and resource scheduling caused by spectrum scarcity in the current cognitive wireless network scheduling process.

[0005] To solve the above problems, the present application provides an edge intelligent scheduling method for a cognitive wireless network, including:

[0006] Build cognitive wireless network systems;

[0007] Based on the preset constraints, with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, an independent time slot optimization model for network scheduling of the cognitive wireless network system is constructed by using the Lyapunov optimization method, wherein the independent time slot optimization model includes various parameters to be optimized, and each of the preset constraints includes at least a preset transmission power constraint condition limited by the spectrum usage of the primary user;

[0008] Reconstructing the independent time slot optimization model using a preset multi-agent Markov decision process model to obtain a target optimization model;

[0009] Based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target optimization model is optimized using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to obtain target parameter values ​​corresponding to each parameter to be optimized for network scheduling of the cognitive wireless network system.

[0010] To solve the above problems, the present application provides a cognitive wireless network edge intelligent scheduling device including:

[0011] System building module, used to build cognitive wireless network system;

[0012] A model building module, for building an independent time slot optimization model for network scheduling of the cognitive wireless network system by using a Lyapunov optimization method with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, wherein the independent time slot optimization model includes various parameters to be optimized, and each of the preset constraints includes at least a preset transmission power constraint condition constrained by a primary user spectrum;

[0013] A model reconstruction module, used to reconstruct the independent time slot optimization model using a preset multi-agent Markov decision process model to obtain a target optimization model;

[0014] An optimization module is used to optimize the target optimization model based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time using a preset deep reinforcement learning algorithm model based on proximal strategy optimization, so as to obtain target parameter values ​​corresponding to each parameter to be optimized for network scheduling of the cognitive wireless network system.

[0015] To solve the above problems, the present application provides a storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the edge intelligent scheduling method of the above-mentioned cognitive wireless network are implemented.

[0016] To solve the above problems, the present application provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned edge intelligent scheduling method of the cognitive wireless network when executing the computer program on the memory.

[0017] Beneficial effects of the present application: The present application constructs a cognitive wireless network system; based on various preset constraints, with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, an independent time slot optimization model is constructed using the Lyapunov optimization method, and the independent time slot optimization model includes various parameters to be optimized; multiple cognitive base stations in the present application collaboratively calculate the single task unloaded by each cognitive terminal device, and allocate computing resources to perform edge computing. In order to achieve harmonious coexistence on the licensed spectrum, the cognitive terminal device and the cognitive base station will consider the peak interference power to the main user, the maximum transmission power of the cognitive terminal device, and the co-channel interference from other unloaded cognitive terminal devices, and strictly control their transmission power. The independent time slot optimization model is reconstructed using a preset multi-agent Markov decision process model to obtain a target optimization model; based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target optimization model is optimized using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to obtain the target parameter values ​​corresponding to each parameter to be optimized. Task and resource scheduling in a dynamic network environment is achieved, and the long-term average system overhead is reduced.

[0018] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0020] Figure 1 A schematic diagram of a process flow of an edge intelligent scheduling method for a cognitive wireless network provided in an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram showing the architecture of a cognitive wireless network system according to an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram of the structure of a preset deep reinforcement learning algorithm model based on proximal strategy optimization provided in an embodiment of the present application is shown;

[0023] Figure 4 A structural block diagram of a cognitive wireless network edge intelligent scheduling device provided by another embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0025] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0026] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0027] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0028] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0029] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0030] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0031] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0032] The present application embodiment provides an edge intelligent scheduling method for a cognitive wireless network, such as Figure 1 As shown, including:

[0033] Step S101: constructing a cognitive wireless network system;

[0034] During the specific implementation of this step, the cognitive wireless network system includes multiple cognitive base stations (Cognitive Base Station, referred to as CBS), multiple cognitive terminal devices (Cognitive End Device, referred to as CED) and a main network corresponding to multiple main users, wherein the multiple cognitive terminal devices divide their respective tasks for computational offloading, and the multiple cognitive base stations allocate computing resources to perform edge computing.

[0035] Step S102: Based on various preset constraints, with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, an independent time slot optimization model for network scheduling of the cognitive wireless network system is constructed using a Lyapunov optimization method, wherein the independent time slot optimization model includes various parameters to be optimized, and each of the preset constraints includes at least a preset transmission power constraint condition limited by the spectrum usage of the primary user;

[0036] During the specific implementation of this step, each of the preset constraints includes a preset end-edge task allocation ratio constraint corresponding to each of the cognitive terminal devices, a preset transmit power constraint subject to the main user spectrum usage constraint and the cognitive terminal device hardware constraint, a preset computing resource constraint and a preset long-term energy consumption constraint; an edge computing delay function corresponding to each of the cognitive base stations is constructed respectively; a local computing delay function corresponding to each of the cognitive terminal devices; a local computing delay function corresponding to each of the cognitive terminal devices; a transmission energy consumption function corresponding to each of the cognitive terminal devices; and a local computing energy consumption function corresponding to each of the cognitive terminal devices; a model is constructed based on each of the edge computing delay functions, each of the local computing delay functions, each of the transmission energy consumption functions, each of the local computing energy consumption functions and a predetermined weight coefficient to obtain the initial optimization model; the initial optimization model is decoupled by using the Lyapunov optimization method to obtain the independent time slot optimization model.

[0037] Step S103: reconstructing the independent time slot optimization model using a preset multi-agent Markov decision process model to obtain a target optimization model;

[0038] During the specific implementation of this step, a function is constructed based on the delay parameters, energy consumption parameters and predetermined penalty function coefficients of each cognitive terminal device in different time slots to obtain a reward function corresponding to each cognitive terminal device; a weighted function is constructed based on each reward function to obtain a target reward function; a model is constructed based on the target reward function, the agent set, system state set and action set of the cognitive wireless network system to obtain the preset multi-agent Markov decision process model; based on the preset multi-agent Markov decision process model and the reward function, the independent time slot optimization model for intelligent scheduling of the cognitive wireless network edge is transformed into a model to obtain a target optimization model for scheduling the cognitive wireless network system.

[0039] Step S104: Based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target optimization model is optimized using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to obtain target parameter values ​​corresponding to each parameter to be optimized for network scheduling of the cognitive wireless network system.

[0040] During the specific implementation of this step, based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target policy network model in the deep reinforcement learning algorithm model preset based on proximal policy optimization is used to perform state prediction to obtain the target parameter values ​​corresponding to each parameter to be optimized.

[0041] The present application constructs a cognitive wireless network system; based on various preset constraints, with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, an independent time slot optimization model for network scheduling of the cognitive wireless network system is constructed by using the Lyapunov optimization method, wherein the independent time slot optimization model includes various parameters to be optimized, and each of the constraints at least includes a preset transmit power constraint constrained by the main user spectrum; in the present application, multiple cognitive base stations collaborate to calculate a single task to be unloaded, and allocate computing resources to perform edge computing. In order to achieve harmonious coexistence on the licensed spectrum, the cognitive terminal device and the cognitive base station will consider the peak interference power of the main user, the maximum transmit power of the cognitive terminal device, and the co-channel interference from other cognitive terminal devices when calculating the unloading, and strictly control their transmit power. The independent time slot optimization model is reconstructed by using a preset independent time slot Lyapunov drift penalty function and a preset multi-agent Markov decision process model to obtain a target optimization model; based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target optimization model is optimized by using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to obtain the target parameter value corresponding to each parameter to be optimized. It realizes task and resource scheduling in a dynamic network environment and reduces the long-term average system overhead.

[0042] Another embodiment of the present application provides another edge intelligent scheduling method of a cognitive wireless network, including:

[0043] Step S201: constructing a cognitive wireless network system;

[0044] In the specific implementation process of this step, Figure 2 As shown, it is a schematic diagram of the architecture of the cognitive wireless network system; the cognitive wireless network system includes multiple cognitive base stations, multiple cognitive terminal devices, and a main network corresponding to multiple main users, wherein the multiple cognitive terminal devices divide their respective tasks for computational offloading, and the multiple cognitive base stations allocate computing resources to perform edge computing. The main network includes: J main users who have obtained spectrum use licenses, represented by PU j (j=1,...,J); the edge computing-cognitive wireless network is located under the main network and includes: N cognitive base stations equipped with MEC servers and continuous energy supply, which can provide computing resources for cognitive terminal devices; M cognitive terminal devices equipped with limited computing and energy resources; the cognitive terminal devices will generate splittable tasks in each different time slot, and the tasks can be calculated locally or at the cognitive base station. The edge computing process refers to the cognitive terminal device CED m (m=1,...,M) can divide the task into different subtasks, and then offload them to different cognitive base stations through the licensed primary user spectrum, so that the cognitive base station can assist in completing the calculation of the task. Let μ m,n (t)∈[0,1] represents the CED at time slot t m With CBS n (n=1,...,N) task allocation ratio, n=0 indicates local computing.

[0045] Step S202: constructing various preset constraint conditions;

[0046] In the specific implementation process of this step, based on the local computing task amount parameters corresponding to each of the cognitive terminal devices and the edge computing task amount parameters corresponding to each of the terminal devices, constraint conditions are constructed to obtain preset task allocation ratio constraint conditions corresponding to each of the cognitive terminal devices; the mathematical expression of the task allocation ratio constraint condition C1 can be expressed as the following formula (1):

[0047]

[0048] Among them, the task allocation ratio constraint C1 represents the m The total amount of local computing and edge computing tasks should not exceed the maximum task amount; μ m,n(t)∈[0,1] represents the CED at time slot t m With CBS n (n=1,...,N) task allocation ratio, n=0 indicates local computing.

[0049] Based on the predetermined maximum transmit power parameter supported by each cognitive terminal device and each cognitive base station in the transmission phase, the peak interference power acceptable to each primary user, the transmit power of each cognitive terminal device, and the channel power gain of each cognitive terminal device to each primary user, a constraint condition is constructed to obtain a preset transmit power constraint condition; the mathematical expression of the preset transmit power constraint condition C2 can be shown as the following formula (2):

[0050]

[0051] Among them, I j Primary User PU j Acceptable peak interference power, P m′ (t), P m (t) are the transmission powers of cognitive terminal m′ and cognitive terminal m, respectively, and g m′,n (t), g j,n (t) are cognitive terminal m′ and cognitive terminal m respectively with primary user PU j The channel power gain; P max is the predetermined maximum transmission power parameter. All unloaded cognitive terminals to the primary user PU j The interference should not be greater than I j ,Right now In this way, the transmit power of each cognitive terminal device is subject to the most stringent power control constraint and the maximum transmit power constraint given by the primary user;

[0052] Based on the predetermined maximum computing resource frequency parameter of each cognitive base station, a constraint condition is constructed to obtain a preset computing resource constraint condition; the mathematical expression of the preset computing resource constraint condition C3 can be expressed as the following formula (3):

[0053]

[0054] Among them, f m,n (t) is CBS n Assigned to CED m The calculation frequency of F max For CBS n The maximum calculation frequency.

[0055] Based on the energy consumption threshold of each cognitive terminal device in each time slot and the preset total number of time slots, a constraint condition is constructed to obtain a preset long-term energy consumption constraint condition. The mathematical expression of the preset long-term energy consumption constraint condition C4 can be shown as the following formula (4):

[0056]

[0057] in, is the expected value; E m (t) is the energy consumption of processing the task of cognitive terminal m in time slot t; T is the total number of time slots; ε m is the CED in each time slot m The maximum energy supply, that is, the energy consumption threshold.

[0058] Step S203: constructing a function based on the calculation frequency allocated by each cognitive base station to each cognitive terminal device, the predetermined calculation frequency required for a unit task, the task data volume of each cognitive terminal device, the task allocation ratio between each cognitive terminal device and each cognitive base station in each different time slot, and the task transmission rate between each cognitive terminal device and each cognitive base station in each different time slot, to obtain an edge computing delay function corresponding to each cognitive base station;

[0059] In the specific implementation process of this step, the edge computing delay function The mathematical expression of can be expressed as follows:

[0060]

[0061] Among them, D m (t) represents CED m The amount of task data generated, C m (t) is the predetermined calculation frequency required for the unit task; R m,n (t) is CED m With CBS n The transmission rate between the two; the calculation of the transmission rate can be shown in the following formula (6):

[0062]

[0063] in, It's CBS n The noise at B m,n (t) and g m,n (t) are CED m and CBS n The bandwidth and channel power gain between them. and They correspond to the interference from other CEDs unloading at the same time and the coexisting primary user spectrum. j (t) is PU j Transmitting power, g m′,n (t) and g j,n (t) is CED m' and PU j CBS n The channel power gain.

[0064] Step S204: constructing a function based on the task data volume of each cognitive terminal device, the required computing frequency of each predetermined unit task, the computing frequency of each cognitive terminal device, and the task allocation ratio of each cognitive terminal device in each different time slot to obtain a local computing delay function corresponding to each cognitive terminal device;

[0065] In the specific implementation process of this step, the local calculation delay function The mathematical expression of can be expressed as follows:

[0066]

[0067] Among them, f m For CED m The calculation frequency of .

[0068] Step S205: constructing a function based on the transmit power of each cognitive terminal device in each different time slot, the amount of task data of each cognitive terminal device, the task transmission rate between each cognitive terminal device and each cognitive base station in each different time slot, and the task allocation ratio between each cognitive terminal device and each cognitive base station in each different time slot, to obtain a transmission energy consumption function corresponding to each cognitive terminal device;

[0069] In the specific implementation process of this step, the transmission energy consumption function The mathematical expression of can be expressed as follows:

[0070]

[0071] Among them, P m (t) Cognitive End Device (CED) m The transmission power of m (t) is cognitive terminal device CED m The amount of task data; μ m,n (t) is cognitive terminal device CED m Cognitive Base Station (CBS) n The task allocation ratio of R m,n (t) is cognitive terminal device CED mCognitive Base Station (CBS) n The transmission rate between.

[0072] Step S206: constructing a function based on the predetermined energy coefficient of each cognitive terminal device, the task data volume of each cognitive terminal device, the required computing frequency of each predetermined unit task, the task allocation ratio of each cognitive terminal device in each different time slot, and the computing frequency of each cognitive terminal device to obtain a local computing energy consumption function corresponding to each cognitive terminal device;

[0073] In the specific implementation process of this step, the local calculation energy consumption function The mathematical expression of can be expressed as follows:

[0074]

[0075] where κ is a predetermined energy coefficient that depends on the hardware architecture; μ m,0 (t) is allocated to cognitive terminal device CED m The task allocation ratio of local computing; D m (t) is cognitive terminal device CED m The amount of task data; C m (t) is the predetermined calculation frequency required for the unit task; f m For CED m The local calculation frequency.

[0076] Step S207: constructing a model based on each of the edge computing delay functions, each of the local computing delay functions, each of the transmission energy consumption functions, each of the local computing energy consumption functions and a predetermined weight coefficient to obtain an initial optimization model;

[0077] In the specific implementation process of this step, a function is constructed based on the local computing energy consumption function and the transmission energy consumption function to obtain a task generation energy consumption function corresponding to each cognitive terminal device; the task generation energy consumption function E m The mathematical expression of (t) can be expressed as follows:

[0078]

[0079] Based on the edge computing delay function and the local computing delay function, a function is constructed to obtain a task generation delay function; the task generation delay function L m The mathematical expression of (t) can be expressed as follows:

[0080]

[0081] Based on the energy consumption function of the task generated by the same cognitive terminal device, the delay function of the task generated by the task and the predetermined weight coefficient ω, a function is constructed to obtain a system overhead model of the cognitive wireless network system; the system overhead of the cognitive wireless network is m The mathematical expression of (t) can be expressed as follows:

[0082] O m (t) = ωL m (t)+(1-ω)E m (t),ω∈[0,1](12)

[0083] Based on various preset constraints, an initial optimization model for network scheduling of the cognitive wireless network system is constructed with the goal of minimizing the long-term average system overhead of the cognitive wireless network system; the mathematical expression of the initial optimization model can be expressed as the following formula (13):

[0084]

[0085] The initial optimization model includes various parameters to be optimized; μ, are the parameters to be optimized, μ is the task allocation ratio of each cognitive terminal device to each cognitive base station, μ={μ m,n} M×(N+1) ; is the transmission power of each cognitive device when transmitting tasks to the cognitive base station, The proportion of computing resources allocated to each cognitive base station when calculating the tasks generated by the cognitive terminal device, and T is the total number of time slots, and t is the current time slot.

[0086] Step S208: Decoupling the initial optimization model using the Lyapunov optimization method to obtain the independent time slot optimization model;

[0087] In the specific implementation process of this step, a virtual energy consumption queue of the cognitive wireless network system is constructed based on the edge computing delay and transmission energy consumption of each cognitive edge device in different time slots; the virtual energy consumption queue is introduced To reconstruct the long-term constraint C4. The length of the energy consumption queue is defined as the energy consumption queue backlog in each time slot, and the initial queue backlog Q m (0) = 0. According to the energy consumption relationship, the updated mathematical formula of the energy consumption queue can be expressed by the following formula (14):

[0088] Q m (t+1)=[Q m (t)-ε m ] + +E m (t) (14)

[0089] in, [x] + It means taking the non-negative part of the internal formula. means "defined as"; ε m Indicates the service rate of the energy consumption queue, which is also the long-term energy consumption threshold.

[0090] Based on the virtual energy consumption queue, a function is constructed to obtain an initial Lyapunov function; the initial Lyapunov function The mathematical expression of can be expressed as follows:

[0091]

[0092] in, The larger it is, the larger the queue backlog will be.

[0093] Based on the virtual energy consumption queue and the initial Lyapunov function, a function is constructed to obtain an independent time slot Lyapunov drift function; in order to reflect the dynamic changes of the cognitive wireless network system from time slot t to time slot t+1, the independent time slot Lyapunov drift function The mathematical expression of can be expressed as follows:

[0094]

[0095] From the perspective of system stability, the smaller the value of the Lyapunov drift function, the more stable the system becomes.

[0096] Based on the initial optimization model and the independent time slot Lyapunov drift function, a function is constructed to obtain an initial Lyapunov drift penalty function; according to the initial optimization model for minimizing the long-term average system overhead, the initial Lyapunov drift penalty function is further defined. The mathematical expression can be expressed as follows:

[0097]

[0098] Among them, ν ≥ 0 is the predetermined penalty function coefficient that controls the importance of system overhead and energy consumption backlog.

[0099] The first constraint condition of the independent time slot Lyapunov drift function and the second constraint condition of the initial Lyapunov drift penalty function are respectively constructed; the first constraint condition is used to represent the upper bound of the Lyapunov drift function, and the mathematical expression of the first constraint condition can be expressed as follows:

[0100]

[0101] The second constraint condition is used to represent the upper bound of the Lyapunov drift penalty function. The mathematical expression of the second constraint condition can be expressed as the following formula (19):

[0102]

[0103] The initial Lyapunov drift penalty function is reconstructed based on the initial optimization model, the first constraint condition, and the second constraint condition to obtain the preset independent time slot Lyapunov drift penalty function; the model is reconstructed based on the preset independent time slot Lyapunov drift penalty function and the initial optimization model to obtain the independent time slot optimization model. The mathematical expression of the independent time slot optimization model can be shown as follows:

[0104]

[0105] Step S209: constructing a function based on the delay parameters, energy consumption parameters and predetermined penalty function coefficients of each cognitive terminal device in different time slots to obtain a reward function corresponding to each cognitive terminal device;

[0106] In the specific implementation process of this step, the reward function r m The mathematical expression of (t) can be expressed as follows:

[0107] r m (t)=-νO m (t)-Q m (t)E m (t)(21)

[0108] Step S210: constructing a weighted function based on each of the reward functions to obtain a target reward function;

[0109] In the specific implementation process of this step, the mathematical expression of the target reward function r(t) can be expressed as follows:

[0110]

[0111] Step S211: constructing a model based on the target reward function, the agent set, the system state set and the action set of the cognitive wireless network system to obtain the preset multi-agent Markov decision process model;

[0112] In the specific implementation of this step, the model includes: state space, action space, and reward function. The state space is the observable information in the environment, based on which action decisions can be made. It mainly includes data volume D(t), calculation cycle C(t), local calculation frequency f(t), bandwidth B(t), primary user transmission power PPU (t) and channel gain g(t); therefore, the state space s(t) in time slot t can be expressed as follows:

[0113] s(t)={D(t),C(t),f(t),B(t),P PU (t),g(t)}(23)

[0114] Where D(t) = {D m (t)} M is the data set of M CEDs, C(t)={C m (t)} M is the set of computing cycles required by M CEDs to generate tasks, f(t) = {f m (t)} M is the set of local computation frequencies given by M CEDs, B(t)={B m,n (t)} M×N is the bandwidth set between M CEDs and N CBSs, P PU (t) = {P j (t)} J is the transmission power set of J users, and g(t) = {{g m,n (t)} M×N ,{g m,j (t)} M×J ,{g j,n (t)} J×N} is the set of channel power gains among CEDs, CBSs and PUs.

[0115] The action space is a set of action strategies a(t) that can be taken by the agent in the environment, including the task offloading ratio, the transmission power of CED and the calculation frequency allocated by CBS, which can be expressed by the following mathematical formula (24):

[0116] a(t)={μ(t),P(t),F(t)} (24)

[0117] Where, μ(t)={μ m,n (t)} M×(N+1) is the set of task allocation ratios between CED and / or CBS, P(t) = {P m (t)} M is the set of transmission powers of M CEDs, and F(t) = {f m,n (t)} M×N The calculation frequency set allocated by N CBSs to M CEDs.

[0118] The preset multi-agent Markov decision process model is constructed based on the state space, action space and reward function.

[0119] Step S212: Based on the preset multi-agent Markov decision process model and the reward function, the optimization model of the cognitive wireless network edge intelligent scheduling is transformed to obtain a target optimization model for scheduling the cognitive wireless network system;

[0120] In the specific implementation process of this step, the optimization model of the cognitive wireless network edge intelligent scheduling is transformed based on the preset multi-agent Markov decision process model and the reward function to obtain a target optimization model for scheduling the cognitive wireless network system. The mathematical expression of the target optimization model can be expressed as the following formula (25):

[0121]

[0122] Step S213: constructing a preset deep reinforcement learning algorithm model based on proximal strategy optimization;

[0123] In the specific implementation process of this step, Figure 3 The figure shows the algorithm structure diagram of the preset deep reinforcement learning algorithm model based on proximal strategy optimization; the initial strategy network model π is constructed respectively θ And the initial value network model V φ ; Based on each historical experience data, the update rate function of the preset strategy network, the advantage estimation function of the preset value network, the preset target update function and the preset loss function, the model parameters of the initial strategy network model and the initial value network model are updated in a loop iteration to obtain the target strategy network model and the target value network model; specifically, a group of historical experience data is randomly collected from the experience data pool, and the historical experience data includes the historical first moment state, the historical first moment action, the historical first moment reward value and the historical second moment state, and the historical first moment state in the historical experience data is calculated and processed using the update rate function to obtain the update rate r θ (t); The mathematical expression of the update rate function can be expressed as follows:

[0124]

[0125] Among them, π θ (a(t)|s(t)) is the probability of taking action a(t) in state s(t) when the current policy network parameter is θ; π θ′(a(t)|s(t)) is the probability obtained using the old strategy θ′; the advantage estimation function is used to perform calculation processing based on the historical first moment state and the historical first moment action in the historical experience data to obtain the advantage estimation value; the mathematical expression of the advantage estimation function A(t) can be shown as follows:

[0126] A(t)=δ(t)+(γλ)δ(t+1)+...+(γλ) T-t+1 δ(t+1)(27)

[0127] in, represents the time-division error, r(t) is the reward, and γ is the discount factor. In addition, the adjustment of λ allows a trade-off between bias and variance. Based on the update rate and the advantage estimate, the preset objective update function is used to perform calculation processing to obtain the model parameter θ of the initial policy network; the mathematical expression of the preset objective function can be shown as follows:

[0128]

[0129] Among them, ∈ . It is a clipping parameter used to control the clipping range. The clip method can be used to θ (t) is clipped between [1-∈, 1+∈] to limit the amplitude of strategy update and avoid excessive strategy changes, thereby improving the stability of learning. Based on the preset loss function, the initial value network model V that minimizes the preset loss function is obtained. φ The model parameter φ; the mathematical expression of the preset loss function can be expressed as follows:

[0130]

[0131] The model parameters of the policy network model and the value network model are updated in a loop until the reward value of the current action obtained by state prediction using the updated current policy network model tends to be stable, and the current policy network model is determined as the target policy network model, and the current value network model is determined as the target value network model. Model construction is performed based on the target policy network model and the target value network model to obtain the preset deep reinforcement learning algorithm model based on proximal policy optimization.

[0132] Step S214: Based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target optimization model is optimized using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to obtain target parameter values ​​corresponding to each parameter to be optimized.

[0133] During the specific implementation of this step, based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target policy network model in the preset deep reinforcement learning algorithm model based on proximal policy optimization is used to perform state prediction to obtain the target parameter value corresponding to each parameter to be optimized; based on each target parameter value, the optimization model of the cognitive wireless network system is used to perform calculation to obtain the target reward value; the target parameter value is executed to obtain the target observation information of each terminal device in the next state; each observation information, each target parameter value, the target reward value and the target observation information are entered into a preset experience database; and the foundation is laid for the subsequent offline update of the preset deep reinforcement learning algorithm model based on proximal policy optimization in specific applications.

[0134] The present application constructs a cognitive wireless network system; based on various preset constraints, with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, an independent time slot optimization model is constructed using the Lyapunov optimization method, and the independent time slot optimization model includes various parameters to be optimized; multiple cognitive base stations in the present application collaboratively calculate the single task of offloading, and allocate computing resources to perform edge computing. In order to achieve harmonious coexistence on the licensed spectrum, the cognitive terminal device and the cognitive base station will consider the peak interference to the main user spectrum, the maximum transmission power of the cognitive terminal device, and the co-channel interference from other unloaded cognitive terminal devices, and strictly control their transmission power. The independent time slot optimization model is reconstructed using a preset multi-agent Markov decision process model to obtain a target optimization model; based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target optimization model is optimized using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to obtain the target parameter values ​​corresponding to each of the parameters to be optimized. Task and resource scheduling in a dynamic network environment is achieved, and the long-term average system overhead is reduced.

[0135] Another embodiment of the present application provides a cognitive wireless network edge intelligent scheduling device, such as Figure 4 As shown, including:

[0136] System building module 1, used to build a cognitive wireless network system;

[0137] A model building module 2 is used to build an independent time slot optimization model for network scheduling of the cognitive wireless network system based on various preset constraints and with the goal of minimizing the long-term average system overhead of the cognitive wireless network system by using the Lyapunov optimization method, wherein the independent time slot optimization model includes various parameters to be optimized, and each of the preset constraints includes at least a preset transmission power constraint condition constrained by the primary user spectrum;

[0138] A model reconstruction module 3 is used to reconstruct the independent time slot optimization model using a preset multi-agent Markov decision process model to obtain a target optimization model;

[0139] Optimization module 4 is used to optimize the target optimization model based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, using a preset deep reinforcement learning algorithm model based on proximal strategy optimization, to obtain target parameter values ​​corresponding to each parameter to be optimized for network scheduling of the cognitive wireless network system.

[0140] During the specific implementation process, the device also includes: a preset constraint condition construction module, which is specifically used to: construct constraints based on the local computing task ratio parameters corresponding to each of the cognitive terminal devices and the edge computing task ratio parameters assigned to the corresponding cognitive base station by each of the terminal devices, so as to obtain preset task allocation ratio constraints corresponding to each of the cognitive terminal devices; construct constraints based on the predetermined maximum transmission power parameters supported by each of the cognitive terminal devices, the acceptable peak interference power of each main user, the transmission power of each of the cognitive terminal devices, and the channel power gain of each of the cognitive terminal devices to each main user, so as to obtain preset transmission power constraints; construct constraints based on the predetermined maximum computing resource frequency parameters of each of the cognitive base stations, so as to obtain preset computing resource constraints; construct constraints based on the energy consumption threshold of each of the cognitive terminal devices in each time slot and the preset total number of time slots, so as to obtain preset long-term energy consumption constraints.

[0141] In the specific implementation process, the model construction module 2 is specifically used to: construct a function based on the calculation frequency assigned to each cognitive base station by each cognitive terminal device, the calculation frequency required for the predetermined unit task, the task data volume of each cognitive terminal device, the task allocation ratio between each cognitive terminal device and each cognitive base station in each different time slot, and the task transmission rate between each cognitive terminal device and each cognitive base station in each different time slot, so as to obtain an edge computing delay function corresponding to each cognitive base station; construct a function based on the task data volume of each cognitive terminal device, the calculation frequency required for each predetermined unit task, the calculation frequency of each cognitive terminal device, and the task allocation ratio of each cognitive terminal device in each different time slot, so as to obtain a local computing delay function corresponding to each cognitive terminal device; construct a function based on the transmission power of each cognitive terminal device in each different time slot, the task data volume of each cognitive terminal device, and the task allocation ratio of each cognitive terminal device in each different time slot. Functions are constructed based on the task transmission rate between each cognitive terminal device and each cognitive base station in each time slot and the task allocation ratio between each cognitive terminal device and each cognitive base station in each different time slot to obtain a transmission energy consumption function corresponding to each cognitive terminal device; functions are constructed based on the predetermined energy coefficient of each cognitive terminal device, the task data volume of each cognitive terminal device, the required computing frequency of each predetermined unit task, the task allocation ratio of each cognitive terminal device in each different time slot, and the computing frequency of each cognitive terminal device to obtain a local computing energy consumption function corresponding to each cognitive terminal device; a model is constructed based on each edge computing delay function, each local computing delay function, each transmission energy consumption function, each local computing energy consumption function and a predetermined weight coefficient to obtain an initial optimization model; the initial optimization model is decoupled by using the Lyapunov optimization method to obtain the independent time slot optimization model.

[0142] In the specific implementation process, the model reconstruction module 3 is specifically used to: construct a function based on the delay parameters, energy consumption parameters and predetermined penalty function coefficients of each cognitive terminal device in each different time slot to obtain a reward function corresponding to each cognitive terminal device; construct a weighted function based on each reward function to obtain a target reward function; construct a model based on the target reward function, the agent set, system state set and action set of the cognitive wireless network system to obtain the preset multi-agent Markov decision process model; based on the preset multi-agent Markov decision process model and the reward function, transform the independent time slot optimization model of the cognitive wireless network edge intelligent scheduling to obtain a target optimization model for scheduling the cognitive wireless network system.

[0143] During the specific implementation process, the device also includes: a preset deep reinforcement learning algorithm model construction module based on proximal policy optimization, and the preset deep reinforcement learning algorithm model construction module based on proximal policy optimization is specifically used to: respectively construct an initial policy network model and an initial value network model; iteratively update the model parameters of the initial policy network model and the initial value network model based on various historical experience data, an update rate function of a preset policy network, an advantage estimation function of a preset value network, a preset target update function, and a preset loss function to obtain a target policy network model and a target value network model; construct a model based on the target policy network model and the target value network model to obtain the preset deep reinforcement learning algorithm model based on proximal policy optimization.

[0144] During the specific implementation process, the optimization module 4 is specifically used to: based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, use the target strategy network model in the preset deep reinforcement learning algorithm model based on proximal strategy optimization to perform state prediction, and obtain the target parameter value corresponding to each parameter to be optimized; wherein each parameter to be optimized includes task allocation ratio, transmission power, and computing resource allocation ratio.

[0145] The present application constructs a cognitive wireless network system; based on various preset constraints, with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, an independent time slot optimization model is constructed using the Lyapunov optimization method, and the independent time slot optimization model includes various parameters to be optimized; multiple cognitive base stations in the present application collaboratively calculate the single task of offloading, and allocate computing resources to perform edge computing. In order to achieve harmonious coexistence on the licensed spectrum, the cognitive terminal device and the cognitive base station will consider the peak interference to the main user spectrum, the maximum transmission power of the cognitive terminal device, and the co-channel interference from other unloaded cognitive terminal devices, and strictly control their transmission power. The independent time slot optimization model is reconstructed using a preset multi-agent Markov decision process model to obtain a target optimization model; based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target optimization model is optimized using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to obtain the target parameter values ​​corresponding to each of the parameters to be optimized. Task and resource scheduling in a dynamic network environment is achieved, and the long-term average system overhead is reduced.

[0146] Another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented:

[0147] Step 1: Build a cognitive wireless network system;

[0148] Step 2: Based on the preset constraints, with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, an independent time slot optimization model for network scheduling of the cognitive wireless network system is constructed using the Lyapunov optimization method, wherein the independent time slot optimization model includes various parameters to be optimized, and each of the preset constraints includes at least a preset transmission power constraint condition constrained by the primary user spectrum;

[0149] Step 3: Reconstruct the independent time slot optimization model using a preset multi-agent Markov decision process model to obtain a target optimization model;

[0150] Step 4: Based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target optimization model is optimized using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to obtain target parameter values ​​corresponding to each parameter to be optimized for network scheduling of the cognitive wireless network system.

[0151] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0152] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0153] The specific implementation process of the above method steps can be referred to the embodiments of the edge intelligent scheduling method of any cognitive wireless network mentioned above, and this embodiment will not be repeated here.

[0154] The present application constructs a cognitive wireless network system; based on various preset constraints, with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, an independent time slot optimization model is constructed using the Lyapunov optimization method, and the independent time slot optimization model includes various parameters to be optimized; multiple cognitive base stations in the present application collaboratively calculate the single task of offloading, and allocate computing resources to perform edge computing. In order to achieve harmonious coexistence on the licensed spectrum, the cognitive terminal device and the cognitive base station will consider the peak interference to the main user spectrum, the maximum transmission power of the cognitive terminal device, and the co-channel interference from other unloaded cognitive terminal devices, and strictly control their transmission power. The independent time slot optimization model is reconstructed using a preset multi-agent Markov decision process model to obtain a target optimization model; based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target optimization model is optimized using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to obtain the target parameter values ​​corresponding to each of the parameters to be optimized. Task and resource scheduling in a dynamic network environment is achieved, and the long-term average system overhead is reduced.

[0155] Another embodiment of the present application provides an electronic device, which may be a server, and the electronic device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external client via a network connection. When the electronic device program is executed by the processor, it implements the functions or steps of the server side of an edge intelligent scheduling method for a cognitive wireless network.

[0156] In one embodiment, an electronic device is provided, which may be a client. The electronic device includes a processor, a memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external server via a network connection. When the electronic device program is executed by the processor, the functions or steps on the client side of an edge intelligent scheduling method of a cognitive wireless network are implemented.

[0157] Another embodiment of the present application provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the following method steps when executing the computer program in the memory:

[0158] Step 1: Build a cognitive wireless network system;

[0159] Step 2: Based on the preset constraints, with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, an independent time slot optimization model for network scheduling of the cognitive wireless network system is constructed using the Lyapunov optimization method, wherein the independent time slot optimization model includes various parameters to be optimized, and each of the preset constraints includes at least a preset transmission power constraint condition constrained by the primary user spectrum;

[0160] Step 3: Reconstruct the independent time slot optimization model using a preset multi-agent Markov decision process model to obtain a target optimization model;

[0161] Step 4: Based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target optimization model is optimized using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to obtain target parameter values ​​corresponding to each parameter to be optimized for network scheduling of the cognitive wireless network system.

[0162] The specific implementation process of the above method steps can be referred to the embodiments of the edge intelligent scheduling method of any cognitive wireless network mentioned above, and this embodiment will not be repeated here.

[0163] The present application constructs a cognitive wireless network system; based on various preset constraints, with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, an independent time slot optimization model is constructed using the Lyapunov optimization method, and the independent time slot optimization model includes various parameters to be optimized; multiple cognitive base stations in the present application collaboratively calculate the single task of offloading, and allocate computing resources to perform edge computing. In order to achieve harmonious coexistence on the licensed spectrum, the cognitive terminal device and the cognitive base station will consider the peak interference to the main user spectrum, the maximum transmission power of the cognitive terminal device, and the co-channel interference from other unloaded cognitive terminal devices, and strictly control their transmission power. The independent time slot optimization model is reconstructed using a preset multi-agent Markov decision process model to obtain a target optimization model; based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target optimization model is optimized using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to obtain the target parameter values ​​corresponding to each of the parameters to be optimized. Task and resource scheduling in a dynamic network environment is achieved, and the long-term average system overhead is reduced.

[0164] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. An edge intelligent scheduling method for a cognitive wireless network, characterized in that: include: Build cognitive wireless network systems; Based on the preset constraints, with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, an independent time slot optimization model for network scheduling of the cognitive wireless network system is constructed by using the Lyapunov optimization method, wherein the independent time slot optimization model includes various parameters to be optimized, and each of the preset constraints includes at least a preset transmission power constraint condition limited by the spectrum usage of the primary user; Reconstructing the independent time slot optimization model using a preset multi-agent Markov decision process model to obtain a target optimization model; Based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target optimization model is optimized by using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to obtain target parameter values ​​corresponding to each parameter to be optimized for network scheduling of the cognitive wireless network system; The cognitive wireless network system includes an edge computing cognitive wireless network and a primary network corresponding to multiple primary users; The edge computing cognitive wireless network includes multiple cognitive base stations and multiple cognitive terminal devices; The parameters to be optimized include the ratio of end-side tasks allocated by each cognitive terminal device to each cognitive base station, the transmission power when each cognitive terminal device transmits tasks to each cognitive base station, and the ratio of computing resources allocated when each cognitive base station calculates the tasks generated by each cognitive terminal device.

2. The method according to claim 1, characterized in that Before constructing an independent time slot optimization model for network scheduling of the cognitive wireless network system using a Lyapunov optimization method based on each preset constraint condition and with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, the method further includes: constructing each preset constraint condition; The construction of the preset constraint conditions specifically includes: Based on the local computing task ratio parameter corresponding to each of the cognitive terminal devices and the edge computing task ratio parameter assigned to the corresponding cognitive base station by each of the terminal devices, a constraint condition is constructed to obtain a preset task allocation ratio constraint condition corresponding to each of the cognitive terminal devices; Based on the predetermined maximum transmit power parameter supported by each cognitive terminal device, the peak interference power acceptable to each primary user, the transmit power of each cognitive terminal device, and the channel power gain of each cognitive terminal device to each primary user, a constraint condition is constructed to obtain a preset transmit power constraint condition; Constructing constraint conditions based on a predetermined maximum computing frequency parameter of each cognitive base station to obtain a preset computing resource constraint condition; Constraint conditions are constructed based on the energy consumption threshold of each cognitive terminal device in each time slot and a preset total number of time slots to obtain preset long-term energy consumption constraint conditions.

3. The method according to claim 1, characterized in that Based on the preset constraints, with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, an independent time slot optimization model for network scheduling of the cognitive wireless network system is constructed using the Lyapunov optimization method, specifically including: Based on the task data volume of each cognitive terminal device in different time slots, the terminal-edge task allocation ratio of each cognitive terminal device, the required computing frequency of each predetermined unit task, the computing frequency allocated to each cognitive terminal device by each cognitive base station, and the task transmission rate between each cognitive terminal device and each cognitive base station, a function is constructed to obtain an edge computing delay function corresponding to each cognitive base station; Based on the task data volume of each cognitive terminal device in different time slots, the terminal-side task allocation ratio of each cognitive terminal device, the required computing frequency of each predetermined unit task, and the computing frequency of each cognitive terminal device, a function is constructed to obtain a local computing delay function corresponding to each cognitive terminal device; A function is constructed based on the task data volume of each cognitive terminal device in different time slots, the terminal-side task allocation ratio of each cognitive terminal device, the transmission power of each cognitive terminal device, and the task transmission rate between each cognitive terminal device and each cognitive base station, to obtain a transmission energy consumption function corresponding to each cognitive terminal device; Based on the task data volume of each cognitive terminal device in different time slots, the terminal-side task allocation ratio of each cognitive terminal device, the predetermined energy coefficient of each cognitive terminal device, the required computing frequency of each predetermined unit task, and the computing frequency of each cognitive terminal device, a function is constructed to obtain a local computing energy consumption function corresponding to each cognitive terminal device; A model is constructed based on each of the edge computing delay functions, each of the local computing delay functions, each of the transmission energy consumption functions, each of the local computing energy consumption functions, and a predetermined weight coefficient to obtain an initial optimization model with the goal of minimizing the long-term average system overhead; The initial optimization model is decoupled by using the Lyapunov optimization method to obtain the independent time slot optimization model.

4. The method according to claim 3, characterized in that The initial optimization model is decoupled using the Lyapunov optimization method to obtain the independent time slot optimization model, which specifically includes: A virtual energy consumption queue of the cognitive wireless network system is constructed based on the edge computing delay and transmission energy consumption of each cognitive edge device in different time slots; Constructing a function based on the virtual energy consumption queue to obtain an initial Lyapunov function; Perform function construction based on the virtual energy consumption queue and the initial Lyapunov function to obtain an independent time slot Lyapunov drift function; Based on the initial optimization model and the independent time slot Lyapunov drift function, a function is constructed to obtain an initial Lyapunov drift penalty function; respectively constructing a first constraint condition of the independent time slot Lyapunov drift function and a second constraint condition of the initial Lyapunov drift penalty function; Reconstructing the initial Lyapunov drift penalty function based on the initial optimization model, the first constraint condition, and the second constraint condition to obtain a preset independent time slot Lyapunov drift penalty function; Model reconstruction is performed based on the preset independent time slot Lyapunov drift penalty function and the initial optimization model to obtain the independent time slot optimization model.

5. The method according to claim 1, characterized in that The independent time slot optimization model is reconstructed using a preset multi-agent Markov decision process model to obtain a target optimization model, which specifically includes: Constructing a function based on the delay parameters, energy consumption parameters and predetermined penalty function coefficients of each cognitive terminal device in different time slots to obtain a reward function corresponding to each cognitive terminal device; Constructing a weighted function based on each of the reward functions to obtain a target reward function; A model is constructed based on the target reward function, the agent set, the system state set and the action set of the cognitive wireless network system to obtain the preset multi-agent Markov decision process model; Based on the preset multi-agent Markov decision process model and the target reward function, the independent time slot optimization model of cognitive wireless network edge intelligent scheduling is transformed to obtain a target optimization model for scheduling the cognitive wireless network system.

6. The method according to claim 1, characterized in that Before optimizing the target optimization model using a preset deep reinforcement learning algorithm model based on proximal strategy optimization based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the method further includes: constructing a preset deep reinforcement learning algorithm model based on proximal strategy optimization; The construction presets a deep reinforcement learning algorithm model based on proximal strategy optimization, specifically including: Construct the initial strategy network model and the initial value network model respectively; Based on historical experience data, an update rate function of a preset strategy network, an advantage estimation function of a preset value network, a preset target update function, and a preset loss function, the model parameters of the initial strategy network model and the initial value network model are updated in a loop iteration to obtain a target strategy network model and a target value network model; A model is constructed based on the target strategy network model and the target value network model to obtain the preset deep reinforcement learning algorithm model based on proximal strategy optimization.

7. The method according to claim 6, characterized in that The observation information of each cognitive terminal device in the cognitive wireless network system collected in real time is optimized by using a preset deep reinforcement learning algorithm model based on proximal strategy optimization to optimize the target optimization model, and obtains the target parameter value corresponding to each parameter to be optimized for network scheduling of the cognitive wireless network system, specifically including: Based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time, the target policy network model in the deep reinforcement learning algorithm model preset based on proximal policy optimization is used to perform state prediction to obtain the target parameter value corresponding to each parameter to be optimized.

8. An edge intelligent scheduling device for a cognitive wireless network, characterized in that: include: System building module, used to build cognitive wireless network system; The cognitive wireless network system includes an edge computing cognitive wireless network and a primary network corresponding to multiple primary users; The edge computing cognitive wireless network includes multiple cognitive base stations and multiple cognitive terminal devices; A model building module, for building an independent time slot optimization model for network scheduling of the cognitive wireless network system based on various preset constraints and with the goal of minimizing the long-term average system overhead of the cognitive wireless network system, using the Lyapunov optimization method, wherein the independent time slot optimization model includes various parameters to be optimized, and each of the preset constraints at least includes a preset transmission power constraint constrained by the primary user spectrum; each of the parameters to be optimized includes the end-side task allocation ratio of each of the cognitive terminal devices to each of the cognitive base stations, the transmission power when each of the cognitive terminal devices transmits a task to each of the cognitive base stations, and the computing resource allocation ratio allocated when each of the cognitive base stations calculates the tasks generated by each of the cognitive terminal devices; A model reconstruction module, used to reconstruct the independent time slot optimization model using a preset multi-agent Markov decision process model to obtain a target optimization model; An optimization module is used to optimize the target optimization model based on the observation information of each cognitive terminal device in the cognitive wireless network system collected in real time by using a preset deep reinforcement learning algorithm model based on proximal strategy optimization, so as to obtain target parameter values ​​corresponding to each parameter to be optimized for network scheduling of the cognitive wireless network system.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the edge intelligent scheduling method of the cognitive wireless network described in any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: At least includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program on the memory, the processor implements the steps of the edge intelligent scheduling method of the cognitive wireless network described in any one of claims 1 to 7.

Images