Hierarchical heterogeneous network resource management data unloading method based on task driving

Abstract

The invention provides a hierarchical heterogeneous network resource management data unloading method based on task driving. Comprising the following steps: modeling a multi-dimensional resource management and task scheduling problem of a heterogeneous multi-access MEC network; determining the priority of task processing, and performing task scheduling and data unloading processing by adopting a data unloading strategy; after the intelligent reaction type jammer detects that the data of the heterogeneous users is unloaded, the data transmission unloading behavior is hindered in a tracking interference or dynamic interference mode; internal users of the heterogeneous multi-access MEC network adjust respective association matching relationships with the wireless access nodes / base stations; and the heterogeneous users adjust respective association strategies and unloading parameters according to perceived jammer behaviors, transmission requirements of own tasks and data unloading behaviors of other users. The model is complete, reasonable and effective.

Description

Technical Field

[0001] The present application relates to the field of wireless communication technology, and in particular to a task-driven hierarchical heterogeneous network resource management data unloading method. Background Art

[0002] With the rapid development of hardware devices and communication technologies, the communication and perception capabilities of mobile users are constantly improving, and the number of IoT devices is growing. Based on delay-sensitive and computationally intensive mobile applications, such as autonomous driving and augmented reality, the overall demand for wireless communication networks is growing exponentially. These applications usually require a huge amount of computing resources. However, the limited computing power of mobile devices themselves has become a key factor restricting their widespread deployment and efficient operation. MEC, as an emerging network architecture, aims to address the above challenges. By offloading computing tasks from mobile devices to nearby base stations (BSs) or wireless access points (APs), MEC technology effectively improves the utilization of computing resources, which can not only alleviate the computing load of the core network, but also meet the strict latency response requirements of the tasks.

[0003] Although the MEC architecture has significant advantages in solving mobile device problems, it also has certain limitations. Specifically, the computing power of a single node is limited, and it is difficult to cope with the concurrent service requests of massive mobile devices. Furthermore, the openness of wireless channels, the diversity of task requirements, and the mobility of devices lead to constant changes in mobile user access requests and frequent adjustments to network topology and association relationships (References: Zhang Y, Kishk MA, Alouini MS. Computation offloading and service caching in heterogeneous MEC wireless networks[J]. IEEE Transactions on Mobile Computing, 2021, 22(6): 3241-3256. References: Wang P, Di B, Song L, et al. Multi-layer computation offloading in distributed heterogeneous mobile edge computing networks[J]. IEEE Transactions on Cognitive Communications and Networking, 2022, 8(2): 1301-1315.), which can easily cause local congestion of network links, leading to load imbalance of edge servers, significantly reducing the overall performance of the system and the user's service experience. In addition, in complex IoT scenarios, the user heterogeneity caused by the significant differences in mobile user types and hardware configurations further exacerbates the irrational allocation of multi-dimensional resources (such as computing resources, storage resources, communication resources, etc.) in MEC networks. In addition, in complex electromagnetic spectrum environments, the intelligence level of malicious interference technology continues to improve, and its interference patterns tend to be diversified. Intelligent Reactive Jammers (IRJs) with perception capabilities pose a more severe threat to users in heterogeneous MEC systems.However, most existing reinforcement learning-based resource allocation schemes (references: Lv Z, Xiao L, Du Y, et al. Efficient Communications in Multi-Agent Reinforcement Learning for Mobile Applications [J]. IEEE Transactions on Wireless Communications, 2024. References: Wang P, Zheng Z, Di B, et al. HetMEC: Latency-optimal task assignment and re-source allocation for heterogeneous multi-layer mobile edge computing [J]. IEEE Transactions on Wireless Communications, 2019, 18 (10): 4942-4956.) mainly focus on fixed network topology scenarios, ignoring the rapid dynamic changes of network topology and the QoS requirements of heterogeneous tasks, and lack in-depth research on the adaptability and effectiveness of multi-dimensional resource allocation management methods in heterogeneous MEC networks. As a result, in practical applications, edge computing servers are prone to load imbalance and link congestion when facing sudden requests and data surges, which seriously limits the performance of intelligent learning algorithms. Therefore, there is an urgent need to study new multi-agent collaborative anti-interference data offloading algorithms to solve the above problems. Summary of the invention

[0004] The present application provides a task-driven hierarchical heterogeneous network resource management data offloading method, which can be used to solve the technical problem that the specific needs of different users in heterogeneous MEC networks in processing heterogeneous tasks in terms of priority, security, latency and energy consumption are not met.

[0005] A task-driven hierarchical heterogeneous network resource management data offloading method, the method comprising:

[0006] Step 1: Model the task-driven multi-dimensional resource management and task scheduling problems in heterogeneous MEC networks; describe it as a partially observable Markov game process, where the participants are heterogeneous users in the heterogeneous network, and optimize the distributed data offloading decision by combining task scheduling, dynamic spectrum access, and dynamic computing resource management;

[0007] Step 2: Heterogeneous users in the heterogeneous network design reward weights based on the queue information of their own cache tasks and task-related information including latency requirements and information age, determine the priority of task processing, and use appropriate edge computing decisions for data processing;

[0008] After the intelligent reactive jammer detects the data offloading behavior of heterogeneous users, it uses tracking interference or dynamic interference to block the data transmission offloading behavior;

[0009] Step 3: Internal users of the heterogeneous network adjust their association and matching relationships with wireless access nodes / base stations according to the current association status of base stations / wireless access nodes and the computing power requirements of their own processing tasks, so as to meet the dynamic computing power adaptation of heterogeneous users in the entire network in processing tasks;

[0010] Step 4: Based on the interconnection of heterogeneous network devices, internal users of the heterogeneous network adjust their own edge computing strategies including channel access, data offloading ratio, transmission power and task scheduling according to the perceived external jammer behavior and data offloading behavior of other users, so as to achieve the optimization goal of maximizing the satisfaction of internal users' needs;

[0011] Step 5: When multiple users in the heterogeneous network complete the adjustment, the crowd intelligence algorithm completes one iteration. When the multi-dimensional resources in all heterogeneous networks are reasonably allocated or the set number of iterations is reached, the process ends.

[0012] Furthermore, the modeling of task-driven multi-dimensional resource management and task scheduling problems in heterogeneous MEC networks includes:

[0013] Step 1.1, model the task-driven multidimensional resource management and task scheduling problems in heterogeneous MEC networks as multi-agent nonlinear mixed integer programming decision problems, and analyze the local computing and mobile edge offloading problems of users within heterogeneous networks.

[0014] Model the adversarial intelligent reactive multi-dimensional resource management and task scheduling task offloading problems as multi-objective optimization functions with a common optimization goal;

[0015] A multi-access MEC model for wide-area heterogeneous networks is considered, which includes multiple different types of mobile devices MUs, a central base station BS, multiple wireless access points APs, and intelligent reactive jammers IRJ, aiming to build a complex and efficient edge computing environment. The system model provides computing power support for MUs through edge computing servers deployed on BS and AP nodes; edge computing servers are installed on base stations and wireless access nodes respectively to provide computing resource access services to MUs; F i lo Indicates MU iThe computing power of Indicates that the AP / BS is assigned to the MU i The computing power of AP / BS is related to the number of associated users. The computing power allocation rule is to evenly distribute the computing power of AP / BS based on the number of associated users. In terms of mobile offloading, MUs has a built-in intelligent algorithm module that can formulate an offloading strategy based on the environmental spectrum information and its own data offloading requirements, and offload part or all of the computing tasks to BS / AP for auxiliary task processing. The external available channel set is set to and meet The number of mobile devices is N; for mobile user i, each task consists of three tuples (d i (t),c i (t),τ i (t)), where d i (t) is the data size of the computing task; c i (t) represents the number of CPU cycles required for the computation task, τ i (t) gives the maximum tolerable delay time of the computing task; when the tth computing task is generated or arrives at a mobile user end, the mobile user system must determine whether the task needs to be partially or completely offloaded to the base station or only calculated locally;

[0016] For the tth computing task, MUs decide whether to offload to a specific AP or execute locally based on task characteristics (such as real-time performance and data volume), spectrum resources, and computing resources; i,m (t) indicates AP m and MU i The association when processing task t; where o i,m (t) = 1, indicating MU i With AP m connected; otherwise, o i,m (t) = 0; in the process of task offloading of the mobile user MU, the selected access channel, i.e., the access frequency point and the device, the offloading ratio, the transmission power and the MU i The unloading task is the key factor affecting users to unload data; they are defined as α i (α i ∈[0,1]),P i,m (P i,m ι[P i,min ,P i,max ]), If the access channel quality is good, MU further optimizes the offloading strategy by adjusting the offloading ratio α i With the transmission power P i,m , aims to achieve efficient and secure offloading of mission data.

[0017] All heterogeneous users, wireless access nodes, and base stations in the heterogeneous multi-access MEC network are divided into send-receive pairs, and all heterogeneous users can choose different base stations (BS) / wireless access nodes (AP) for task offloading. In the heterogeneous network, there is an intelligent reactive jammer that can carry out targeted indiscriminate attacks on the unloaded data according to the data offloading behavior of heterogeneous users. However, when the reactive jammer cannot detect the user's data offloading behavior, the reactive jammer uses dynamic interference to pollute the external spectrum environment, thereby hindering the data offloading behavior of users in the heterogeneous network. The interference effect of the intelligent reactive jammer is related to its interference power, channel gain, and perception ability; there is mutual interference between users in the heterogeneous multi-access MEC network, and the size of the mutual interference is related to the mutual interference coefficient, the selected transmission channel, and the transmission power.

[0018] Step 1.2, model and analyze the multi-dimensional resource management and task scheduling problems of heterogeneous network data offloading, discuss the mobile edge computing problems of heterogeneous tasks (delay-sensitive applications and computation-intensive applications) in heterogeneous networks, and model and describe delay-sensitive applications and computation-intensive applications.

[0019] Heterogeneous users need to offload data to APs based on task type, timeliness, and priority to meet diverse QoS requirements; all heterogeneous users are set to periodically generate and process tasks with different QoS requirements; the communication services involved in the tasks are divided into two categories: delay-sensitive application services and computing-intensive application services; delay-sensitive applications give priority to the security of data transmission and pursue high reliability and low latency; applications ensure data security by exchanging key messages between MUs / APs; and computing-intensive applications mainly meet users' general entertainment services, involving a large amount of data transmission and complex calculations; although computing-intensive applications require high data rates, they have a higher tolerance for reliability and latency. The specific modeling process of the two service applications is as follows:

[0020] Delay-sensitive application model: using a six-dimensional tuple model (d i (t),c i (t),θ i (t),τ i (t),ξ i (t),L i (t))) to characterize latency-sensitive application services. The tuple model consists of the following elements: data size d i (t), calculate the number of CPU cycles c required i (t), maximum tolerable delay θ i (t), maximum AoI limit τ i (t), task urgency index ξi (t), and the task type identifier L i (t); where L i (t) = 1 represents delay sensitive type, L i (t) = 0 represents computation intensive; the local computation delay T of a heterogeneous user processing the tth computation task i l (t) is as follows:

[0021]

[0022] MU i Tolerable delay θ i The constraint on (t) is Therefore, based on local computing, the energy consumption in the tth computing task is The expression is:

[0023]

[0024] in For MU i The effective capacitance switch parameters of heterogeneous users adopt partial offloading mode for delay-sensitive applications; MU i Offloading tasks to AP m The SINR utility function is defined as:

[0025]

[0026] Where P i Indicates MU i The transmission power, Indicates intelligent response J k Interference power, W i,m For MU i With AP m The channel gain of the transmission link between For IRJJ k With AP m The channel gain of the interfering link between n is the transmission power of other MUs, W n,m For other users and APs in the same frequency band m The link gain of AP m The noise spectral density of

[0027] According to the channel capacity theory, user MU i To base station AP m The unload transfer rate R i,m (t) is:

[0028]

[0029] where λ m Indicates AP m Therefore, for the tth computing task, MU i The total offload delay is expressed as:

[0030]

[0031] Transmission energy consumption of the tth computing task for:

[0032]

[0033] In order to ensure high transmission reliability and meet users' needs for reducing latency and saving energy, a normalized index of energy consumption of heterogeneous users when processing latency-sensitive application tasks is introduced. and latency normalization index The normalized index is calculated as follows:

[0034]

[0035] in

[0036] Computation-intensive application model: Considering the high data rate requirements of computation-intensive applications, the utility function of such applications can be modeled by offloading throughput. i For example, computationally intensive tasks are modeled as a four-dimensional tuple To model: where d i (t) is the data size, c i (t) is the number of CPU cycles required for the calculation, represents the minimum transfer rate that allows computationally intensive tasks to be offloaded; therefore, MU i The transfer rate R that allows the transfer of computationally intensive tasks i,m (t) is denoted as:

[0037]

[0038] For computationally intensive applications, mobile users adopt binary offloading mode; for the tth computationally intensive task, the normalized utility function is equal to MU i is closely related to the transmission throughput, which is defined as:

[0039]

[0040] Furthermore, in step 2, the mobile users in the heterogeneous MEC network design reward weighted values ​​according to the task type, task information, information age and other related heterogeneous task requirements of their own offloaded tasks, determine the priority of related task processing, and adopt appropriate edge computing decisions for data processing; the reward weighted values ​​are as follows:

[0041] For latency-sensitive applications, data freshness is extremely important. The freshness of data information is quantified by AoI, which represents the time delay from data packet generation to task processing. The longer the delay, the lower the freshness of the task. Latency-sensitive applications adopt a critical task priority transmission processing mode, that is, before processing each latency-sensitive application task, the user will evaluate the status of its task queue and the freshness of the information to decide whether to prioritize transmission or discard the data in the queue. Computation-intensive application data is processed according to the order of arrival, using a first-come, first-served processing mode.

[0042] New data packets enter the user's buffer according to the mean value λ of the Poisson distribution; the data packets in the user's internal cache are distributed with Y i <ξ,τ> , where ξ represents the urgency index and τ represents the maximum AoI limit. The urgency level of each data packet is ξ∈{1,2,…,v}, where ξ=0 means the task queue is empty; when MU i When the processed task exceeds the maximum AoI threshold, the corresponding information will be removed from the cache; when the task processed by the user exceeds the maximum AoI threshold, the task is discarded and the relevant information is removed from the emergency state vector; based on this, the weighted value of the relevant reward function is determined for the characteristics of delay-sensitive applications to encourage priority transmission behavior, where the reward weight value ι i Defined as:

[0043]

[0044] where t age Indicates the waiting time of a task before transmission.

[0045] Furthermore, in step 3, the internal users of the heterogeneous network adjust their respective association matching relationships with the wireless access nodes / base stations according to the association status of the current base stations / wireless access nodes and the computing power requirements of the tasks they need to process, so as to meet the dynamic computing power adaptation of the heterogeneous users in the entire network in processing tasks;

[0046] This application provides a fast matching algorithm that combines matching game and potential game, aiming to achieve many-to-many Aps-Mus fast matching, and consider peer effect and satisfaction factors to optimize computing resource allocation and improve network performance. The relevant association matching satisfaction function is formulated:

[0047]

[0048] Where f(MU i ,AP m ) indicates MU i With AP m The association satisfaction between them ranges from 0 to 1; ζ is a constant ζ ≥ 7; when AP m Assigned to MU i The computing power of i,m Exceeds the task computing power requirement δ′ i,m When , the matching satisfaction function satisfies: The matching satisfaction function uses an S-shaped curve to represent the relationship between the computing power requirements of different tasks and the matching satisfaction; i,m Indicates the urgency of the unloading task and reflects the changing trend of the curve; the lower the i,m Corresponding to a flatter utility curve slope, it indicates that the task requires less computing power; on the contrary, ι i,m The higher the value of is, the steeper the curve is, indicating that the task has a greater demand for computing resources; the global optimization problem of association satisfaction is defined as:

[0049]

[0050] Among them, the association satisfaction of the whole network is Λ; Equation (24) aims to maximize the satisfaction of the whole network by optimizing the association access strategy of all MUs; MU and AP have autonomous decision-making capabilities: MU needs to select a suitable AP and calculate whether the computing resources allocated by its device association meet the demand; AP determines whether to accept the access request of the current MU based on the network satisfaction of the whole network;

[0051] MUs collect external channel state information CSI and build their preferred AP association access priority ranking based on their own task processing capabilities and computing power requirements. The proposed fast matching game algorithm is divided into two stages:

[0052] In Phase I, users need to rebuild the priority list for APs based on the CSI information and location information perceived by the outside world: mobile users that do not meet the task offloading requirements will make matching requests to their preferred wireless access nodes, with the number of links not exceeding the number of MU associations allowed to be equipped; mobile users that have met the task offloading requirements will reduce the number of options for trying to access other APs;

[0053] In phase II, after receiving the matching request from the mobile user, the AP will exchange information with the relevant APs and determine whether to accept the user's matching request based on specific criteria (such as formula (24)); this decision-making process takes into account multiple factors, including the overall performance of the network, user satisfaction, and the effective use of computing resources. If the change in the matching result can improve the satisfaction of the entire network, the user's access request will be accepted.

[0054] Step 4: Internal users in heterogeneous networks adjust their task scheduling strategies and edge computing strategies based on the current network matching relationship, perceived jammer behavior, and other users’ task offloading behavior, so as to achieve the optimization goal of maximizing the satisfaction of internal users’ needs.

[0055] Furthermore, in step 4, internal users in the heterogeneous network adjust their respective association relationships, task scheduling strategies, and edge computing strategies based on the perceived jammer behavior and other users' task offloading behavior, so as to achieve the optimization goal of maximizing the satisfaction of internal users' needs in the network.

[0056] The user demand satisfaction function based on heterogeneous tasks is:

[0057]

[0058] where μ and x i,m Represents the weight value 0≤μ≤1 and the security transmission index x i,m ={-1,0,1}; MU sets the weight value μ according to the device performance and the current task's preference for latency and energy consumption; During the data offloading process, ensuring data security and task calculation success is the basic requirement of auxiliary computing tasks. Among these requirements, data security is particularly important. Introducing the secure transmission index x i,m To measure the safety of task offloading; x i,m =1 means the safety of the unloading process can be guaranteed and the task calculation is successfully executed x i,m =0 means that the security of the uninstall process can be guaranteed, but the user task processing fails (f i,m ≠f J 、max(T i tot (t),T i l (t))>θ i (t) or ); x i,m =-1 means that the task is interfered with during the unloading process.

[0059] This application determines a distributed multi-dimensional network resource management solution for the offloading task type, priority, and computing cost of users in the HetMEC network. The optimization goal of this solution is to maximize the satisfaction of heterogeneous users in the entire network while ensuring that the QoS requirements of heterogeneous tasks are met. Therefore, the optimization problem is expressed as:

[0060] In the optimization problem, the goal is to find an optimal data offloading decision {P i ,f i ,α i ,Y i}, so that the demand satisfaction RS i (t) maximization, that is, while ensuring that the task aims to minimize the computational cost of each user in terms of delay and energy consumption, while ensuring the security of task offloading in the presence of eavesdropping jammers; the in-network optimization problem is expressed as:

[0061]

[0062] Where f, α are the heterogeneous users connected to the base station / wireless access node The selected frequency, power, and unloading ratio resource allocation matrix; Y i (t) represents MU i The transmission task selected at time t; the constraints represent the range of multi-dimensional heterogeneous decisions and tasks selected by the MU.

[0063] The fusion of hierarchical optimization strategy and multi-agent game learning mechanism facilitates cross-layer precise management and dynamic adjustment of network resources. The upper-layer algorithm module is deployed in base stations and wireless access nodes, and is responsible for channel access of users in the HetMEC network using the Q-mix architecture and matching game; the lower-layer algorithm module is deployed on heterogeneous users, and a distributed heterogeneous multi-agent reinforcement learning mechanism is used to decide data offloading strategies including data offloading ratio, transmission power and task scheduling. The upper-layer algorithm module is deployed in base stations and wireless access nodes to supervise the global network resource allocation and matching, while the lower-layer algorithm focuses on local optimization to meet the QoS requirements of heterogeneous tasks. Specifically, the upper-layer algorithm module uses the Q-mix architecture and matching game to be mainly responsible for the division of spectrum resources and the reasonable allocation of computing power resources in the HetMEC network; the upper-layer reinforcement learning algorithm uses the reinforcement learning mechanism to adaptively adjust the spectrum resource allocation to effectively avoid dynamic interference. At the same time, the many-to-many fast non-replacement matching game algorithm quickly matches and associates base stations / APs with heterogeneous users to ensure efficient allocation of computing power. The lower-level algorithm module is deployed on heterogeneous users, and a distributed heterogeneous multi-agent reinforcement learning mechanism is used to solve the data offloading behavior between heterogeneous users. The optimal data offloading strategy is formulated by analyzing real-time spectrum information, wireless access node computing power allocation, and heterogeneous task requirements.

[0064] Step 5: When multiple users in the heterogeneous user group complete the above adjustments, the crowd intelligence algorithm completes one iteration. When the multi-dimensional communication resources of all heterogeneous network users are reasonably allocated or the set number of iterations is reached, the process ends, including:

[0065] Step 5.1: The mobile user observes the current state through channel perception and randomly selects the upper channel access decision based on probability ε and lower-level data offloading actions

[0066] Step 5.2: Uninstall the data when the user After that, the base station will send the feedback reward back to the user according to the data unloading situation and input the generated data into the neural network for fitting, and the historical reward experience Store in the experience revisit pool and obtain the new state O i,m (t+1);

[0067] Step 5.3, when i∈L high The Q function of the upper network is fitted When i∈L low The Q function of the underlying network is fitted And calculate the expected return based on the reward, and update the upper and lower Q-mix networks and their respective sub-network parameters with the minimum loss;

[0068] Step 5.4 iterates in a loop until the maximum number of iterations is reached, and the optimal edge computing decision for users in the heterogeneous network is found to meet the computing and offloading requirements of each user's task.

[0069] Furthermore, the specific process of neural network fitting in step 5.3 is as follows:

[0070] The upper layer network is deployed on the AP, and the Q-Mix network architecture is used to manage and allocate channel resources of all APs in the HetMEC network. Each AP is responsible for selecting the optimal channel access strategy for its associated MUs to avoid severe channel fading and dynamic interference attacks within the network, and to prevent frequency conflicts between MUs covered by the same AP or adjacent APs. The high-order Q-Mix hybrid network is deployed on the largest AP, i.e., BS, and collects the Q-network parameters of all APs to maximize the channel resource benefits. In the high-level network, each MU sends local observations to its AP, and the AP selects a discrete channel access strategy for its MU based on the observations. DDQN uses the advantages of the dual neural network in DQN to alleviate the over-estimation problem in traditional methods; the Q network is trained by minimizing the iterative loss function, which is expressed as:

[0071]

[0072] The hybrid network of the lower layer Q-mix algorithm is deployed on the AP to maximize the user satisfaction of all MUs associated with it;

[0073] For the lower layer network part, MU i The continuous parameter χ′ is chosen according to its new observations:

[0074]

[0075] in It is the channel access and device association strategy obtained from the high-level network; it converts the task information, location information, current channel information and Combine into a new state And input to the lower-level network; comprehensively consider the dynamic channel access decision and local observation information, use the lower-level network advantage function The mixed decision of each agent is used as input to generate MU i The lower network value of , the update strategy is:

[0076]

[0077] in Indicates MU i The lower layer advantage function; y lowIt is the one-step target value obtained by the lower layer mixing network.

[0078] Compared with the prior art, the present invention has the following significant advantages: (1) It fully considers the interactive constraint mechanism among task scheduling, offloading strategy, dynamic allocation of communication resources and computing resources, and provides an efficient joint multi-dimensional network resource optimization and task scheduling problem modeling, with the optimization goal of maximizing the satisfaction of internal network users' needs, and realizes the quantitative solution of the problem. It accurately characterizes and models the task scheduling problem faced by heterogeneous users in processing heterogeneous tasks; a more accurate and physically clearer heterogeneous network resource management and task scheduling model can better characterize the user data offloading scenario in the heterogeneous MEC network; (2) It proposes an anti-interference data offloading algorithm based on task-driven hierarchical heterogeneous network resource management combined with dynamic spectrum access technology, which selects safe data offloading decisions and transmission channels through independent users' autonomous perception of the external channel environment, and introduces relevant system overhead factors to improve the stability of multi-user channel access decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a system model diagram of a heterogeneous network data offloading model for processing heterogeneous tasks provided by the present application;

[0080] Figure 2 It is an algorithm flow chart of the task-driven hierarchical heterogeneous network resource management data offloading algorithm provided by this application;

[0081] Figure 3 It is a schematic diagram of heterogeneous data packet queuing processing based on the task-driven hierarchical heterogeneous network resource management data offloading algorithm provided by the present application;

[0082] Figure 4 It is a performance comparison chart of the task-driven hierarchical heterogeneous network resource management data offloading algorithm that resists tracking interference and dynamic random periodic interference in the embodiments provided in the present application. DETAILED DESCRIPTION

[0083] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0084] The following first introduces the embodiments of the present application in conjunction with the accompanying drawings.

[0085] The heterogeneous network multi-dimensional resource management data offloading model for processing heterogeneous tasks proposed in the present invention aims to more accurately characterize the multi-dimensional resource management and data offloading process in the heterogeneous multi-access MEC network where users process heterogeneous tasks. This model divides all heterogeneous, wireless access nodes and base stations in the heterogeneous multi-access MEC network into send-receive pairs, and all heterogeneous can select different base stations or wireless access nodes for task offloading. In the heterogeneous network, there is an intelligent reactive jammer that can track and interfere with the unloaded data according to the data offloading behavior of the heterogeneous user. However, when the intelligent reactive jammer cannot perceive the user's unloading behavior, the intelligent reactive jammer uses dynamic interference to hinder the data unloading of heterogeneous users in the heterogeneous network. The interference effect of the intelligent reactive jammer is related to its perception ability, interference power and channel gain; there is mutual interference between heterogeneous users in the heterogeneous network, and the size of the mutual interference is related to the mutual interference coefficient, the selected channel and the unloading power. The purpose of the present invention is to provide a hierarchical heterogeneous network resource management data offloading method based on task driving. Taking into account the specific needs of each heterogeneous user in terms of priority, security, and transmission cost in processing heterogeneous tasks, an efficient multi-dimensional resource joint optimization and task scheduling scheme is designed by analyzing the mutual constraints and interaction mechanisms between dynamic resource allocation, task scheduling, and edge offloading strategies, with the optimization goal of maximizing the satisfaction of internal network users' needs to solve heterogeneous tasks. While satisfying user security and delay constraints, the method of the present invention jointly optimizes the channel access, dynamic computing power scheduling, transmission power, and task scheduling of multiple users, and realizes the reasonable allocation of multi-dimensional resources and efficient data offloading.

[0086] Figure 1 This is a system model diagram of a heterogeneous network data offloading model for processing heterogeneous tasks. In the system model, heterogeneous mobile devices are equipped with intelligent modules. The mobile devices make corresponding mobile offloading decisions by sensing external information and their own data offloading requirements, and offload the data to the base station for auxiliary calculation. Among them, the reactive jammer has the ability to eavesdrop on the data information of the mobile terminal and interfere with data offloading. When the mobile terminal is unloading data, the reactive jammer perceives the user's data offloading behavior. When the user's unloading behavior is perceived by the reactive jammer, the reactive jammer launches tracking interference to destroy the data offloading behavior. When the reactive jammer cannot perceive the task offloading behavior, the eavesdropper uses common dynamic interference (sweep frequency, double sweep frequency, comb and other combined dynamics) to hinder the task offloading of heterogeneous users.

[0087] Figure 2This is the algorithm flow chart of the task-driven hierarchical heterogeneous network resource management data offloading algorithm proposed in the present invention. The proposed algorithm design pays close attention to user needs and network conditions. The core of the design is to integrate hierarchical optimization strategies and multi-agent game learning mechanisms to facilitate cross-layer precise management and dynamic adjustment of network resources. The upper-layer algorithm module is deployed in base stations and wireless access nodes to supervise global network resource allocation and matching, while the lower-layer algorithm focuses on local optimization to meet the QoS requirements of heterogeneous tasks. Specifically, the upper-layer algorithm module uses the Q-mix architecture and matching game to be mainly responsible for the division of spectrum resources and the reasonable allocation of computing power resources in heterogeneous MEC networks; the upper-layer reinforcement learning algorithm uses the reinforcement learning mechanism to adaptively adjust the spectrum resource allocation to effectively avoid dynamic interference. At the same time, the many-to-many fast non-replacement matching game algorithm quickly and accurately associates APs with Mus to ensure efficient computing power allocation. The lower-layer algorithm module is deployed on MUs, and a distributed heterogeneous multi-agent reinforcement learning mechanism is used to solve the data offloading behavior between heterogeneous MUs. This module formulates the optimal data offloading strategy by analyzing real-time spectrum information, AP computing power allocation, and heterogeneous task requirements.

[0088] Figure 3 It is a schematic diagram of the data packet queuing system of the task-driven hierarchical heterogeneous network resource management data offloading algorithm proposed in the present invention. The task scheduling method proposed in the present invention is mainly based on the freshness and urgency of task information. For delay-sensitive applications, data freshness is crucial, which is quantified by the age of information (AoI). AoI refers to the time delay experienced by a data packet from generation to task processing. The longer the delay, the lower the task freshness. Before processing each task, the user will evaluate the status of its own task queue and the freshness of the information to decide whether to give priority to transmission or discard the data in the queue. For compute-intensive applications, data is processed in sequence according to the order in which the tasks arrive. Similarly, before processing each task, the user will also evaluate the status of the task queue and the freshness of the information to determine whether to give priority to transmission or discard the data in the queue.

[0089] At each discrete time step, the user decides whether to prioritize or discard the data in the task queue by observing the execution status, information age, and external environment information of its own task queue. The environmental characteristics involved here have been described in the system model and channel model.

[0090] From the perspective of information theory, users need to use network resources to provide deterministic services for more high-priority tasks while ensuring safe unloading. The present invention considers the priority of task transmission and uses the urgency of the task itself as the linear weight of its information age, so as to achieve more reasonable and efficient task scheduling.

[0091] Figure 4It is a performance comparison chart of the task-driven hierarchical heterogeneous network resource management data offloading algorithm for counteracting intelligent reactive interference in an embodiment of the present invention.

[0092] The embodiment of the present invention is described in detail as follows. The system simulation adopts Python language and is based on the TensorFlow deep learning framework. The simulation scenario mainly consists of 3 APs (one base station and two microcell nodes), 6 MUs and 2 IRJs. The service coverage radius of the BS is set to 250m. The microcell nodes are symmetrically placed on both sides of the base station service area, and the service radius of each microcell is 80m, ensuring that there is no overlapping service coverage between microcells. The number of available channels in the outside world is set to 10, and the unloading bandwidth of MUs is 4MHz. APs, MUs and IRJs perform task unloading and confrontation in the 40MHz frequency band, allowing MUs to dynamically adjust channels, power and unloading ratios in each time slot. Mobile users move at a constant speed away from or close to the base station, and the speed remains unchanged during the period, which is set to 1m / s. The proposed algorithm is trained under different learning rates and heterogeneous task Qos configurations, which will significantly affect wireless communication when set differently. The transmission power ranges of heterogeneous users are 7dBm-20dBm and 10dBm-23dBm, respectively. Both the communication signal and the interference signal adopt the rising cosine waveform. The user task generation model is introduced to generate computing tasks of 200Kb-600Kb (uniformly distributed), and the task arrival process conforms to the Poisson distribution. The user computing power (CPU cycle number) is 1.5×10 8 cycles / s. The inherent computing power of mobile users is 1.5×10 8 cycles / s, and its energy efficiency factor is 10 -27 J / cycle. The computing power that the base station can allocate to each MU is 1×10 9 cycles / s. The total computing power that a microcell node can provide to connected users is 3×10 9 cycles / s, and its users evenly share the computing power of the micro-cell node providing the service.

[0093] Intelligent reactive jamming consists of a perception module, a tracking jamming module, and a conventional jamming module. In the conventional jamming module, a strategy combining dynamic jamming and conventional jamming modes is adopted to pollute the external spectrum environment. Specifically, IRJ intelligent reactive jamming randomly selects comb jamming and sweeping frequency jamming. The duration for maintaining the current type of jamming is randomly selected, and the duration selection range is 10s-20s. Comb jamming is divided into three fixed frequency bands: 4MHz-8MHz, 12MHz-16MHz, 20MHz-24MHz, and 28MHz-32MHz. In order to verify the performance superiority and effectiveness of the proposed algorithm, this chapter compares and deeply analyzes the network utility with several other MEC anti-jamming algorithms: In order to evaluate the effectiveness of the proposed algorithm, the task processing success rate with the other three anti-jamming data offloading algorithms is mainly compared:

[0094] 1. MA-PDQN algorithm: The MA-PQN algorithm combines the P-DQN algorithm with MADDPG to solve the distributed hybrid heterogeneous strategy problem.

[0095] 2. MADDPG algorithm: MADDPG is a MARL algorithm based on the CTDE paradigm. It discretizes the continuous action space into a discrete action space to implement channel access and task scheduling strategies.

[0096] 3. Q-mix algorithm: The traditional Q-mix algorithm is applied to data offloading in the HetMEC scenario. The algorithm does not adopt a hierarchical architecture and discretizes continuous decision dimensions such as offloading ratio and offloading power.

[0097] Figure 4 This is a performance comparison chart of the algorithm based on task scheduling and resource allocation for countering the intelligent reactive jammer in Example 1 of the present invention. Figure 4 It can be seen that the proposed algorithm is effective and practical, and energy-efficient communication resource scheduling and data offloading are achieved.

[0098] Depend on Figure 4 It can be seen that the proposed algorithm comprehensively considers the heterogeneity and mobility of mobile users, the impact of intelligent reactive interference and the dynamic changes of network topology, adopts the RTDE training model, and customizes its unique anti-interference data offloading strategy for heterogeneous MUs by perceiving the external channel environment, task characteristics and interference patterns.

[0099] In summary, the present invention organically integrates advanced intelligent algorithms such as matching games, multi-agent reinforcement learning (MARL), and federated learning, and proposes a task-driven hierarchical heterogeneous network resource management data offloading algorithm, which aims to effectively respond to and comprehensively consider the quality of service (QoS) requirements of time-varying heterogeneous tasks by jointly optimizing device association, network resource allocation, and task scheduling. The proposed task-driven hierarchical heterogeneous network resource management data offloading algorithm can effectively solve the NP-hard optimization problem in multi-dimensional resource management. Specifically, the algorithm architecture is divided into two layers: the upper-layer strategy focuses on the optimization of device association and spectrum access strategy, which can quickly respond to dynamic changes in network topology and quickly realize the association matching between wireless access nodes / base stations and mobile users.

[0100] The lower layer focuses on the fine-tuning of heterogeneous data offloading and task scheduling. By utilizing the collaboration and competition mechanism among intelligent agents, the algorithm can explore the optimal strategy under the constraints of limited resources.

[0101] On the one hand, the proposed algorithm can further improve resource utilization, so that various resources in the network can be more fully and reasonably utilized, avoiding resource waste; on the other hand, it can ensure the fairness of resource allocation, ensuring that different users and devices can obtain relatively fair resource shares in the network, thereby avoiding the impact of excessive or insufficient resource occupation on the overall network performance of some users or devices. This is of great significance for improving the operating efficiency and stability of heterogeneous multi-access MEC networks in intelligent reactive interference scenarios.

[0102] The above-described embodiments of the present application do not constitute a limitation on the protection scope of the present application.

Claims

1. A task-driven hierarchical heterogeneous network resource management data offloading method, characterized in that: The method comprises: Step 1: Model the task-driven multi-dimensional resource management and task scheduling problems in heterogeneous MEC networks. The modeling is described as a partially observable Markov game process, where the participants are heterogeneous users in the heterogeneous network, and joint task scheduling, dynamic spectrum access, and dynamic computing resource management are used to optimize distributed data offloading decisions. Step 2: Heterogeneous users in the heterogeneous network design reward weights based on the queue information of their own cache tasks and task-related information including latency requirements and information age, determine the priority of task processing, and use appropriate edge computing decisions for data processing; After the intelligent reactive jammer detects the data offloading behavior of heterogeneous users, it uses tracking interference or dynamic interference to block the data transmission offloading behavior; Step 3: Internal users of the heterogeneous network adjust their association and matching relationships with wireless access nodes / base stations according to the current association status of base stations / wireless access nodes and the computing power requirements of their own processing tasks, so as to meet the dynamic computing power adaptation of heterogeneous users in the entire network in processing tasks; Step 4: Based on the interconnection of heterogeneous network devices, internal users of the heterogeneous network adjust their own edge computing strategies including channel access, data offloading ratio, transmission power and task scheduling according to the perceived external jammer behavior and data offloading behavior of other users, so as to achieve the optimization goal of maximizing the satisfaction of internal users' needs; Step 5: When multiple users in the heterogeneous network complete the adjustment, the crowd intelligence algorithm completes one iteration. When the multi-dimensional resources in all heterogeneous networks are reasonably allocated or the set number of iterations is reached, the process ends.

2. The method according to claim 1, characterized in that Step 1: Model the task-driven multi-dimensional resource management and task scheduling problems in heterogeneous MEC networks, including: Step 1.1, model the task-driven multi-dimensional resource management and task scheduling problems in heterogeneous MEC networks as multi-agent nonlinear mixed integer programming decision problems, and analyze the local computing and mobile edge offloading problems of users within heterogeneous networks; Step 1.2, model and analyze the multi-dimensional resource management and task scheduling problems of heterogeneous network data offloading, analyze the mobile edge computing problems of heterogeneous tasks such as delay-sensitive applications and computing-intensive applications in heterogeneous networks, and model and describe delay-sensitive applications and computing-intensive applications.

3. The method according to claim 2, characterized in that Step 1.1: Model the task-driven multidimensional resource management and task scheduling problems in heterogeneous MEC networks as multi-agent nonlinear mixed integer programming decision problems, and analyze the local computing and mobile edge offloading problems of users within heterogeneous networks, including: Model the adversarial intelligent reactive multi-dimensional resource management and task scheduling task offloading problems as multi-objective optimization functions with a common optimization goal; A multi-access MEC model for wide-area heterogeneous networks is considered, which includes multiple different types of mobile devices MUs, a central base station BS, multiple wireless access points APs, and intelligent reactive jammers IRJ. Edge computing servers deployed on BS and AP nodes provide computing power support for MUs; edge computing servers are installed on base stations and wireless access nodes respectively to provide computing resource access services to MUs; Indicates MU i The computing power of Indicates that the AP / BS is assigned to the MU i The computing power of AP / BS is related to the number of associated users. The computing power allocation rule is to evenly distribute the computing power of AP / BS based on the number of associated users. In terms of mobile offloading, MUs has a built-in intelligent algorithm module that can formulate an offloading strategy based on the environmental spectrum information and its own data offloading requirements, and offload part or all of the computing tasks to BS / AP for auxiliary task processing. The external available channel set is set to and meet The number of mobile devices is N; for mobile user i, each task consists of three tuples (d i (t),c i (t),τ i (t)), where d i (t) is the data size of the computing task; c i (t) represents the number of CPU cycles required for the computation task, τ i (t) gives the maximum tolerable delay time of the computing task; when the tth computing task is generated or arrives at a mobile user end, the mobile user system must determine whether the task needs to be partially or completely offloaded to the base station or only calculated locally; For the tth computing task, MUs decide whether to offload to a specific AP or execute locally based on task characteristics, spectrum resources, and computing resources; i,m (t) indicates AP m and MU i The association when processing task t; where o i,m (t) = 1, indicating MU i With AP m connected; otherwise, o i,m (t) = 0; in the process of task offloading of the mobile user MU, the selected access channel, i.e., the access frequency point and the device, the offloading ratio, the transmission power and the MU i The unloading task is the key factor affecting users to unload data; they are defined as α i (α i ∈[0,1]),P i,m (P i,m ∈[P i,min ,P i,max ]), If the access channel quality is good, MU further optimizes the offloading strategy by adjusting the offloading ratio α i With the transmission power P i,m .

4. The method according to claim 3, characterized in that Step 1.2, model and analyze the multi-dimensional resource management and task scheduling problems of heterogeneous network data offloading, discuss the mobile edge computing problems of heterogeneous tasks in heterogeneous networks, such as delay-sensitive applications and computation-intensive applications, and model and describe delay-sensitive applications and computation-intensive applications, including: Heterogeneous users need to offload data to APs based on task type, timeliness, and priority; all heterogeneous users are set to periodically generate and process tasks with different QoS requirements; the communication services involved in the tasks are divided into two categories: delay-sensitive application services and computation-intensive application services. The specific modeling process is as follows: Delay-sensitive application model: using a six-dimensional tuple model (d i (t),c i (t),θ i (t),τ i (t),ξ i (t),L i (t))) to characterize latency-sensitive application services. The tuple model consists of the following elements: data size d i (t), calculate the number of CPU cycles c required i (t), maximum tolerable delay θ i (t), maximum AoI limit τ i (t), task urgency index ξ i (t), and the task type identifier L i (t); where L i (t) = 1 represents delay sensitive type, L i (t) = 0 represents computation intensive; local computation delay of heterogeneous users processing the tth computation task As shown below: MU i Tolerable delay θ i The constraint on (t) is Therefore, based on local computing, the energy consumption in the tth computing task is The expression is: in For MU i The effective capacitance switch parameters of heterogeneous users adopt partial offloading mode for delay-sensitive applications; MU i Offloading tasks to AP m The SINR utility function is defined as: Where P i Indicates MU i The transmission power, Indicates intelligent response J k Interference power, W i,m For MU i With AP m The channel gain of the transmission link between For IRJJ k With AP m The channel gain of the interfering link between n is the transmission power of other MUs, W n,m For other users and APs in the same frequency band m The link gain of AP m The noise spectral density of According to the channel capacity theory, user MU i To base station AP m The unload transfer rate R i,m (t) is: where λ m Indicates AP m Therefore, for the tth computing task, MU i The total offload delay is expressed as: Transmission energy consumption of the tth computing task for: Introducing a normalized index of energy consumption for heterogeneous users when processing latency-sensitive application tasks and latency normalization index The normalized index is calculated as follows: in Computation-intensive application model: for heterogeneous user MU i For example, computationally intensive tasks are modeled as a four-dimensional tuple To model: where d i (t) is the data size, c i (t) is the number of CPU cycles required for the calculation, represents the minimum transfer rate that allows computationally intensive tasks to be offloaded; therefore, MU i The transfer rate R that allows the transfer of computationally intensive tasks i,m (t) is denoted as: For computationally intensive applications, mobile users adopt binary offloading mode; for the tth computationally intensive task, the normalized utility function is equal to MU i is closely related to the transmission throughput, which is defined as:

5. The method according to claim 4, characterized in that The reward weights in step 2 are as follows: The freshness of data information is quantified by AoI, which represents the time delay from data packet generation to task processing. The longer the delay, the lower the freshness of the task. Delay-sensitive applications adopt the critical task priority transmission processing mode, that is, before processing each delay-sensitive application task, the user will evaluate the status of its task queue and the freshness of the information to decide whether to prioritize transmission or discard the data in the queue. Computation-intensive application data is processed according to the order of arrival, using a first-come, first-served processing mode. New data packets enter the user's buffer according to the mean value λ of the Poisson distribution; data packets in the user's internal cache are distributed with , where ξ represents the urgency index and τ represents the maximum AoI limit. The urgency level of each data packet is ξ∈{1,2,…,v}, where ξ=0 means the task queue is empty; when MU i When the processed task exceeds the maximum AoI threshold, the corresponding information will be removed from the cache; when the task processed by the user exceeds the maximum AoI threshold, the task is discarded and the relevant information is removed from the emergency state vector; based on this, the weighted value of the relevant reward function is determined for the characteristics of delay-sensitive applications to encourage priority transmission behavior, where the reward weight value ι i Defined as: where t age Indicates the waiting time of a task before transmission.

6. The method according to claim 5, characterized in that The association matching relationship in step 3 is determined by the following method: Formulate the association matching satisfaction function: Where f(MU i ,AP m ) indicates MU i With AP m The association satisfaction between them ranges from 0 to 1; ζ is a constant ζ ≥ 7; when AP m Assigned to MU i The computing power of i,m Exceeds the task computing power requirement δ′ i,m When , the matching satisfaction function satisfies: The matching satisfaction function uses an S-shaped curve to represent the relationship between the computing power requirements of different tasks and the matching satisfaction; i,m Indicates the urgency of the unloading task and reflects the changing trend of the curve; the lower the i,m Corresponding to a flatter utility curve slope, it indicates that the task requires less computing power; on the contrary, ι i,m The higher the value of is, the steeper the curve is, indicating that the task has a greater demand for computing resources; the global optimization problem of association satisfaction is defined as: Among them, the association satisfaction of the whole network is Λ; Equation (24) aims to maximize the satisfaction of the whole network by optimizing the association access strategy of all MUs; MU and AP have autonomous decision-making capabilities: MU needs to select a suitable AP and calculate whether the computing resources allocated by its device association meet the demand; AP determines whether to accept the access request of the current MU based on the network satisfaction of the whole network; MUs collect external channel state information CSI and build their preferred AP association access priority ranking based on their own task processing capabilities and computing power requirements. The fast matching game algorithm is divided into two stages: In Phase I, users need to rebuild the priority list for APs based on the CSI information and location information perceived by the outside world: mobile users that do not meet the task offloading requirements will make matching requests to their preferred wireless access nodes, with the number of links not exceeding the number of MU associations allowed to be equipped; mobile users that have met the task offloading requirements will reduce the number of options for trying to access other APs; In Phase II, after receiving the matching request from the mobile user, the AP will exchange information with the relevant APs and determine whether to accept the user's matching request based on specific criteria; if the change in the matching result can improve the satisfaction of the entire network, the user's access request will be accepted.

7. The method according to claim 6, characterized in that In step 4, the user demand satisfaction function based on heterogeneous tasks is: where μ and x i,m Represents the weight value 0≤μ≤1 and the security transmission index x i,m ={-1,0,1}; MU sets the weight value μ according to the device performance and the current task’s preference for latency and energy consumption; introduces the secure transmission index x i,m To measure the safety of task offloading; x i,m =1 means that the safety of the unloading process can be guaranteed and the task calculation is successfully executed (f i,m ≠f J , or ); x i,m =0 means that the security of the uninstall process can be guaranteed, but the user task processing fails (f i,m ≠f J , or ); x i,m =-1 means that the unloading task is attacked by interference; In the optimization problem, the goal is to find an optimal data offloading decision {P i ,f i ,α i ,Y i }, so that the demand satisfaction RS i (t) maximization, that is, while ensuring that the task aims to minimize the computational cost of each user in terms of delay and energy consumption, while ensuring the security of task offloading in the presence of eavesdropping jammers; the in-network optimization problem is expressed as: Where f, α are the heterogeneous users connected to the base station / wireless access node The selected frequency, power, and unloading ratio resource allocation matrix; Y i (t) represents MU i The transmission task selected at time t; the constraints represent the range of multi-dimensional heterogeneous decisions and tasks selected by the MU.

8. The method according to claim 7, characterized in that Step 5: When multiple users in the heterogeneous network complete the adjustment, the crowd intelligence algorithm completes one iteration. When the multi-dimensional resources in all heterogeneous networks are reasonably allocated or the set number of iterations is reached, the process ends, including: Step 5.1: The mobile user observes the current state through channel perception and randomly selects the upper channel access decision based on probability ε and lower layer data offloading actions Step 5.2: Uninstall the data when the user After that, the base station will send the feedback reward back to the user according to the data unloading situation and input the generated data into the neural network for fitting, and the historical reward experience Store in the experience revisit pool and obtain the new state O i,m (t+1); Step 5.3, when i∈L high The Q function of the upper network is fitted When i∈L low The Q function of the underlying network is fitted And calculate the expected return based on the reward, and update the upper and lower Q-mix networks and their respective sub-network parameters with the minimum loss; Step 5.4 iterates in a loop until the maximum number of iterations is reached, and the optimal edge computing decision for users in the heterogeneous network is found to meet the computing and offloading requirements of each user's task.

9. The method according to claim 8, characterized in that Step 5.3 The specific process of neural network fitting is: The upper layer network is deployed on the AP and uses the Q-Mix network architecture to manage and allocate channel resources of all APs in the HetMEC network. Each AP is responsible for selecting the optimal channel access strategy for its associated MUs. The high-order Q-Mix hybrid network is deployed on the largest AP, namely the BS, and collects the Q-network parameters of all APs to maximize the channel resource benefits; in the high-level network, each MU sends a local observation value to its AP, and the AP selects a discrete channel access strategy for its MU based on the observation value. The Q network is trained by minimizing the iterative loss function, expressed as: The hybrid network of the lower layer Q-mix algorithm is deployed on the AP to maximize the user satisfaction of all MUs associated with it; For the lower layer network part, MU i The continuous parameter χ′ is chosen based on the new observations: in It is the channel access and device association strategy obtained from the high-level network; it converts the task information, location information, current channel information and Combine into a new state And input to the lower-level network; comprehensively consider the dynamic channel access decision and local observation information, use the lower-level network advantage function The mixed decision of each agent is used as input to generate MU i The lower network value of , the update strategy is: in Indicates MU i The lower layer advantage function; y low It is the one-step target value obtained by the lower hybrid network.

Images