A method for allocating transmission and computing resources in industrial wireless networks under imperfect channels

Abstract

The present invention proposes a method for allocating transmission and computing resources of industrial wireless networks under imperfect channels, the steps of which include: S1: establishing an end-to-end collaborative computing architecture for D2D-assisted learning in industrial wireless networks with imperfect channels; S2: considering the different real-time and computing resource requirements of heterogeneous tasks, combining multi-channel access and task offloading problems to obtain a distributed partially observable Markov decision process of a graph structure; S3: based on a multi-agent deep reinforcement learning framework that integrates Transformer and ACK feedback recovery mechanisms, a joint task offloading and multi-channel access algorithm that can be migrated to cope with imperfect channels is designed based on the end-to-end collaborative computing architecture to obtain a computing resource allocation method, and industrial equipment makes task offloading and multi-channel access decisions based on its own observations to minimize the total delay of task processing. The present invention comprehensively considers the different real-time requirements and computing resources of heterogeneous tasks, and has high system security and stability.

Description

Technical Field

[0001] The present invention relates to the technical field of transmission and computing resource allocation in industrial wireless networks, and in particular to a method for allocating transmission and computing resources in industrial wireless networks under imperfect channels. Background Art

[0002] With the development of intelligent manufacturing, industrial equipment such as automated guided vehicles (AGVs), robotic arms, and mobile robots are widely used in smart factories. These devices are interconnected via industrial wireless networks and generate a large number of heterogeneous tasks with varying deadlines and computational requirements. Meeting the real-time requirements of these heterogeneous tasks within the limited communication and computational resources of industrial wireless networks has become a challenge.

[0003] Using multi-access edge computing technology, industrial devices can offload heterogeneous tasks to edge servers for computation, reducing processing latency. Previous work has primarily focused on optimizing task offloading strategies to reduce the overall latency of heterogeneous task processing. However, in industrial wireless networks, massive heterogeneous tasks competing for limited channel resources during high-concurrency access can lead to access conflicts, which in turn impacts the real-time performance of task processing. Therefore, to meet the real-time requirements of heterogeneous tasks in smart factories, it is necessary to jointly optimize task offloading and multi-channel access strategies. However, the heterogeneous nature of the tasks generated by devices makes their offloading requirements time-varying, making them difficult to accurately model. Furthermore, since distributed devices lack access to global information, this exacerbates conflicts associated with multi-channel access. Multi-agent reinforcement learning algorithms can model devices as agents, alleviating non-stationary scenarios caused by limited local observations. They approximate optimal policies through interaction between devices and the environment, eliminating the need for precise modeling. Consequently, multi-agent reinforcement learning algorithms are increasingly being applied to address the joint task offloading and multi-channel access problems.

[0004] However, the current algorithmic models still have the following two problems that hinder their application in actual industrial scenarios. First, in actual industrial scenarios, the number of devices is time-varying. The current algorithm lacks generalization. Because the input dimension of the neural network is bound to the number of devices, once the number of devices changes, the algorithm model needs to change the input dimension and start training from scratch. Second, the training of multi-agent reinforcement learning algorithms requires the interaction between devices and the environment to generate feedback signals. Current algorithms often assume that devices can always receive correct feedback from their interactions with the environment. However, the existence of imperfect channels with strong noise interference in industrial wireless networks causes feedback loss, which in turn affects the normal training of devices.

[0005] In order to make up for the shortcomings of the above joint task offloading and multi-channel access algorithm and make the algorithm more suitable for actual industrial scenarios, it is necessary to design a joint task offloading and multi-channel access method with better generalization and coping with imperfect channels to allocate transmission and computing resources of industrial wireless networks. Summary of the Invention

[0006] In response to the technical problem of poor stability in transmission and computing resource allocation in industrial wireless networks, the present invention proposes an industrial wireless network transmission and computing resource allocation method under imperfect channels, which has high system security and stability.

[0007] In order to achieve the above object, the technical solution of the present invention is implemented as follows: a method for industrial wireless network transmission and computing resource allocation under imperfect channels, characterized in that the steps include:

[0008] S1: Establishing a device-edge collaborative computing architecture for D2D-assisted learning in industrial wireless networks with imperfect channels;

[0009] S2: Considering the different real-time and computing resource requirements of heterogeneous tasks, we combine multi-channel access and task offloading to obtain a distributed partially observable Markov decision process with a graph structure.

[0010] S3: Based on a multi-agent deep reinforcement learning framework that integrates Transformer and ACK feedback recovery mechanisms, a portable joint task offloading and multi-channel access algorithm is designed based on the edge-to-edge collaborative computing architecture to cope with imperfect channels. This method also provides a computing resource allocation method. Industrial equipment makes task offloading and multi-channel access decisions based on its own observations, minimizing the total task processing latency.

[0011] The edge-to-edge collaborative computing architecture described in step S1 includes an industrial base station, an edge server, and M industrial devices. The edge server corresponds to the industrial base station. The industrial devices are connected to the edge server via a wireless network. The industrial devices communicate with each other via D2D and generate heterogeneous tasks. The heterogeneous tasks generated by industrial device m at time slot t are d m,n , c m,n , Represent the task size, required computational delay, and maximum tolerable delay, respectively. Represents task n m (t) collection.

[0012] The method for obtaining the distributed partially observable Markov decision process of the graph structure in step S2 is:

[0013] The observable Markov decision process includes graph structure observations, actions, and reward functions. Each industrial device is set as an entity with z features. The observation of industrial device m includes its own features and m-1 features of other entities. The feature information of other entities is obtained through D2D communication.

[0014] At time slot t, the entity i, i∈M observed by industrial equipment m has the following characteristics:

[0015] ent m,i (t)=[ack i (t),d i,n (t),c i,n (t),h i,n (t)];

[0016] Among them, ack i (t) represents the base station confirmation feedback received by entity i, ack i (t)={null,-1,0,1},ack i (t) = null means ack feedback is lost, ack i (t) = -1 indicates that the task transmission fails, ack i (t)=0 means the task is offloaded locally, ack i (t) = 1 means the task is successfully completed, d i,n (t), c i,n (t), h i,n (t) represents the task n generated by entity i m (t) is the task size, required computing resources and remaining lifespan, n represents task n m (t) index;

[0017] At time slot t, the graph observation space o of industrial equipment m m (t) is an m*z matrix, expressed as:

[0018] o m (t)=[ent m,1 (t),ent m,2 (t),...,ent m,i (t)];

[0019] The overall observation space of the system can be expressed as:

[0020] o(t)={o1(t),o2(t),...,o M (t)};

[0021] Among them, ent m,i (t) is the physical characteristic observed by the industrial equipment, o M(t) is the graph structure observation, and the industrial equipment m observes the channel selection action a at time slot t m (t) is defined as follows:

[0022] a m (t) = {0, 1, ..., K};

[0023] when a m When (t) = 0, it means that industrial equipment m chooses to offload the task locally. m When (t) = k, it means that industrial equipment m selects channel k for task offloading; k∈K;

[0024] The reward function is designed by comprehensively considering task processing delay optimization and task overdue penalty.

[0025] The method for designing a reward function by comprehensively considering task processing delay optimization and task overdue penalty is:

[0026] In order to minimize the processing delay of tasks, industrial equipment needs to offload the tasks to the edge server for calculation. Therefore, the delay reward d of industrial equipment m at time slot t is m (t) is expressed as:

[0027]

[0028] in, For task n m (t) local processing delay at industrial device m, Represents task n m (t) the processing delay offloaded by industrial device m to the edge server;

[0029] In order to reduce the overdue rate of tasks, the overdue penalty p of industrial equipment m at time slot t is m (t) is expressed as

[0030]

[0031] in, For task n m (t) queuing delay in the buffer of industrial equipment m;

[0032] In summary, at time slot t, the compound reward function r of industrial equipment m is m (t) can be expressed as:

[0033] r m (t) = d m (t)-α·p m (t);

[0034] Among them, α is the weight parameter;

[0035] When the feedback of industrial equipment m is lost, the compound reward function r m (t) is set to null.

[0036] It is characterized in that the method of designing a joint task offloading and multi-channel access algorithm that can be migrated in the case of imperfect channels in step S3 is:

[0037] S31: Build an Agent network deployed on industrial equipment;

[0038] S32: Build a Mixing network deployed on edge servers;

[0039] S33: Design a feedback recovery mechanism to cope with ACK feedback loss caused by imperfect channels;

[0040] S34: A joint task offloading and multi-channel access algorithm that can be transferred to cope with imperfect channels is obtained. During training, each industrial device observes the output action according to the graph structure and obtains rewards. The industrial wireless network judges the value of the output action based on the rewards and performs network updates.

[0041] The method for building an Agent network deployed on industrial equipment in step S31 is:

[0042] The core architecture of the Agent network consists of the Embedding layer, Transformer module, and output layer connected in sequence;

[0043] The Embedder layer first creates a matrix Emb with a shape of z*h m Observe the graph structure o m (t) is encoded, h represents the embedding dimension of the Embedder layer, and z represents the matrix Emb m dimension, the final encoding matrix E(t) can be obtained by the formula E(t)=o m (t)Emb m Calculated; m The same features in (t) are in the matrix Emb m share the same weights, and the matrix Emb m It has nothing to do with the number of industrial equipment;

[0044] The input of the Transformer module includes the matrix E(t) and h m (t-1), where h m (t-1) is the hidden state containing the action and observation history information of the industrial equipment m. The Transformer module processes the matrix E(t) and h in parallel through the multi-head self-attention mechanism. m (t-1), learn the edge relationships between entities in the input matrix and obtain the improved graph representation matrix Em (t) and h m (t), where the matrix E m The vertex vector of (t) fuses the adjacent related vertex information; the output layer passes h m (t) and E m (t) Information generates Q value Q m (τ m ,a m (t)), used to evaluate the quality of the current actions taken by industrial equipment, τ m Action-observation history information for industrial equipment.

[0045] The method for building a Mixing network deployed on the edge server in step S32 is as follows:

[0046] The core architecture of the Mixing network consists of a sequentially connected Embedder layer, a Transformer module, and an output layer. The output layer consists of two deep neural networks: a parameter generation network and an inference network.

[0047] The input of the Embedder layer of the Mixing network is the observation s(t) of all industrial equipment. The encoded observation s(t) is sent to the Transformer module for feature learning, and finally combined with the parameter generation network to generate neuron weights w1 and w2; the inference network generates a global Q value through the neuron weights w1 and w2 and the Q value of all industrial equipment. tot (τ,a(t)), the optimization goal of the Agent network is to maximize its own Q value, and the optimization goal of the Mixing network is to minimize Q tot The mean square error between (τ,a(t)) and the target Q value, the Mixing network is constrained Ensure the consistency of the two optimization goals and learn the optimal network resource allocation strategy.

[0048] Step S33: A feedback recovery mechanism is designed to cope with ACK feedback loss caused by an imperfect channel, including the following steps:

[0049] S331: Model the channel in the edge-device collaborative computing architecture as an imperfect channel with data loss probability. Collect information about the interaction between industrial devices and the industrial wireless network, including observations, industrial device actions, and rewards. This includes information about feedback loss and reward loss to construct a training dataset.

[0050] S332: Construct the interactions into a pandas data frame and use the deep learning-based DataWig model to predict missing data;

[0051] S333: ACK signals and industrial equipment actions in the edge server are classified and encoded using one-hot vectors, and feature extraction is performed using the Embedder layer. The remaining data is vector-feature concatenated using a normalized neural network.

[0052] S334: Use the classification cross entropy loss function and the regression mean square error loss function to jointly train the classification target and the continuous numerical target respectively;

[0053] S335: Use the trained neural network model to output predicted values ​​to replace the missing feedback in the original data, thereby restoring the complete empirical data to cope with the challenges of imperfect channels to model training.

[0054] The classification targets in step S334 include ack signals and industrial equipment actions, and the continuous numerical targets include the remaining observation values ​​and reward values.

[0055] The steps of the joint task offloading and multi-channel access algorithm for migrating in the case of imperfect channels described in step S34 are as follows:

[0056] S341: Initialize the parameters of the algorithm's neural network model and Transformer module;

[0057] S342: Constructing a graph structure observation based on the D2D communication range between industrial devices. The features of other industrial devices within the observation range of industrial device m are added to the observation of industrial device m, and the features of industrial devices outside the observation range are filled with 0.

[0058] S343: Industrial devices interact with the industrial wireless network environment based on graph structure observation output actions and receive rewards. If the feedback signal of the environmental interaction is lost due to imperfect channel interference, it is recovered using the feedback recovery mechanism.

[0059] S344: Collect the complete observations, states, actions, and rewards of industrial equipment as experience trajectories and update the algorithm's neural network;

[0060] S345: Repeat S341-S344 until the number of simulation rounds is reached to complete the training. After the training is completed, each industrial device can perform distributed adaptive task offloading and channel access decisions to achieve a reasonable allocation of transmission and computing resources in the industrial network;

[0061] The steps of the feedback recovery mechanism in step S344 are:

[0062] S3441: Model the channel as an imperfect channel device with an ε data loss probability. Collect information about the interaction between industrial devices and the industrial wireless network, including observations, device actions, and rewards. This includes information about feedback loss and reward loss to construct a training dataset.

[0063] S3442: Construct the interactions into a pandas data frame and use the deep learning-based DataWig model to predict missing data;

[0064] S3443: ACK signals and device actions in the edge server are classified and encoded using one-hot vectors, and feature extraction is performed using the Embedder layer. The remaining data is vector-feature concatenated using a normalized neural network.

[0065] S3444: Use the classification cross entropy loss function and the regression mean square error loss function to jointly train the ack signal and device action and other observations and reward values ​​respectively;

[0066] S3445: Use the predicted values ​​output by the trained neural network model to replace the missing feedback in the original data to restore the complete empirical data to address the challenges of imperfect channels to model training.

[0067] The beneficial effects of the present invention are:

[0068] 1. This invention comprehensively considers the different real-time requirements and computing resources of heterogeneous tasks, and designs a joint task offloading and multi-channel access algorithm based on a data-driven multi-agent reinforcement learning algorithm. Without the need for precise system modeling, industrial equipment can adaptively access based on interaction with the environment, with high system security and stability.

[0069] 2. The present invention has strong generalization and practicality. The algorithm model can adapt to the industrial wireless network environment under time-varying topology scenarios, which makes up for the shortcomings of previous algorithms that need to be retrained after slight changes in topology.

[0070] 3. The present invention has strong robustness. The designed feedback recovery mechanism enables the algorithm to adapt to industrial scenarios with harsh communication environments, thereby improving the performance and reliability of the algorithm in actual industrial applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0072] Figure 1 An embodiment of the present invention provides an edge-to-end collaborative computing architecture for D2D-assisted learning.

[0073] Figure 2A schematic diagram of a joint task offloading and multi-channel access algorithm model based on multi-agent deep reinforcement learning provided by an embodiment of the present invention.

[0074] Figure 3 A flowchart of a method for allocating industrial wireless network transmission and computing resources under an imperfect channel provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0075] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0076] like Figure 3 As shown, the present invention proposes a distributed multi-channel access method for multi-source heterogeneous cooperative transmission based on deep reinforcement learning, comprising the following steps:

[0077] Step 1: Establish an edge-to-edge collaborative computing architecture for D2D-assisted learning in industrial wireless networks with imperfect channels.

[0078] like Figure 1 As shown in the figure, in order to alleviate the environmental non-stationarity caused by the difficulty of distributed devices in obtaining global information, an edge-to-end collaborative computing architecture suitable for D2D-assisted industrial wireless networks is first established.

[0079] The edge server is connected to the industrial base station through a wired network to meet the computing needs of heterogeneous tasks. The equipment can offload the generated tasks to the edge server through the industrial base station for computing. Industrial equipment includes distributed intelligent devices such as automatic guided vehicles (AGVs) and robotic arms. They are connected to the industrial base station through a wireless network and generate heterogeneous tasks. The device m, m∈M, generates heterogeneous tasks at time slot t using d m,n , c m,n , Represent the task size, required computation and maximum tolerable delay respectively, Represents task n m (t) collection.

[0080] Industrial equipment must make offloading decisions based on task attributes. When offloading heterogeneous tasks to edge servers, they must compete for channels for transmission, leading to issues with task offloading and multi-channel access. Due to the harsh factory communication environment and the presence of noise interference in the channels, task transmission fails during uplink transmission when multiple devices select the same channel for transmission or when the channel is subject to noise interference. Failed tasks must wait in the device's buffer for retransmission in the next time slot using a first-in-first-out (FIFO) mode. During downlink transmission, the base station's ACK feedback signal can be lost due to interference, impacting the algorithm's training process.

[0081] Step 2: Considering the different real-time and computing resource requirements of heterogeneous tasks, the joint multi-channel access and task offloading problem is formulated as a distributed partially observable Markov decision process with a graph structure.

[0082] Since distributed devices are in a partially observable state, each device can only observe its own access information and cannot obtain global information. To solve this problem, we first model the task offloading and multi-channel access problem of distributed devices into a distributed partially observable Markov decision model according to the following steps:

[0083] First, each industrial device is set as an entity with z features. The observation of industrial device m includes its own features and m-1 features of other entities. The feature information of other entities is obtained through D2D communication. The state space of the graph structure helps the designed algorithm better extract device feature information.

[0084] At time slot t, the entity i, i∈M observed by device m, is characterized as follows:

[0085] ent m,i (t)=[ack i (t),d i,n (t),c i,n (t),h i,n (t)],

[0086] Among them ack i (t) = {null, -1, 0, 1} indicates the base station confirmation feedback received by entity i, ack i (t) = null means ack feedback is lost, ack i (t) = -1 indicates that the task transmission fails, ack i (t)=0 means the task is offloaded locally, ack i (t) = 1 means the task is successfully completed, d i,n (t), c i,n (t), h i,n (t) represents the task n generated by entity i m(t) task size, required computing resources and remaining lifetime, etc.

[0087] At time slot t, the graph observation space o of device m m (t) is an m*z matrix, which can be expressed as o m (t)=[ent m,1 (t),ent m,2 (t),...,ent m,i (t)], ent m,i (t) is the physical feature observed by the industrial equipment, and the overall observation space of the system can be expressed as o(t) = {o1(t), o2(t), ..., o M (t)}.

[0088] Industrial device m observes the channel selection action a at time slot t m (t) is defined as follows:

[0089] a m (t) = {0, 1, ..., K}.

[0090] when a m When (t) = 0, it means that industrial equipment m chooses to offload the task locally. m When (t) = k, it means that industrial equipment m selects channel k (k∈K) for task offloading;

[0091] The reward function is designed by comprehensively considering task processing delay optimization and task overdue penalty, including the following steps:

[0092] In order to minimize the processing delay of the task, the device needs to offload the task to the mobile edge server for calculation. Therefore, the delay reward d of device m at time slot t is m (t) is expressed as

[0093]

[0094] in For task n m (t) local processing delay at device m, Task n m (t) the processing delay offloaded by device m to the mobile edge server;

[0095] In order to reduce the overdue rate of the task, the overdue penalty p of device m at time slot t is m (t) is expressed as:

[0096]

[0097] in For task n m (t) queuing delay in the buffer of device m;

[0098] In summary, at time slot t, the compound reward function r of device m is m (t) can be expressed as

[0099] r m (t) = d m (t)-α·p m (t), α is the weight parameter;

[0100] When the feedback from device m is lost, r m (t) is set to null.

[0101] Step 3: Based on a multi-agent deep reinforcement learning framework that integrates Transformer and ACK feedback recovery mechanisms, a joint task offloading and multi-channel access algorithm that can be transferred to cope with imperfect channels is designed. Industrial equipment makes task offloading and multi-channel access decisions based on its own observations to minimize the total delay of task processing. The overall network architecture of the algorithm is as follows: Figure 2 shown.

[0102] First, build an Agent network deployed on the device. The core architecture of the Agent network consists of an Embedding layer, a Transformer module, and an output layer.

[0103] The Embedder layer first creates a matrix Emb with a shape of z*h m Observe the graph structure o m (t) is encoded, h represents the embedding dimension of the Embedder layer, and the final encoding matrix E(t) can be obtained by the formula E(t) = o m (t)Emb m Calculated. m Similar features in (t) (such as task size of different entities, etc.) are m share the same weight, and Emb m It is independent of the number of devices, so the algorithm can adapt to industrial scenarios with different numbers of devices and achieve portability.

[0104] The input of the Transformer module includes E(t) and h m (t-1), where h m (t-1) is the hidden state containing the action and observation history information of device m. The Transformer module processes the matrix E(t) in parallel through the multi-head self-attention mechanism, learns the edge relationship between entities in the input matrix, and obtains the improved graph representation matrix E m (t), and h m (t), where the matrix E mThe vertex vector of (t) fuses the information of adjacent related vertices. The output layer passes h m (t) and E m (t) Information generates Q value Q m (τ m ,a m (t)), used to evaluate the quality of the action currently taken by the device, τ m Action-observation history information for device m.

[0105] Secondly, we built a Mixing network deployed on edge servers. The core architecture of the Mixing network consists of an Embedder layer, a Transformer module, and an output layer. The output layer consists of two deep neural networks: a parameter generation network and an inference network.

[0106] The input of the Embedder layer of the Mixing network is the observation of all devices, defined as s(t). The encoded s(t) is sent to the Transformer module for feature learning, and finally combined with the parameter generation network to generate neuron weights w1 and w2. The final inference network generates a global Q value through the neuron weights w1 and w2 and the Q values ​​of all devices. tot (τ,a(t)), the optimization goal of the Agent network is to maximize its own Q value, and the optimization goal of the Mixing network is to minimize Q tot The mean square error between (τ,a(t)) and the target Q value, the Mixing network is constrained Ensure the consistency of the two optimization goals and learn the optimal network resource allocation strategy.

[0107] Finally, a feedback recovery mechanism is designed to deal with the ACK feedback loss caused by imperfect channels, and to avoid the loss of neural network training data through the recovery mechanism. The feedback recovery mechanism design includes the following steps:

[0108] (1) The channel is modeled as an imperfect channel device with a data loss probability of ε. Information about the interaction between the device and the industrial wireless network is collected, including observations, device actions, and rewards. The training dataset is constructed by including information about feedback loss and reward loss.

[0109] (2) The interactions are constructed into a data frame in pandas format and missing data prediction is performed using the deep learning-based DataWig model;

[0110] (3) The ack signal and device action are classified and encoded using one-hot vectors, and feature extraction is performed using the Embedder layer. The remaining data is vector-feature concatenated using a normalized neural network.

[0111] (4) Use the classification cross entropy loss function and the regression mean square error loss function to jointly train the classification targets (such as ack signals and device actions) and continuous numerical targets (such as other observations, reward values, etc.);

[0112] (5) The predicted values ​​output by the trained neural network model are used to replace the missing feedback in the original data, thereby restoring the complete empirical data to cope with the challenges of imperfect channels to model training.

[0113] like Figure 3 As shown, a joint task offloading and multi-channel access algorithm mainly includes the following steps:

[0114] 1) Initialize the parameters of the algorithm's neural network model and Transformer module;

[0115] 2) Construct a graph structure observation based on the D2D communication range between devices. The features of other devices within the observation range of device m are added to the observation of the device, and the features of devices outside the observation range are filled with 0;

[0116] 3) The device observes the output action based on the graph structure to interact with the industrial wireless network environment and obtain rewards. If the feedback signal of the environmental interaction is lost due to the interference of the imperfect channel, it is restored using the feedback recovery mechanism;

[0117] 4) Collect the complete observations, states, actions, and rewards of industrial equipment as experience trajectories and update the algorithm’s neural network;

[0118] 5) Repeat steps 1)-4) until the number of simulation rounds is reached to complete the training. After training is completed, each industrial device can perform distributed adaptive task offloading and channel access decisions to achieve a reasonable allocation of transmission and computing resources in the industrial network. At the same time, the algorithm can adapt to the time-varying number of devices and can cope with feedback loss caused by imperfect channels.

[0119] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for allocating industrial wireless network transmission and computing resources under imperfect channels, characterized in that the steps include: S1: Establishing a device-edge collaborative computing architecture for D2D-assisted learning in industrial wireless networks with imperfect channels; S2: Considering the different real-time and computing resource requirements of heterogeneous tasks, we combine multi-channel access and task offloading to obtain a distributed partially observable Markov decision process with a graph structure. S3: Based on a multi-agent deep reinforcement learning framework that integrates Transformer and ACK feedback recovery mechanisms, a portable joint task offloading and multi-channel access algorithm is designed based on the edge-to-edge collaborative computing architecture to cope with imperfect channels. This method also provides a computing resource allocation method. Industrial equipment makes task offloading and multi-channel access decisions based on its own observations, minimizing the total task processing latency.

2. The industrial wireless network transmission and computing resource allocation method under imperfect channels according to claim 1 is characterized in that: The edge-to-edge collaborative computing architecture described in step S1 includes an industrial base station, an edge server, and M industrial devices. The edge server corresponds to the industrial base station. The industrial devices are connected to the edge server via a wireless network. The industrial devices communicate with each other via D2D and generate heterogeneous tasks. The heterogeneous tasks generated by industrial device m at time slot t are d m,n , c m,n , Represent the task size, required computational delay, and maximum tolerable delay, respectively. Represents task n m (t) collection.

3. The method for allocating industrial wireless network transmission and computing resources under imperfect channels according to claim 2, characterized in that: The method for obtaining the distributed partially observable Markov decision process of the graph structure in step S2 is: The observable Markov decision process includes graph structure observations, actions, and reward functions. Each industrial device is set as an entity with z features. The observation of industrial device m includes its own features and m-1 features of other entities. The feature information of other entities is obtained through D2D communication. At time slot t, the entity i, i∈M observed by industrial equipment m has the following characteristics: ent m,i (t)=[ack i (t),d i,n (t),c i,n (t),h i,n (t)]; Among them, ack i (t) represents the base station confirmation feedback received by entity i, ack i (t)={null,-1,0,1},ack i (t) = null means ack feedback is lost, ack i (t) = -1 indicates that the task transmission fails, ack i (t)=0 means the task is offloaded locally, ack i (t) = 1 means the task is successfully completed, d i,n (t), c i,n (t), h i,n (t) represents the task n generated by entity i m (t) is the task size, required computing resources and remaining lifespan, n represents task n m (t) index; At time slot t, the graph observation space o of industrial equipment m m (t) is an m*z matrix, expressed as: o m (t)=[ent m,1 (t),ent m,2 (t),...,ent m,i (t)]; The overall observation space of the system can be expressed as: o(t)={o1(t),o2(t),...,o M (t)}; Among them, ent m,i (t) is the physical feature observed by the industrial equipment, o M (t) is the graph structure observation, and the industrial equipment m observes the channel selection action a at time slot t m (t) is defined as follows: a m (t)={0,1,...,K}; when a m When (t) = 0, it means that industrial equipment m chooses to offload the task locally. m When (t) = k, it means that industrial equipment m selects channel k for task offloading; k∈K; The reward function is designed by comprehensively considering task processing delay optimization and task overdue penalty.

4. The method for allocating industrial wireless network transmission and computing resources under imperfect channels according to claim 3 is characterized in that: The method for designing a reward function by comprehensively considering task processing delay optimization and task overdue penalty is: In order to minimize the processing delay of tasks, industrial equipment needs to offload the tasks to the edge server for calculation. Therefore, the delay reward d of industrial equipment m at time slot t is m (t) is expressed as: in, For task n m (t) local processing delay at industrial device m, Represents task n m (t) the processing delay offloaded by industrial device m to the edge server; In order to reduce the overdue rate of tasks, the overdue penalty p of industrial equipment m at time slot t is m (t) is expressed as in, For task n m (t) queuing delay in the buffer of industrial equipment m; In summary, at time slot t, the compound reward function r of industrial equipment m is m (t) can be expressed as: r m (t)=d m (t)-α·p m (t); Among them, α is the weight parameter; When the feedback of industrial equipment m is lost, the compound reward function r m (t) is set to null.

5. The method for allocating industrial wireless network transmission and computing resources under an imperfect channel according to any one of claims 1 to 4, characterized in that: The method for designing a joint task offloading and multi-channel access algorithm that can be migrated to cope with imperfect channels in step S3 is: S31: Build an Agent network deployed on industrial equipment; S32: Build a Mixing network deployed on edge servers; S33: Design a feedback recovery mechanism to cope with ACK feedback loss caused by imperfect channels; S34: A joint task offloading and multi-channel access algorithm that can be transferred to cope with imperfect channels is obtained. During training, each industrial device observes the output action according to the graph structure and obtains rewards. The industrial wireless network judges the value of the output action based on the rewards and performs network updates.

6. The method for allocating industrial wireless network transmission and computing resources under imperfect channels according to claim 5, characterized in that: The method for building an Agent network deployed on industrial equipment in step S31 is: The core architecture of the Agent network consists of the Embedding layer, Transformer module, and output layer connected in sequence; The Embedder layer first creates a matrix Emb with a shape of z*h m Observe the graph structure o m (t) is encoded, h represents the embedding dimension of the Embedder layer, and z represents the matrix Emb m dimension, the final encoding matrix E(t) can be obtained by the formula E(t)=o m (t)Emb m Calculated; m The same features in (t) are in the matrix Emb m share the same weights, and the matrix Emb m It has nothing to do with the number of industrial equipment; The input of the Transformer module includes the matrix E(t) and h m (t-1), where h m (t-1) is the hidden state containing the action and observation history information of the industrial equipment m. The Transformer module processes the matrix E(t) and h in parallel through the multi-head self-attention mechanism. m (t-1), learn the edge relationships between entities in the input matrix and obtain the improved graph representation matrix E m (t) and h m (t), where the matrix E m The vertex vector of (t) fuses the adjacent related vertex information; the output layer passes h m (t) and E m (t) Information generates Q value Q m (τ m ,a m (t)), used to evaluate the quality of the current actions taken by industrial equipment, τ m Action-observation history information for industrial equipment.

7. The method for allocating industrial wireless network transmission and computing resources under imperfect channels according to claim 6, characterized in that: The method for building a Mixing network deployed on the edge server in step S32 is as follows: The core architecture of the Mixing network consists of a sequentially connected Embedder layer, a Transformer module, and an output layer. The output layer consists of two deep neural networks: a parameter generation network and an inference network. The input of the Embedder layer of the Mixing network is the observation s(t) of all industrial equipment. The encoded observation s(t) is sent to the Transformer module for feature learning, and finally combined with the parameter generation network to generate neuron weights w1 and w2; the inference network generates a global Q value through the neuron weights w1 and w2 and the Q value of all industrial equipment. tot (τ,a(t)), the optimization goal of the Agent network is to maximize its own Q value, and the optimization goal of the Mixing network is to minimize Q tot The mean square error between (τ,a(t)) and the target Q value, the Mixing network is constrained Ensure the consistency of the two optimization goals and learn the optimal network resource allocation strategy.

8. The industrial wireless network transmission and computing resource allocation method under imperfect channels according to claim 6 or 7, characterized in that: Step S33: A feedback recovery mechanism is designed to cope with ACK feedback loss caused by an imperfect channel, including the following steps: S331: Model the channel in the edge-device collaborative computing architecture as an imperfect channel with data loss probability. Collect information about the interaction between industrial devices and the industrial wireless network, including observations, industrial device actions, and rewards. This includes information about feedback loss and reward loss to construct a training dataset. S332: Construct the interactions into a pandas data frame and use the deep learning-based DataWig model to predict missing data; S333: ACK signals and industrial equipment actions in the edge server are classified and encoded using one-hot vectors, and feature extraction is performed using the Embedder layer. The remaining data is vector-feature concatenated using a normalized neural network. S334: Use the classification cross entropy loss function and the regression mean square error loss function to jointly train the classification target and the continuous numerical target respectively; S335: Use the trained neural network model to output predicted values ​​to replace the missing feedback in the original data, thereby restoring the complete empirical data to cope with the challenges of imperfect channels to model training.

9. The method for allocating industrial wireless network transmission and computing resources under imperfect channels according to claim 8, characterized in that: The classification targets in step S334 include ack signals and industrial equipment actions, and the continuous numerical targets include the remaining observation values ​​and reward values.

10. The method for allocating industrial wireless network transmission and computing resources under imperfect channels according to claim 9, characterized in that: The steps of the joint task offloading and multi-channel access algorithm for migrating in the case of imperfect channels described in step S34 are as follows: S341: Initialize the parameters of the algorithm's neural network model and Transformer module; S342: Constructing a graph structure observation based on the D2D communication range between industrial devices. The features of other industrial devices within the observation range of industrial device m are added to the observation of industrial device m, and the features of industrial devices outside the observation range are filled with 0. S343: Industrial devices interact with the industrial wireless network environment based on graph structure observation output actions and receive rewards. If the feedback signal of the environmental interaction is lost due to imperfect channel interference, it is recovered using the feedback recovery mechanism. S344: Collect the complete observations, states, actions, and rewards of industrial equipment as experience trajectories and update the algorithm's neural network; S345: Repeat S341-S344 until the number of simulation rounds is reached to complete the training. After the training is completed, each industrial device can perform distributed adaptive task offloading and channel access decisions to achieve a reasonable allocation of transmission and computing resources in the industrial network; The steps of the feedback recovery mechanism in step S344 are: S3441: Model the channel as an imperfect channel device with an ε data loss probability. Collect information about the interaction between industrial devices and the industrial wireless network, including observations, device actions, and rewards. This includes information about feedback loss and reward loss to construct a training dataset. S3442: Construct the interactions into a pandas data frame and use the deep learning-based DataWig model to predict missing data; S3443: ACK signals and device actions in the edge server are classified and encoded using one-hot vectors, and feature extraction is performed using the Embedder layer. The remaining data is vector-feature concatenated using a normalized neural network. S3444: Use the classification cross entropy loss function and the regression mean square error loss function to jointly train the ack signal and device action and other observations and reward values ​​respectively; S3445: Use the predicted values ​​output by the trained neural network model to replace the missing feedback in the original data to restore the complete empirical data to address the challenges of imperfect channels to model training.

Images