A product surface defect detection reasoning task acceleration method based on reinforcement learning

Abstract

The application discloses a product surface defect detection reasoning task acceleration method based on reinforcement learning, S1: obtaining a product to-be-reasoned task; S2: constructing an end-edge-cloud collaborative system model, including a deep neural network deployment model, a task unloading model, a wireless channel allocation model and a computing resource allocation model; S3: constructing a target function model; S4: converting the target function model by using Lyapunov optimization theory; S5: training the converted target function model by using an improved deep reinforcement learning model, obtaining an optimized target function model, and realizing acceleration processing of the product to-be-reasoned task. The improved deep reinforcement learning model is embedded with an edge computing resource allocation optimization model and a dynamic programming deep neural network deployment optimization model, and based on the optimized target function model obtained after training, the total service cost is minimized under the condition of guaranteeing the long-term reasoning accuracy of the task.

Description

Technical Field

[0001] This invention relates to the field of product surface defect detection technology, and in particular to a method for accelerating product surface defect detection inference tasks based on reinforcement learning. Background Technology

[0002] In recent years, the rapid development of the Internet of Things (IoT) and deep neural network technologies has continuously driven the application of machine learning (ML) in Industrial Internet of Things (IIoT) scenarios, such as product classification, product defect (e.g., refractory brick defect) detection, facility monitoring, and fault diagnosis. In IIoT applications for product defect detection, sensors capture images of each product on an automated production line. Then, deep neural networks are used for task inference to detect potential defects in the images (e.g., refractory brick images) and classify the products into the appropriate categories for further processing.

[0003] In an Industrial Internet of Things (IIoT) environment, end-only solutions, due to resource scarcity, can only support localized deployments of lightweight deep neural networks for sensor models, which are primarily optimized through model compression techniques. Cloud-only solutions deploy full-size deep neural networks on cloud server models, providing inference services for all sensor model tasks. However, due to the significant distance between the sensor models and the cloud server models, performance is affected by task transmission latency. End-Edge solutions, by configuring edge servers on base stations, enable edge computing capabilities, supporting full-size deep neural network deployments and providing near-field task inference services. However, due to limited storage resources on edge servers, only a small number of full-size deep neural network deployments can be supported, and the service latency of edge servers deteriorates as task inference requests increase. Therefore, none of these three solutions can independently provide high-precision, low-latency task inference for all tasks. Summary of the Invention

[0004] This invention provides a method for accelerating product surface defect detection inference tasks based on reinforcement learning, overcoming the technical problem that existing solutions cannot independently provide high-precision, low-latency task inference for all inference tasks.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] A method for accelerating product surface defect detection inference tasks based on reinforcement learning, comprising the following steps:

[0007] S1: Obtain the product reasoning task based on the sensor model;

[0008] S2: Construct a three-party collaborative model, namely the end-edge-cloud collaborative system model, with several sensor models, edge server models and cloud server models as participants. The end-edge-cloud collaborative system model includes a deep neural network deployment model, a task offloading model, a wireless channel allocation model and a computing resource allocation model.

[0009] S3: Construct an objective function model that minimizes the total service cost based on deep neural network deployment model, task offloading model, wireless channel allocation model, and computing resource allocation model;

[0010] S4: Using Lyapunov optimization theory, a precision model is introduced, a precision deficit queue is established based on the precision model, and the upper boundary of Lyapunov's drift is calculated, thereby transforming the objective function model to obtain the transformed objective function model;

[0011] S5: The transformed objective function model is trained using an improved deep reinforcement learning model to obtain an optimized objective function model, thereby accelerating the processing of the product's reasoning task.

[0012] Specifically, in S2, the end-edge-cloud collaborative system model operates in a time-slot manner. This model includes a deep neural network deployment model, a task offloading model, a wireless channel allocation model, and a computing resource model, wherein:

[0013] The deep neural network deployment model is as follows:

[0014]

[0015] Where X(t) represents the set of all deep neural network deployment metrics within time slot t. Let N be the number of deep neural network inference tasks, where n represents the nth task class. x n If x(t) = 1, it means that the nth type of full-size deep neural network is deployed on an edge server. If the nth type of full-size deep neural network is not deployed on an edge server, then x n (t) = 0;

[0016] The task unloading model is as follows:

[0017]

[0018]

[0019] Where α(t) represents the set of edge server task offloading indicators within time slot t. Let m represent the total number of sensors, and m represent the m-th sensor. mn represents the nth type of task of sensor m; α mn (t) = 1, indicating that task mn within time slot t is offloaded to the edge server model; otherwise, α mn (t) = 0; β(t) represents the set of cloud server task offloading instructions within time slot t, β mn (t) = 1, indicating that task mn is unloaded onto the cloud server model; otherwise, β mn (t)=0;1-α mn (t)-β mn (t) = 1, indicating that task mn is executed in the sensor model;

[0020] The wireless channel allocation model is as follows:

[0021]

[0022] Where, γ m (t) represents the set of channel allocation indicators. Let k represent the set of wireless channels, and k represent the k-th channel. γ mk If γ(t) = 1, it means that sensor m is assigned to the k-th channel; otherwise, γ mk (t) = 0;

[0023] The computing resource allocation model is as follows:

[0024]

[0025] Among them, F e (t) represents the set of computational resource allocation metrics for all full-size deep neural networks deployed on the edge server model. This represents the computational resources allocated to the nth class of full-size deep neural networks deployed on an edge server model in time slot t, and f e This represents the total computing resources of the edge server model.

[0026] Specifically, in S3, an objective function model that minimizes the total service cost is constructed based on a deep neural network deployment model, a task offloading model, a wireless channel allocation model, and a computing resource model. The objective function model is as follows:

[0027]

[0028] Set the constraints as follows:

[0029]

[0030]

[0031]

[0032]

[0033]

[0034]

[0035] C7: α mn (t)+β mn (t)≤1

[0036]

[0037]

[0038]

[0039]

[0040]

[0041] Where C1 represents the resource range occupied by the deep neural network deployed on the edge server model; C2 represents the accuracy requirement range met by the actual task; C3 represents the value range of the deep neural network deployment indicator; and C4 represents the value range when α... mn When (t) = 1, x n (t) is 1; C5 and C6 are the value ranges of the task offloading indication for the edge server model and cloud server model, respectively, i.e., whether the task is offloaded to the edge server model and cloud server model; C7 indicates that the task can be independently offloaded to the edge server model or cloud server model; C8 is the value range of the channel allocation indication, if the task of sensor model m is in local inference, i.e., α mn (t)+β mn If (t) = 0, channel allocation is disabled; C9 indicates that a sensor can be allocated a maximum of 1 channel; C10 indicates that a channel can be allocated a maximum of one sensor; C11 is the range of values ​​for the allocated computing resources of the edge server model; C12 indicates that the sum of the allocated computing resources equals the total computing resources of the edge server model; e n denoted as , where is the resource usage of the nth full-size deep neural network; e is the total resource usage of the edge server model.

[0042] The total service cost u(t) within time slot t is expressed as:

[0043]

[0044] Where partI represents the total processing latency of task mn; partII represents the total penalty for queue overflow; w p Indicates the penalty parameter;

[0045] in, This represents the computational latency of a lightweight deep neural network deployed on a sensor model, i.e.:

[0046]

[0047] Wherein, 1-α mn (t)-β mn (t) represents the sensor model task execution instruction; ζ mn (t) represents the raw input data generated on sensor m, and ζ mn (t)=v mn (t)s n v mn (t) represents the task arrival rate within time slot t, s n This represents the original data size of the nth type of task to be processed; This represents the computational resources of sensor m; B represents the computational cost of the nth type of inference task processed by a lightweight deep neural network; m，s (t) represents the computational tasks backlogged in the computation queue of sensor m within time slot t, in bits, and is represented as follows within time slot t+1:

[0048]

[0049] in,[*] + =max{*, 0}; The task queue capacity of the sensor model is represented by τ; the length of time slot t is represented by τ.

[0050] This represents the number of tasks that were discarded in the computation queue of sensor model m, i.e.:

[0051]

[0052] in, This indicates that a computation queue overflow event occurred in sensor m within time slot t;

[0053] This represents the offloading latency of the edge server model, including the wireless transmission latency from sensor m to the edge server model, the computation latency of the full-size deep neural network on the edge server model, and the download latency of the full-size deep neural network from the cloud server model to the edge server model.

[0054]

[0055] Where, α mn (t) indicates the edge server model task offloading indication within time slot t; b c2es This indicates the download speed from the edge server to the cloud server, measured in bits per second. γ represents the computational cost of processing the nth type of task by a full-size deep neural network; mk (t) represents the wireless channel allocation index; This represents the computational resources allocated to the nth class of full-size deep neural networks on the edge server in time slot t; x n (t) represents the deep neural network deployment index within time slot t; r mk (t) represents the data transmission rate of sensor m to the base station through the k-th channel, in bits / s, expressed as:

[0056]

[0057] Where B represents the bandwidth of the channel; p represents the signal-to-interference-plus-noise ratio. mk and h mk (t) represents the transmit power and channel gain, respectively; N0 represents the noise power spectral density; χ represents the average co-channel interference power controlled by the interference management scheme; B n，e (t) represents the computational tasks backlogged in the edge queue of the nth type of task within time slot t, which is represented as follows within time slot t+1:

[0058]

[0059] in, Let be the capacity of the edge queue for the nth type of task. When the edge queue is full, newly arriving tasks will be discarded.

[0060] This represents the number of tasks of type n that were discarded in the edge computing queue, i.e.:

[0061]

[0062] Among them, Ψ n，e (t)>0 indicates that an edge computing queue overflow event occurred on edge server m within time slot t;

[0063] This represents the cloud server model offloading latency, including the wireless transmission latency of task mn from sensor m to the edge server model, the wired transmission latency from the edge server model to the cloud server model, and the computation latency of the nth full-size deep neural network on the cloud server model, i.e.:

[0064]

[0065] Where, β mn (t) indicates the cloud server model task offloading instruction within time slot t; b es2c This represents the rate at which data is uploaded from the edge server model to the cloud server model, measured in bits per second; B n，c (t) represents the computational tasks backlogged in the cloud queue of the nth type of task within time slot t, which is represented as follows within time slot t+1:

[0066]

[0067] in, This represents the computing resources allocated by the cloud server model to each full-size deep neural network; This represents the capacity of the cloud queue for the nth type of task. When the cloud queue is full, newly arriving tasks will be discarded.

[0068] This represents the number of tasks of type n that were dropped in the cloud queue, i.e.:

[0069]

[0070] Among them, Ψ n，c (t)>0 indicates that a cloud queue overflow event occurred in cloud server model m within time slot t.

[0071] Specifically, in S4, the process of transforming the objective function model to obtain the transformed objective function model is as follows:

[0072] S41: Construct the accuracy model as follows:

[0073]

[0074] Where, η mn (t) represents the actual task precision of task mn in time slot t; and These represent the accuracy of lightweight deep neural networks and full-size deep neural networks, respectively.

[0075] S42: Construct the inference precision deficit queue for task mn, whose dynamic evolution is represented as follows:

[0076]

[0077] in, Indicates the required accuracy of the task; z mn (t) represents the deviation between the actual instantaneous accuracy and the long-term accuracy requirement, with its initial state being z. mn (0) = 0;

[0078] The state of satisfaction of long-term accuracy constraints, characterized by Lyapunov functions, is as follows:

[0079] L(z mn (t))=(z mn (t)) 2 / 2

[0080] Among them, L(z) mn The smaller the value of (t)), the higher the degree of satisfaction of long-term accuracy constraints;

[0081] S43: Using Lyapunov drift to capture the change of the Lyapunov function over two adjacent time slots, as follows:

[0082] Δ(z mn (t))=L(z mn (t+1))-L(z mn (t))

[0083] Its upper boundary is represented as:

[0084]

[0085] in, It is a constant. This represents the minimum inference precision required for task mn;

[0086] S44: The transformed objective function model, obtained by subtracting the total service cost objective function model, is as follows:

[0087]

[0088] st C1、C3-C12 (16)

[0089] Where V is a positive parameter.

[0090] Specifically, in S5, the process of training the transformed objective function model using the improved deep reinforcement learning model is as follows:

[0091] S51: Define a state space s(t), where the state space includes the sensor model task queue. Edge Task Queue Cloud Task Queue Channel state and task arrival rate Right now:

[0092]

[0093] S52: Define the action space, where the action space a(t) = {X(t), α(t), β(t), γ(t), F} e(t)}, and each index in a(t) must satisfy the constraint conditions C1-C12;

[0094] S53: Define the reward function as r(t) = -Δ(z) mn (t))-V·u(t);

[0095] S54: Employ an improved deep reinforcement learning model, randomly initialize all neural network parameters and the experience pool, and obtain the initial state s. t Set the time slot to t, determine the actions X(t), α(t), β(t), and γ(t), and solve F using an edge computing resource allocation optimization model. e (t), send X(t), α(t), β(t), γ(t) to the sensor model, and the sensor model reacts according to the action a. t Perform reasoning tasks and receive corresponding rewards. t And obtain the set of all deep neural network deployment metrics X(t+1) for the next time slot, with the new state being s. t+1 , the sequence {s t a t r t s t+1 Stored in the experience pool;

[0096] The improved deep reinforcement learning model uses randomly sampled data stored in the experience pool to retrain the transformed objective function model, and after training, an optimized objective function model is obtained.

[0097] Specifically, in S54, F is solved using an edge computing resource allocation optimization model. e The process of (t) is as follows:

[0098] Within time slot t, an edge computing resource allocation optimization model is constructed, which is as follows:

[0099]

[0100] st C11, C12 (17)

[0101] The edge computing resource allocation decision optimization model is obtained by constructing a Lagrangian function and solving it using the KKT conditions, thus yielding the optimal solution for the edge computing resource allocation decision optimization model as follows:

[0102]

[0103] in,

[0104] Specifically, in S54, the process of obtaining the set of all deep neural network deployment metrics X(t+1) for the next time slot is as follows:

[0105] Construct a dynamic programming deep neural network deployment optimization model, wherein the dynamic programming deep neural network deployment optimization model is as follows:

[0106]

[0107] stC1, C3 (19)

[0108] P4 is a constrained 0-1 knapsack problem, which can be solved using dynamic programming to obtain an approximate optimal solution. The state transition equation for the dynamic programming solution is as follows:

[0109]

[0110] Among them, f n，j (t) represents the maximum resource quantity, j represents the resource capacity, and e n This represents the amount of resources used by a full-size deep neural network of type n.

[0111] Beneficial Effects: This invention proposes a method to accelerate the inference task of product surface defect detection based on reinforcement learning. It quantifies a deep neural network deployment model, task offloading model, wireless channel allocation model, and computing resource model based on an edge-cloud collaborative system model, and establishes an objective function model that minimizes the total service cost. The objective function model is then transformed using Lyapunov optimization theory to obtain a transformed objective function model. This invention is based on an improved deep reinforcement learning model that embeds an edge computing resource allocation optimization model and a dynamic programming deep neural network deployment optimization model. After training the transformed objective function model, an optimized objective function model is obtained and used for inference of the task, minimizing the total service cost while ensuring the long-term inference accuracy requirement. Attached Figure Description

[0112] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0113] Figure 1 This is a flowchart of the method for accelerating the inference task of product surface defect detection based on reinforcement learning in this invention;

[0114] Figure 2 This is a schematic diagram of a scenario set in an embodiment of the present invention;

[0115] Figure 3This is a simulation result diagram of the convergence of the deep reinforcement learning model in the simulation experiment of the embodiment of the present invention;

[0116] Figure 4 This is a comparison chart of the total cost and task completion rate in the simulation experiment of this invention embodiment;

[0117] Figure 5 This is a comparison chart of the total cost and channel bandwidth results of the simulation experiment in this embodiment of the invention;

[0118] Figure 6 This is a comparison chart of the total cost of the simulation experiment in this embodiment of the invention with the computing resources of the edge server model;

[0119] Figure 7 This is a comparison chart showing the total cost of the simulation experiment in this embodiment of the invention with the storage resource results of the edge server model. Detailed Implementation

[0120] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0121] The collaborative inference of deep neural networks using resources from sensors, edge servers, and cloud servers is called edge-cloud collaboration. In collaborative inference, tasks can be inferred by lightweight deep neural networks on local sensors, offloaded to full-scale deep neural networks on edge servers, or offloaded to full-scale deep neural networks in the cloud. Through efficient resource management, the performance of the edge-cloud collaborative system model can be maximized. Two key aspects need to be considered throughout the collaborative inference process: 1) deep neural network deployment; 2) resource allocation. Furthermore, the task queuing problem on the sensor model, edge server model, and cloud server model should be considered. Tasks are queued before being inferred by the deep neural network, and the queue length represents the queuing time. Therefore, to avoid performance degradation of the edge-cloud collaborative system model due to increasing task queuing time, a maximum capacity is set for the queue. When the task length exceeds the queue capacity, the task is discarded. However, discarding too many tasks can affect task accuracy, so a penalty is needed for task discarding.

[0122] This embodiment sets up a three-layer target scene, such as Figure 2 As shown, M sensors are set up, denoted as a set. Set up a base station (BS) based on Orthogonal Frequency Division Multiple Access (OFDMA) and equip it with an edge server and a leased remote cloud server. Assume there are N types of deep neural network inference tasks, denoted as... The nth type of task is inferred using either a lightweight deep neural network or a full-size deep neural network corresponding to the nth type of task. The optimized lightweight deep neural network can be deployed on resource-scarce sensors. This can be achieved through model compression techniques, effectively reducing computational complexity while maintaining tolerable inference accuracy. Due to resource scarcity, it is assumed that a sensor can only generate a single type of task and support the deployment of lightweight deep neural networks (LDNNs, with fewer neurons and layers) corresponding to that task. The corresponding full-size deep neural networks (FDNNs, with more neurons and layers) can be deployed on cloud servers or edge servers. However, due to limited resources on edge servers, only a small number of full-size deep neural network deployments can be supported. In this embodiment, all full-size deep neural networks are deployed on resource-rich cloud servers. When a certain type of full-size deep neural network needs to be deployed on an edge server, it needs to be downloaded from the cloud server. The tasks generated by the sensor can be inferred locally on the sensor or offloaded to an edge server or cloud server via a wireless channel for inference.

[0123] Based on the above principles, this embodiment provides a method for accelerating the inference task of product surface defect detection based on reinforcement learning, such as... Figure 1 As shown, the specific steps include:

[0124] S1: Obtain the product reasoning task based on the sensor model;

[0125] S2: Construct a three-party collaborative model, namely the end-edge-cloud collaborative system model, with several sensor models, edge server models and cloud server models as participants. The end-edge-cloud collaborative system model includes a deep neural network deployment model, a task offloading model, a wireless channel allocation model and a computing resource allocation model.

[0126] S3: Construct an objective function model that minimizes the total service cost based on deep neural network deployment model, task offloading model, wireless channel allocation model, and computing resource allocation model;

[0127] S4: Using Lyapunov optimization theory, a precision model is introduced, a precision deficit queue is established based on the precision model, and the upper boundary of Lyapunov's drift is calculated, thereby transforming the objective function model to obtain the transformed objective function model;

[0128] S5: The transformed objective function model is trained using an improved deep reinforcement learning model to obtain an optimized objective function model, thereby accelerating the processing of the product's reasoning task.

[0129] In a specific embodiment, in S2, the end-edge-cloud collaborative system model operates in a time-slot manner, defining... The set of time slots is defined, with a length of τ for each time slot. The edge-cloud collaborative system model includes a deep neural network deployment model, a task offloading model, a wireless channel allocation model, and a computing resource model, wherein:

[0130] The deep neural network deployment model is as follows:

[0131]

[0132] Where X(t) represents the set of all deep neural network deployment metrics within time slot t. Let N be the number of deep neural network inference tasks, where n represents the nth task class. x n If x(t) = 1, it means that the nth type of full-size deep neural network is deployed on an edge server. If the nth type of full-size deep neural network is not deployed on an edge server, then x n (t) = 0;

[0133] The task unloading model is as follows:

[0134]

[0135]

[0136] Where α(t) represents the set of edge server task offloading indicators within time slot t. Let m represent the total number of sensors, and m represent the m-th sensor. mn represents the nth type of task of sensor m; α mn (t) = 1, indicating that task mn within time slot t is offloaded to the edge server model; otherwise, α mn (t) = 0; β(t) represents the set of cloud server task offloading instructions within time slot t, β mn (t) = 1, indicating that task mn is unloaded onto the cloud server model; otherwise, β mn (t)=0;1-α mn (t)-β mn (t) = 1, indicating that task mn is executed in the sensor model;

[0137] The wireless channel allocation model is as follows:

[0138]

[0139] Where, γ m (t) represents the set of channel allocation indicators. Let k represent the set of wireless channels, and k represent the k-th channel. γ mk If γ(t) = 1, it means that sensor m is assigned to the k-th channel; otherwise, γ mk (t) = 0;

[0140] The computing resource allocation model is as follows:

[0141]

[0142] Among them, F e (t) represents the set of computational resource allocation metrics for all full-size deep neural networks deployed on the edge server model. This represents the computational resources allocated to the nth class of full-size deep neural networks deployed on an edge server model in time slot t, and f e This represents the total computing resources of the edge server model.

[0143] In a specific embodiment, in S3, an objective function model that minimizes the total service cost is constructed based on a deep neural network deployment model, a task offloading model, a wireless channel allocation model, and a computing resource model. The objective function model is as follows:

[0144]

[0145] Set the constraints as follows:

[0146]

[0147]

[0148]

[0149]

[0150]

[0151]

[0152] C7: α mn (t)+β mn (t)≤1

[0153]

[0154]

[0155]

[0156]

[0157]

[0158] Where C1 represents the resource range occupied by the deep neural network deployed on the edge server model; C2 represents the accuracy requirement range met by the actual task; C3 represents the value range of the deep neural network deployment indicator; and C4 represents the value range when α... mn When (t) = 1, x n (t) is 1; C5 and C6 are the value ranges of the task offloading indication for the edge server model and cloud server model, respectively, i.e., whether the task is offloaded to the edge server model and cloud server model; C7 indicates that the task can be independently offloaded to the edge server model or cloud server model; C8 is the value range of the channel allocation indication, if the task of sensor model m is in local inference, i.e., α mn (t)+β mn If (t) = 0, channel allocation is disabled; C9 indicates that a sensor can be allocated a maximum of 1 channel; C10 indicates that a channel can be allocated a maximum of one sensor; C11 is the range of values ​​for the allocated computing resources of the edge server model; C12 indicates that the sum of the allocated computing resources equals the total computing resources of the edge server model; e n denoted as , where is the resource usage of the nth full-size deep neural network; e is the total resource usage of the edge server model.

[0159] The total service cost u(t) within time slot t is expressed as:

[0160]

[0161] Where partI represents the total processing latency of task mn; partII represents the total penalty for queue overflow; w p Indicates the penalty parameter;

[0162] in, This represents the computational latency of a lightweight deep neural network deployed on a sensor model, i.e.:

[0163]

[0164] Wherein, 1-α mn (t)-β mn (t) represents the sensor model task execution instruction; ζ mn (t) represents the raw input data generated on sensor m, and ζ mn (t)=v mn (t)s n v mn(t) represents the task arrival rate within time slot t, s n This represents the original data size of the nth type of task to be processed; This represents the computational resources of sensor m; B represents the computational cost of the nth type of inference task processed by a lightweight deep neural network; m，s (t) represents the computational tasks backlogged in the computation queue of sensor m within time slot t, in bits, and is represented as follows within time slot t+1:

[0165]

[0166] in,[*] + =max{*, 0}; The task queue capacity of the sensor model is represented by τ; the length of time slot t is represented by τ.

[0167] This represents the number of tasks that were discarded in the computation queue of sensor model m, i.e.:

[0168]

[0169] in, This indicates that a computation queue overflow event occurred in sensor m within time slot t;

[0170] This represents the offloading latency of the edge server model, including the wireless transmission latency from sensor m to the edge server model, the computation latency of the full-size deep neural network on the edge server model, and the download latency of the full-size deep neural network from the cloud server model to the edge server model.

[0171]

[0172] Where, α mn (t) indicates the edge server model task offloading indication within time slot t; b c2es This indicates the download speed from the edge server to the cloud server, measured in bits per second. γ represents the computational cost of processing the nth type of task by a full-size deep neural network; mk (t) represents the wireless channel allocation index; This represents the computational resources allocated to the nth class of full-size deep neural networks on the edge server in time slot t; x n (t) represents the deep neural network deployment index within time slot t; r mk (t) represents the data transmission rate of sensor m to the base station through the k-th channel, in bits / s, expressed as:

[0173]

[0174] Where B represents the bandwidth of the channel; p represents the signal-to-interference-plus-noise ratio. mk and h mk (t) represents the transmit power and channel gain, respectively; N0 represents the noise power spectral density; χ represents the average co-channel interference power controlled by the interference management scheme; B n，e (t) represents the computational tasks backlogged in the edge queue of the nth type of task within time slot t, which is represented as follows within time slot t+1:

[0175]

[0176] in, Let be the capacity of the edge queue for the nth type of task. When the edge queue is full, newly arriving tasks will be discarded.

[0177] This represents the number of tasks of type n that were discarded in the edge computing queue, i.e.:

[0178]

[0179] Among them, Ψ n，e (t)>0 indicates that an edge computing queue overflow event occurred on edge server m within time slot t;

[0180] This represents the cloud server model offloading latency, including the wireless transmission latency of task mn from sensor m to the edge server model, the wired transmission latency from the edge server model to the cloud server model, and the computation latency of the nth full-size deep neural network on the cloud server model, i.e.:

[0181]

[0182] Where, β mn (t) indicates the cloud server model task offloading instruction within time slot t; b es2c This represents the rate at which data is uploaded from the edge server model to the cloud server model, measured in bits per second; B n，c (t) represents the computational tasks backlogged in the cloud queue of the nth type of task within time slot t, which is represented as follows within time slot t+1:

[0183]

[0184] in, This represents the computing resources allocated by the cloud server model to each full-size deep neural network; This represents the capacity of the cloud queue for the nth type of task. When the cloud queue is full, newly arriving tasks will be discarded.

[0185] This represents the number of tasks of type n that were dropped in the cloud queue, i.e.:

[0186]

[0187] Among them, Ψ n，c (t)>0 indicates that a cloud queue overflow event occurred in cloud server model m within time slot t.

[0188] In this embodiment, P1 is a mixed-integer nonlinear programming model, meaning the variables to be solved are continuous-discrete-mixed and deeply coupled; the objective function model is nonconvex. Therefore, directly solving P1 is difficult. Furthermore, due to the high complexity of P1, iterative numerical methods based on problem decomposition and variable relaxation are time-consuming and do not meet the real-time requirements of this embodiment. Further analysis of problem P1 reveals that for any time slot t, the future values ​​of each model variable depend only on the current model state and are independent of the previous system model state. Therefore, problem P1 exhibits Markov property, making it a typical constrained Markov decision problem (CMDP). The key to solving CMDP problems lies in eliminating constraints. Lyapunov optimization is a stochastic optimization technique that can decompose long-term constraints into specific constraints for each time slot, i.e., the MDP problem within each time slot. Currently, the most common method for solving MDP problems is using deep reinforcement learning models. Therefore, this embodiment proposes an improved approximate solution method based on deep reinforcement learning models to obtain the asymptotic optimal solution for the entire problem.

[0189] In a specific embodiment, in S4, the inference accuracy of the nth type of task depends on the type of deep neural network processing the task (i.e., a lightweight deep neural network or a full-size deep neural network). The process of transforming the objective function model to obtain the transformed objective function model is as follows:

[0190] S41: Considering the three scenarios where the task performs inference on sensors, edge servers, and cloud servers, the accuracy model is constructed as follows:

[0191]

[0192] Where, η mn (t) represents the actual task precision of task mn in time slot t; and These represent the accuracy of lightweight deep neural networks and full-size deep neural networks, respectively.

[0193] S42: Construct the inference precision deficit queue for task mn, whose dynamic evolution is represented as follows:

[0194]

[0195] in, Indicates the required accuracy of the task; z mn (t) represents the deviation between the actual instantaneous accuracy and the long-term accuracy requirement, with its initial state being z. mn (0) = 0;

[0196] The state of satisfaction of long-term accuracy constraints, characterized by Lyapunov functions, is as follows:

[0197] L(z mn (t))=(z mn (t)) 2 / 2

[0198] Among them, L(z) mn The smaller the value of (t)), the higher the degree of satisfaction of long-term accuracy constraints;

[0199] S43: To ensure long-term accuracy constraints, a smaller value of the Lyapunov function is obtained. Lyapunov drift is used to capture the change of the Lyapunov function over two adjacent time slots, as follows:

[0200] Δ(z mn (t))=L(z mn (t+1))-L(z mn (t))

[0201] Its upper boundary is represented as:

[0202]

[0203] in, It is a constant. Let m represent the minimum inference precision for task mn; the first inequality is based on the substitution in equation (14), and the second inequality is based on... get.

[0204] S44: Based on Lyapunov optimization theory, while ensuring long-term accuracy, the P1 problem, which minimizes the total service cost, is transformed into a problem that minimizes both drift and total cost in each time slot. That is, in each time slot t, P1 is transformed into P2. The transformed objective function model, which minimizes the total service cost, is as follows:

[0205]

[0206] stC1, C3-C12 (16)

[0207] Here, V is a positive parameter used to balance the minimization of total service cost and the satisfactory state of long-term accuracy constraints. Its basic principle is that if the long-term accuracy constraint is violated, i.e., z...mn When (t) > 0, improving instantaneous inference accuracy is more urgent than reducing total service cost. Furthermore, the optimal solution of the objective function model transformed based on Lyapunov optimization theory exhibits asymptotic optimality.

[0208] In a specific embodiment, in S5, the process of training the transformed objective function model using an improved deep reinforcement learning model is as follows:

[0209] S51: Define a state space s(t), where the state space includes the sensor model task queue. Edge Task Queue Cloud Task Queue Channel state and task arrival rate Right now:

[0210]

[0211] S52: Define the action space, where the action space a(t) = {X(t), α(t), β(t), γ(t), F} e (t)}, and each index in a(t) must satisfy the constraint conditions C1-C12;

[0212] S53: Define the reward function as r(t) = -Δ(z) mn (t))-V·u(t);

[0213] S54: Employ an improved deep reinforcement learning model, randomly initialize all neural network parameters and the experience pool, and obtain the initial state s. t Set the time slot to t, determine the actions X(t), α(t), β(t), and γ(t), and solve F using an edge computing resource allocation optimization model. e (t), send X(t), α(t), β(t), γ(t) to the sensor model, and the sensor model reacts according to the action a. t Perform reasoning tasks and receive corresponding rewards. t And obtain the set of all deep neural network deployment metrics X(t+1) for the next time slot, with the new state being s. t+1 , the sequence {s t a t r t s t+1 Stored in the experience pool;

[0214] The improved deep reinforcement learning model uses randomly sampled data stored in the experience pool to retrain the transformed objective function model, and after training, an optimized objective function model is obtained.

[0215] In a specific embodiment, in S54, as shown in equation (2), edge computing resource allocation is unrelated to inference accuracy performance, and therefore only affects the service latency performance. Furthermore, edge computing resource allocation only affects edge offloading latency. Therefore, within time slot t, based on the determined task offloading model and channel allocation model, F is solved using the edge computing resource allocation optimization model. e The process of (t) is as follows:

[0216] Within time slot t, an edge computing resource allocation decision optimization model is constructed, which is as follows:

[0217]

[0218] st C11, C12 (17)

[0219] Where P3 is a convex problem, the edge computing resource allocation optimization model is solved by constructing a Lagrangian function and using the KKT conditions, thus obtaining the optimal solution of the edge computing resource allocation decision optimization model as follows:

[0220]

[0221] in, The edge computing resource allocation optimization model is used to allocate edge computing resources. Embedding it into the proposed deep reinforcement learning model can reduce the complexity of model training.

[0222] In a specific embodiment, in S54, the corresponding type of task can only be offloaded to the edge server for inference after the full-size deep neural network model is deployed there. However, due to the limited resources of the edge server, only a portion of the full-size deep neural network deployments can be satisfied. This embodiment mainly considers the active deployment of the full-size deep neural network, that is, when deploying the deep neural network model, based on the value of α(t) and satisfying constraint C1, in order to actively deploy the full-size deep neural network model to make the most of the edge server resources, a dynamic programming deep neural network deployment optimization model is set up. The process of obtaining the set of all deep neural network deployment indicators X(t+1) for the next time slot is as follows:

[0223] Construct a dynamic programming deep neural network deployment optimization model, wherein the dynamic programming deep neural network deployment optimization model is as follows:

[0224]

[0225] stC1, C3 (19)

[0226] P4 is a constrained 0-1 knapsack problem, which can be solved using dynamic programming to obtain an approximate optimal solution. The state transition equation for the dynamic programming solution is as follows:

[0227]

[0228] Among them, f n，j (t) represents the maximum resource quantity, j represents the resource capacity, and e n This represents the resource consumption of the nth class of full-size deep neural networks. The solution process is shown in Algorithm 1:

[0229]

[0230] The improved deep reinforcement learning model in this embodiment is based on the DDPG model technique. It primarily embeds an edge computing resource allocation optimization model and a deep neural network proactive deployment decision model. During computation, the edge server monitors and collects network state information, then sends actions to the sensors. DDPG is a typical Actor-Critic structure, consisting of an Actor network and a Critic network. The Actor network determines and obtains the action to be executed based on the current state, while the Critic network evaluates the current state and the actions already executed under that state. As shown in Algorithm 2, the algorithm in this embodiment runs in a time-slot manner.

[0231]

[0232] In the simulation experiment, this embodiment uses YoloV4-Tiny and YoloV5 as lightweight and full-size deep neural networks, respectively. These are currently popular compact and standard models in the Yolo series for two-stage object detection applications. A refractory brick defect database, including defects such as missing corners, missing edges, cracks, and holes, was used. The following datasets were used for five product defect classification services: a textured surface defect dataset, a hot-rolled strip steel (steel) surface defect database (NEUSDD), an image dataset of defective electronic commutators (KolektorSDD), and a magnetic tile surface defect dataset (MTDD), i.e., n = 1, 2, 3, 4, 5. Each dataset contains product defect image samples, where each image is labeled with the location and category of the defect. The defect types are included. Two deep neural networks were trained using samples from each dataset, and the inference accuracy and image data size were measured by varying the resolution of each image in the dataset. Table 1 lists the inference accuracy for the five types of tasks.

[0233] Table 1 Task Accuracy Table

[0234]

[0235]

[0236] This embodiment considers a product defect classification IIoT scenario with multiple sensors, including a base station equipped with an edge server and a remote cloud server. YoloV4-Tiny and YoloV5 were used as lightweight and full-size deep neural networks, respectively, and were trained using the refractory brick defect database, DAGM2007 Dataset, NEUSDD, KolektorSDD, and MTDD. In the environmental parameter settings, the sensor arrival rate in each time slot was randomly distributed between [1, 1.5] / s, and the channel conditions were modeled as "good," "normal," and "bad." The default parameter values ​​used in the experiment are shown in Tables 2 and 3. The algorithm proposed in this patent will be compared with the following benchmark algorithms:

[0237] (1) SFDD scheme: The deep neural network on the edge server is deployed in a fixed manner.

[0238] (2) EARA scheme: adopts an equivalent computing resource allocation and random channel allocation scheme. The computing resources are evenly allocated to each deep neural network deployed on the edge server, and the channels are randomly allocated to the sensors.

[0239] Table 2 System Model Parameters

[0240]

[0241] Table 3 Algorithm Parameters

[0242]

[0243] The simulation results are as follows:

[0244] (1) Convergence of the improved deep reinforcement learning model in this embodiment: such as Figure 3 As shown, it can be clearly seen that as the number of training sets increases, the average cost gradually decreases and converges, thus verifying the convergence of the improved deep reinforcement learning model in this embodiment.

[0245] (2) Impact of task arrival rate: Once offline training shows good results, the performance of the proposed algorithm in online inference can be evaluated. For example... Figure 4As shown, increasing the task arrival rate from the default value of 0.6 by 1 results in a decrease in performance for SFDD, ERA, and the algorithm proposed in this paper. In the SFDD scheme, a portion of tasks are fixedly deployed in the edge server model; therefore, the remaining tasks must be executed locally or offloaded to the cloud server. The ERA scheme performs the worst, mainly because it does not consider the resource optimization configuration of the system model, but only adopts the simplest averaging and randomization methods. The scheme proposed in this embodiment, however, provides more comprehensive resource management, thus offering a better resource configuration scheme and outperforming other schemes.

[0246] (3) Impact of Communication Bandwidth: In the experiment, the impact of wireless transmission on the system model performance was verified by increasing the channel bandwidth from the default value of 5MHz to 25MHz. Figure 5 It can be seen that with the improvement of wireless transmission, the system model performance of each scheme has improved, and the scheme proposed in this embodiment is still the best. The scheme proposed in this embodiment reduces the cost by 5.41% and 15.25% compared with SFDD and EAER schemes, respectively. When the bandwidth is small, it can be found that SFDD performs better than the scheme proposed in this embodiment, mainly because bandwidth affects the model download latency, leading to an increase in total cost.

[0247] (4) Impact of edge server computing resources: such as Figure 6 As shown, the performance of all solutions improves when the total computing resources of the edge server model increase from 1.0 GHz to 1.20 GHz. The proposed solution in this patent reduces the total cost very quickly. This is mainly because the increased edge computing resources allow more tasks to be offloaded to the edge server model, thereby reducing the load on the sensor model and cloud server model. Compared to the proposed solution in this patent, the SFDD solution uses a fixed deep neural network deployment strategy, and the ERA solution does not consider optimizing resource allocation. Therefore, neither the SFDD nor the ERA solution fully utilizes the benefits of increased computing resources in the edge server model.

[0248] (5) Impact of storage resources on edge server models: such as Figure 7 As shown, the system model performance of each scheme improves with increasing storage resources in the edge server model. However, it can be observed from the figure that when storage resources are limited, the SFDD (Fixed Deep Neural Network Deployment) scheme performs better. This is because the storage resources of the edge server model can only deploy a small number of full-size deep neural network models, and frequent deployment of deep neural network models leads to increased transmission latency, thereby increasing the total cost. However, as storage resources increase, the performance of the scheme proposed in this embodiment and EAER gradually improves, surpassing the SFDD scheme.

[0249] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A product surface defect detection inference task acceleration method based on reinforcement learning, characterized in that, The specific steps include: S1: Obtain the product reasoning task based on the sensor model; S2: Construct a three-party collaborative model, namely the end-edge-cloud collaborative system model, with several sensor models, edge server models and cloud server models as participants. The end-edge-cloud collaborative system model includes a deep neural network deployment model, a task offloading model, a wireless channel allocation model and a computing resource allocation model. S3: Construct an objective function model that minimizes the total service cost based on deep neural network deployment model, task offloading model, wireless channel allocation model, and computing resource allocation model; S4: Using Lyapunov optimization theory, a precision model is introduced, a precision deficit queue is established based on the precision model, and the upper boundary of Lyapunov's drift is calculated, thereby transforming the objective function model to obtain the transformed objective function model; In S4, the process of transforming the objective function model to obtain the transformed objective function model is as follows: S41: Construct the accuracy model as follows: wherein, representing a task in a time slot actual task accuracy; with representing the accuracy of a light-weight deep neural network and a full-size deep neural network, respectively; S42: Constructing a task the reasoning accuracy deficit queue, which dynamically evolves as: wherein, represents the task accuracy requirement; represents the deviation of the actual instantaneous accuracy from the long-term accuracy requirement, whose initial state is ; The state of satisfaction of long-term accuracy constraints, characterized by Lyapunov functions, is as follows: wherein, The smaller the value, the higher the degree of satisfaction of the long-term accuracy constraint. S43: Using Lyapunov drift to capture the change of the Lyapunov function over two adjacent time slots, as follows: Its upper boundary is represented as: in, It is a constant. Indicates task The lowest inference accuracy; S44: The transformed objective function model, obtained by subtracting the total service cost objective function model, is as follows: in, Indicates time slot The set of metrics for all deep neural network deployments within the system. Indicates shared ownership Deep neural network-like inference tasks, Indicates the first Class of tasks; Indicates time slot The set of edge server task offloading instructions within the system. Indicates the total number of sensors. Indicates the first One sensor; Indicates time slot A collection of cloud server task unloading instructions; Represents the set of channel allocation indicators; This represents the set of computational resource allocation metrics for all full-size deep neural networks deployed on an edge server model. It is a positive parameter; All are constraints; Indicates time slot Total service cost within ; S5: The transformed objective function model is trained using an improved deep reinforcement learning model to obtain an optimized objective function model, thereby accelerating the processing of the product's reasoning task.

2. The reinforcement learning-based product surface defect detection inference task acceleration method according to claim 1, characterized in that, In S2, the end-edge-cloud collaborative system model operates in a time-slot manner. This model includes a deep neural network deployment model, a task offloading model, a wireless channel allocation model, and a computing resource model, wherein: The deep neural network deployment model is as follows: in, Indicates time slot The set of metrics for all deep neural network deployments within the system. Indicates shared ownership Deep neural network-like inference tasks, Indicates the first Class of tasks, and ; Then it means the first A full-size deep neural network is deployed on an edge server, if the first If a full-size deep neural network is not deployed on an edge server, then... ; The task unloading model is as follows: ； in, Indicates time slot The set of edge server task offloading instructions within the system. Indicates the total number of sensors. Indicates the first One sensor, and , Indicates sensor The Class of tasks; , indicating time slot Internal tasks It is offloaded to the edge server model, otherwise ; Indicates time slot The set of cloud server task unloading instructions within the system. , indicating task It is unloaded onto the cloud server model; otherwise, ; , indicating task Executed in the sensor model; The wireless channel allocation model is as follows: in, Represents the set of channel allocation indicators. Represents the set of wireless channels. Indicates the first One channel, and ; , indicating sensor Assigned to the One channel, otherwise ; The computing resource allocation model is as follows: in, This represents the set of computational resource allocation metrics for all full-size deep neural networks deployed on an edge server model. This indicates the first [server] deployed on the edge server model. Full-size deep neural networks in time slots The computing resources allocated to it, and , This represents the total computing resources of the edge server model.

3. The reinforcement learning-based product surface defect detection inference task acceleration method according to claim 2, characterized in that, In S3, an objective function model that minimizes the total service cost is constructed based on a deep neural network deployment model, a task offloading model, a wireless channel allocation model, and a computing resource model. The objective function model is as follows: Set the constraints as follows: Where C1 represents the resource range occupied by the deep neural network deployed on the edge server model; C2 represents the accuracy requirement range that the actual task accuracy must meet; C3 represents the value range of the deep neural network deployment indicator; and C4 represents the value range when... hour, =1; C5 and C6 are the value ranges of the task offloading indication for the edge server model and cloud server model, respectively, i.e., whether the task is offloaded to the edge server model and cloud server model; C7 indicates that the task can be independently offloaded to the edge server model or cloud server model; C8 is the value range of the channel allocation indication, if the sensor model The task involves local reasoning, i.e. If C10 is not specified, channel allocation is disabled; C9 indicates that a sensor can be allocated a maximum of 1 channel; C10 indicates that a channel can be allocated to a maximum of one sensor; C11 is the range of values ​​for the computing resources allocated to the edge server model; C12 indicates that the sum of the allocated computing resources equals the total computing resources of the edge server model. For the first The amount of resources used by a full-size deep neural network; This represents the total resources of the edge server model. wherein the time slot total cost of service within the time slot is represented as: wherein, representing the total processing delay of a task representing the total processing delay of a task representing the total penalty of a queue overflow; representing a penalty parameter; wherein, denotes the computational latency of a lightweight deep neural network deployed on the sensor model, i.e.: in, Indicates the sensor model task execution instructions; Indicates sensor The original input data generated above, and , Indicates in time slot Within the task arrival rate, Indicates the first The size of the original data for the task to be processed; Indicates sensor Computing resources; This indicates the first process processed by a lightweight deep neural network. The computational complexity of reasoning tasks; Indicates in time slot Internal sensor The backlog of computation tasks in the computation queue, measured in bits, occurs in time slots. The inner representation is as follows: wherein ; represents the task queue capacity of the sensor model; represents the length of a time slot ; Representation of sensing model The number of tasks dropped in the calculation queue, i.e.: wherein, indicates that the sensor transmits data within a time slot a compute queue overflow event has occurred; representing edge server model offload latency, including sensor wireless transmission latency to the edge server model, computation latency of the full-size deep neural network on the edge server model, and download latency of the edge server model downloading the full-size deep neural network from the cloud server model, i.e.: in, This indicates the download speed from the edge server to the cloud server, measured in bits per second. This represents the first process processed by a full-size deep neural network. The computational cost of the task; Indicates the wireless channel allocation index; Represents the first on the edge server Full-size deep neural networks in time slots The computing resources allocated to it; Indicates time slot In-house deep neural network deployment metrics; Indicates sensor Through the first The data transmission rate from each channel to the base station, measured in bits per second, is expressed as: in, Indicates the bandwidth of the channel; This represents the signal-to-interference-plus-noise ratio. and These are transmit power and channel gain, respectively. Indicates the noise power spectral density; This represents the average co-channel interference power controlled by the interference management scheme; Indicates in time slot Inner The computational tasks that are backed up in the edge queue of the task class, which are in time slots The inner representation is as follows: wherein, is the capacity of the edge queue for tasks of the class, when the edge queue is full, newly arrived tasks will be discarded; the number of tasks of the first class that are discarded in the edge computing queue, i.e.: the number of tasks of the second class that are discarded in the edge computing queue, i.e.: in, Indicates in time slot Inner edge server An edge computing queue overflow event occurred; This indicates the cloud server model offloading delay, including tasks. From the sensor Wireless transmission latency to the edge server model, wired transmission latency from the edge server model to the cloud server model, and the third The computational latency of a full-size deep neural network on a cloud server model, namely: in, express ; This indicates the rate at which data is uploaded from the edge server model to the cloud server model, expressed in bits per second. Indicates in time slot Inner The backlog of computational tasks in the cloud queue of similar tasks, which are in time slots The inner representation is as follows: in, This represents the computing resources allocated by the cloud server model to each full-size deep neural network; Indicates the first The capacity of the cloud queue for this type of task; when the cloud queue is full, newly arriving tasks will be discarded. the number of tasks of the first class that are discarded in the cloud queue, i.e.: the number of tasks of the second class that are discarded in the cloud queue, i.e.: the number of tasks of the third class that are discarded in wherein, represents a cloud server model within a time slot represents a cloud server model within a time slot A cloud queue overflow event has occurred.

4. The reinforcement learning-based product surface defect detection inference task acceleration method according to claim 3, characterized in that, In S5, the process of training the transformed objective function model using the improved deep reinforcement learning model is as follows: S51: Define the state space The state space includes a sensor model task queue. Edge task queue Cloud task queue Channel state and task arrival rate ,Right now: S52: Define the action space, wherein the action space ,and Each indicator must meet the constraints. ; S53: define a reward function as ; S54: Employ an improved deep reinforcement learning model, randomly initialize all neural network parameters and the experience pool to obtain the initial state. Set the time slot to Determine the action And solve it through the edge computing resource allocation optimization model. ,Will Send to the sensor model, the sensor model based on the action Perform reasoning tasks and receive corresponding rewards. And obtain the set of all deep neural network deployment metrics for the next time slot. The new status is , convert the sequence Store in the experience pool; The improved deep reinforcement learning model uses randomly sampled data stored in the experience pool to retrain the transformed objective function model, and after training, an optimized objective function model is obtained.

5. The reinforcement learning-based product surface defect detection inference task acceleration method according to claim 4, characterized in that, In S54, the edge computing resource allocation optimization model is solved The process is as follows: In a time slot An edge computing resource allocation optimization model is constructed, which is: The edge computing resource allocation optimization model is obtained by constructing a Lagrangian function and solving it using the KKT conditions, thus yielding the optimal solution for the edge computing resource allocation optimization model as follows: wherein .

6. The reinforcement learning-based product surface defect detection inference task acceleration method according to claim 5, characterized in that, In S54, a next time slot all deep neural network deployment index set is obtained The process is as follows: Construct a dynamic programming deep neural network deployment optimization model, wherein the dynamic programming deep neural network deployment optimization model is as follows: wherein, is a 0-1 knapsack problem with constraints, and its approximate optimal solution is obtained by dynamic programming, and the state transition equation of the dynamic programming solution is: wherein, represents the maximum resource amount, represents the resource capacity, represents the resource amount occupied by the first class full-size deep neural network.

Images