Global delay optimization-oriented cloud-edge-end collaborative CNN inference method and system

Abstract

Disclosed in the present invention is a global delay optimization-oriented cloud-edge-end collaborative CNN inference method. A block inference delay prediction model is constructed and trained, theoretical data transmission delay is calculated, a delay loss prediction model is constructed and trained, and a global delay prediction model is constructed; the global delay prediction model is used to calculate the theoretical global delay of each edge server, and the minimization of global delay is taken as an optimization objective, so as to determine the selection of an edge server participating in collaborative inference and the allocation of inference task volumes. The present invention also provides a global delay optimization-oriented cloud-edge-end collaborative CNN inference system. Compared with conventional collaborative inference research work which focuses only on delay optimization of current inference tasks in terms of delay indicator settings, the method takes possible delay influence of processing of current inference tasks on other inference tasks processed in parallel into consideration for decision making. In addition, the present invention further provides a CNN segmentation optimization method, which can effectively reduce the decision-making delay in collaborative inference.

Description

Cloud-edge-device collaborative CNN inference method and system for global latency optimization Technical Field

[0001] The present invention belongs to the field of cloud-edge-end collaborative computing, and specifically relates to a cloud-edge-end collaborative CNN inference method and system for global latency optimization. Background Art

[0002] Recent advances in convolutional neural networks (CNNs) are driving a growing number of intelligent applications at the edge of networks, such as smart homes, smart factories, and smart cities. To deploy computationally intensive CNNs on resource-constrained physical devices, traditional approaches rely on offloading inference to a remote cloud or optimizing computation locally on the edge device. However, cloud-assisted approaches are hampered by unreliable and latency-prone wide area networks, while local computation methods are limited by the limited computing power of edge devices.

[0003] To meet the demands of low-latency and high-accuracy CNN inference, the emerging cloud-edge-device collaborative inference computing paradigm has become a research focus. However, current collaborative inference research focuses solely on latency optimization for the current inference task, without considering the potential latency impact on other parallel inference tasks.

[0004] Summary of the Invention

[0005] The technical problem to be solved by the present invention is: in view of the problem that the existing cloud-edge collaborative reasoning calculation does not consider the possible delay impact on other parallel processing reasoning tasks, a cloud-edge collaborative CNN reasoning method and system for global delay optimization is provided to better combine the cloud server and edge computing paradigms, and fully tap the computing potential of the edge server. At the same time, the CNN segmentation optimization method proposed in the present invention makes collaborative reasoning decisions between CNN blocks, which can effectively save decision delays in collaborative reasoning.

[0006] To solve the above technical problems, the present invention provides the following technical solution: a cloud-edge-device collaborative CNN inference method for global latency optimization, comprising the following steps:

[0007] S1. Based on the trained and N-layer CNN model, divide the N-layer CNN model into n blocks: CNN1, CNN2, ...CNN j ,…CNN n ,1≤j≤n,n>1;

[0008] S2, offline learning phase: using the known edge server e I FS floating point computing capability I, the computational complexity of the CNN block being processed, the average computational complexity of the CNN block to be processed, and the number of operation layers contained in the CNN block to be processed are taken as input, and the corresponding edge server e I Calculate the actual block inference delay generated by the CNN block to be processed as output, and build and train a block inference delay prediction model; the operation layer includes a convolution layer, a fully connected layer, and a pooling layer;

[0009] S3. Using the known floating-point computing capability of the edge server, the computational complexity of the CNN block to be processed, the average computational complexity of the CNN block being processed, and the number of operation layers contained in the CNN block being processed as input, and using the actual delay loss caused by the CNN block to be processed to the CNN block being processed calculated by the corresponding edge server as output, a delay loss prediction model is constructed and trained.

[0010] S4. Based on the physical terminal, the initial image data is processed into tensor data with the same resolution and data size, and the tensor data is sent to the edge server in the same local area network; the edge server collects the network bandwidth between itself and other edge servers in the local area network and calculates the theoretical data transmission delay;

[0011] S5. Input the actual floating-point computing capability of the edge server, the computational complexity of the CNN block being processed, the average computational complexity of the CNN block to be processed, and the number of layers contained in the CNN block to be processed into the block inference delay prediction model obtained in step S2 to calculate the theoretical block inference delay generated by the edge server computing the CNN block to be processed.

[0012] The actual floating-point computing capability of the edge server, the computational complexity of the CNN block to be processed, the average computational complexity of the CNN block being processed, and the number of operation layers contained in the CNN block being processed are input into the delay loss prediction model obtained in step S3 to calculate the theoretical delay loss caused by the edge server calculating the CNN block to be processed on the CNN block being processed.

[0013] S6. Using the theoretical data transmission delay obtained in S4, the theoretical block inference delay generated by the edge server calculating the CNN block to be processed obtained in step S5, the theoretical delay loss generated by the edge server calculating the CNN block to the CNN block being processed as input, and the theoretical global delay of the edge server as output, a global delay prediction model is constructed;

[0014] S7. Calculate the theoretical global delay of each edge server using the global delay prediction model;

[0015] S8. Determine whether the theoretical global delay of the current edge server is less than or equal to the theoretical global delay of other edge servers. If so, the current edge server calculates the CNN block to be processed, and then executes step S9; otherwise, the CNN block to be processed is transmitted to other edge servers for calculation. The selected other edge servers complete the calculation of the CNN block to be processed, and then execute step S9 to complete the decision on the next CNN block.

[0016] S9. Determine whether all calculations of the CNN block are completed. If so, the edge server sends the calculation results of the CNN block to the physical terminal that issued the task request. Otherwise, return to step S7 to complete the decision on the next CNN block.

[0017] Furthermore, in the aforementioned step S2, the block inference delay prediction model is as follows:

[0018] Where, FS I Is the server e I Its own floating-point computing capabilities, It is the CNN block e that it is processing I _CNN j The computational complexity of _pt, is the CNN block to be processed j The average computational complexity of CNN is m. j The number of operation layers included, e I _tmd j _p is the edge server e I Computing CNN j The resulting theoretical block inference latency output.

[0019] Furthermore, in the aforementioned step S3, the delay loss prediction model is as follows:

[0020] Where, FS I is the floating point computing capability of the edge server, is the CNN block to be processed j The computational complexity of is the CNN block being processed I _CNN j _pt is the average computational complexity, m_pt is the number of operation layers contained in the CNN block being processed, e I _t loss _p is the edge server e I Computing CNN j For the e being processed I _CNN j The theoretical delay loss caused by _pt.

[0021] Furthermore, in the aforementioned step S4, the theoretical data transmission delay is calculated according to the following formula: IJ =Td q-1 / bw IJ ,

[0022] In the formula, Td q-1 is tensor data, bw IJ is the network bandwidth between edge servers.

[0023] Furthermore, in the aforementioned step S6, the global delay prediction model is as follows:

[0024] Furthermore, in the aforementioned step S1, CNN j Contains at least two consecutive tensor data, for any CNN containing m tensor data j In terms of tensor data and operation layers, it contains: Td j-1 Td j-2 ,…Td j-m , Ly j-1 ,Ly j-2 ,…Ly j-m , m>1 where Td j-1 The computational complexity is greater than that of CNN j The computational complexity of any other tensor data in ; For any two adjacent CNN blocks, that is, CNN p and CNN q :CNN1,…CNN p , CNN q ,…CNN n , 1≤p<q≤n, qp=1, CNN p The last tensor data and CNN in q The first tensor in is the same as the data.

[0025] Furthermore, in the aforementioned step S4, the initial image data is processed into tensor data with the same resolution and data size based on the physical terminal, and the tensor data is sent to the edge server in the same local area network. Specifically, the initial image data is processed into image feature data with the same data size and resolution, and the image feature data is used as the tensor data Td 1-1 Input CNN1, Td 1-1 After the operation layer calculation in CNN1, tensor data Td is generated 2-1 And pass it to CNN2, that is: in CNN j In the tensor data Td j-1 Through the operating layer Ly j-1 , Lyj-2 ,…Ly j-m After calculation, tensor data Td is generated j-m And pass it to the next CNN block, CNN block CNN n The generation of the last tensor data in marks the end of CNN inference.

[0026] Furthermore, the aforementioned cloud-edge-end collaborative CNN inference method for global latency optimization, CNN block CNN j The total computational complexity of all operation layers in is: Denoted as CNN j The computational complexity of CNN j The average computational complexity is recorded as:

[0027] Computing CNN j The block inference delay generated by all the operation layers included is tmd j , For any operating layer Ly i The computational complexity of .

[0028] Furthermore, the aforementioned cloud-edge-end collaborative CNN inference method for global latency optimization, each CNN operation layer Ly i The computational complexity is:

[0029] In the formula, the operating layer Ly i is the convolution layer, and its computational complexity is Operation layer Ly i is a fully connected layer, and its computational complexity is H, W, C in Represents the input operation layer Ly in turn i Tensor data Td i The height, width, and number of channels of K represent the input operation layer Ly i The side length of the convolution kernel, C out Represents the output operation layer Ly i Tensor data Td i+1 The number of channels, I and Q represent the input operation layer Ly i Tensor data Td i Dimension, output operation layer Ly i Tensor data Td i+1 The dimension of .

[0030] On the other hand, the present invention proposes a cloud-edge-end collaborative CNN inference system for global latency optimization, comprising: a cloud server c, at least two edge servers within the communication range of the cloud server c, the edge servers being deployed on a WIFI access point or base station, and at least one physical terminal being provided within the communication range of each edge server; for any edge server e within the communication range of the cloud server c, I , edge server e I The other o edge servers within the communication range whose physical distance is less than the preset distance x are recorded as: e1, e2, ...e J ,…e o , 1≤J≤o, o>1, and these O edge servers are also within the communication range of cloud server c. These o edge servers are the same as edge server e I Together they form an edge cluster; the cloud server includes: a convolutional neural network, a model training center, and a communication module;

[0031] The edge server includes: a situation awareness center, a convolutional neural network, a strategy generation center, and a communication module;

[0032] The physical terminal includes: a communication module;

[0033] The model training center is used to train convolutional neural networks, inference delay prediction models, and delay loss prediction models;

[0034] The convolutional neural network is used for a CNN that has been fully trained and serves intelligent applications and is divided into n CNN blocks. j n-layer CNN;

[0035] The communication module is used to send and receive data between the cloud server, edge server and physical terminal;

[0036] The situation awareness center includes a workload awareness module and a network telemetry module; the workload awareness module is used to collect the floating point computing capacity FS of the edge server itself. I , the CNN block that is responsible for reasoning I _CNN j The computational complexity of _pt And the average computational complexity Other o edge servers e J Floating point computing capability FS J , other o edge servers e J The CNN block responsible for inference J _CNN j The computational complexity of _pt And the average computational complexity

[0037] The network telemetry module is used to collect edge server e I and edge servers J The network bandwidth bw IJ ;

[0038] The strategy generation center includes a block reasoning delay prediction module, a transmission delay calculation module, a global delay prediction module, an offline sample data storage module and a decision information generation module;

[0039] The block reasoning delay prediction module is used to predict the time delay of the edge server using the known edge server e I FS floating point computing capability I , the computational complexity of the CNN block being processed, the average computational complexity of the CNN block to be processed, and the number of layers contained in the CNN block to be processed are taken as input, and the corresponding edge server e I Calculate the actual block inference latency generated by the CNN block to be processed as output, and build and train a block inference latency prediction model;

[0040] The transmission delay calculation module is used to process the image data into tensor data with the same resolution and data size based on the physical terminal, and send the tensor data to the edge server in the same local area network; the edge server collects the network bandwidth between itself and other edge servers in the local area network and calculates the theoretical data transmission delay;

[0041] The global delay prediction module is used to construct a global delay prediction model with the theoretical data transmission delay, the theoretical block inference delay generated by the edge server calculating the CNN block to be processed, and the theoretical delay loss generated by the edge server calculating the CNN block to the CNN block being processed as input, and the edge server theoretical global delay as output;

[0042] The offline sample storage module is used to collect and store edge server e I The floating point computing capability is FS I , the CNN block e that it is processing I _CNN j The computational complexity of _pt is When the number of computational layers is m and the average computational complexity is CNN block CNN j The actual block inference delay e I _tmd j ; Collection and storage edge server e I The floating point computing capability is FS I , the number of operation layers being processed is m_pt and the average computational complexity is CNN block I_CNN j _pt, the processing computation complexity is CNN block CNN j For the e being calculated I _CNN j The actual delay loss t caused by _pt loss ;

[0043] The decision information generation module is used to determine whether the theoretical global delay of the current edge server is less than or equal to the theoretical global delay of other edge servers. If so, the current edge server calculates the CNN block to be processed, and then determines whether all the CNN blocks are calculated. If so, the edge server sends the calculation result of the CNN block to the physical terminal that issued the task request; otherwise, the CNN block to be processed is transmitted to other edge servers for calculation until all the CNN blocks are calculated.

[0044] Compared with the existing technology, the beneficial effects of the present invention are as follows: Different from the CNN collaborative reasoning system dominated by terminal devices and cloud computing centers, the present invention combines cloud servers and edge computing paradigms, where the cloud server is responsible for model training and completes the CNN collaborative reasoning work within the edge cluster, fully tapping the computing potential of the edge server.

[0045] The present invention theoretically models the block reasoning delay, data transmission delay and delay loss in the CNN collaborative reasoning process, and takes global delay minimization as the optimization goal to determine the selection of edge servers participating in collaborative reasoning and the allocation of reasoning tasks.

[0046] Compared with traditional collaborative reasoning research work that only focuses on the latency optimization of the current reasoning task in terms of latency index formulation, the present invention also takes into account the possible latency impact of the current reasoning task on other parallel reasoning tasks. At the same time, the present invention also provides a CNN segmentation optimization method, which makes collaborative reasoning decisions between CNN blocks and blocks, and can effectively save decision delays in collaborative reasoning. CNN collaborative reasoning work dominated by edge clusters places the computing work on the edge side closer to the physical terminal, which can effectively ensure the security of computing data and reduce the occupancy of network bandwidth. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] FIG1 is a schematic diagram of the CNN segmentation optimization principle of the present invention.

[0048] FIG2 is a technical principle diagram of the present invention.

[0049] FIG3 is a schematic diagram showing the module composition of the strategy generation center of the present invention.

[0050] FIG4 is a flowchart of the workflow of the present invention. DETAILED DESCRIPTION

[0051] In order to better understand the technical content of the present invention, specific embodiments are given below in conjunction with the accompanying drawings.

[0052] Various aspects of the present invention are described herein with reference to the accompanying drawings, which show a number of illustrative embodiments. The embodiments of the present invention are not limited to those described in the accompanying drawings. It should be understood that the present invention can be implemented by any of the various concepts and embodiments described above, as well as the concepts and implementations described in detail below, because the concepts and embodiments disclosed herein are not limited to any particular implementation. In addition, some aspects disclosed herein may be used alone or in any appropriate combination with other aspects disclosed herein.

[0053] Combined with Figure 1, CNN is a multi-layer structure. For an N-layer CNN, it has N operating layers: Ly1, Ly2, ...Ly i ,…Ly N , 1≤i≤N+1, N>1, its tensor data has N+1: Td1, Td2, ...Td i ,…Td N+1 , where the tensor data Td i Through the operating layer Ly i After calculation, tensor data Td is generated i+1 The delay consumption caused by CNN inference is mainly caused by the calculation of each operation layer Ly i The generated layered calculation delay Tmd i Composition. Operation layer Ly i It can be divided into three categories: convolutional layer, fully connected layer and pooling layer. Among them, the computational complexity of convolutional layer is the highest, and the computation of pooling layer is the simplest among all the fully connected layers, so it can be ignored. i The computational complexity of is expressed as follows:

[0054] In the formula, the operating layer Ly i is the convolution layer, and its computational complexity is Operation layer Ly i is a fully connected layer, and its computational complexity is H, W, C in Represents the input operation layer Ly in turn i Tensor data Td i The height, width, and number of channels of K represent the input operation layer Ly i The side length of the convolution kernel, C out Represents the output operation layer Ly i Tensor data Td i+1 The number of channels, I and Q represent the input operation layer Ly i Tensor data Tdi Dimension, output operation layer Ly i Tensor data Td i+1 The dimension of .

[0055] For an N-layer CNN, the present invention performs segmentation optimization operations on it and divides it into n blocks: CNN1, CNN2, ...CNN j ,…CNN n , 1≤j≤n, n>1. Any CNN block CNN j The composition characteristics are as follows:

[0056] Each CNN j Contains at least two consecutive tensor data;

[0057] For any CNN with m tensor data j In terms of tensor data and operation layers, it contains: Td j-1 , Td j-2 ,…Td j-m , Ly j-1 , Ly j-2 ,…Ly j-m , m>1, where Td j-1 The computational complexity is greater than that of CNN j The computational complexity of any other tensor data in ;

[0058] For any two adjacent CNN blocks: CNN p and CNN q (CNN1, ...CNN p , CNN q ,…CNN n , 1≤p<q≤n, qp=1), then CNN p The last tensor data and CNN in q The first tensor data in is the same;

[0059] In the specific CNN reasoning process, the initial image data is processed into image feature data with the same data size and resolution. The image feature data is used as tensor data Td 1-1 Input CNN1, Td 1-1 After a series of operation layer calculations in CNN1, tensor data Td is generated 2-1 And pass it to CNN2, that is: in CNN j In the tensor data Td j-1 Through the operating layer Ly j-1 , Ly j-2 ,…Ly j-m After calculation, tensor data Td is generated j-mAnd pass it to the next CNN block, CNN block CNN n The generation of the last tensor data in marks the end of CNN inference.

[0060] CNN Block CNN j The total computational complexity of all operation layers in Denoted as CNN j The computational complexity of CNN j The average computational complexity is recorded as:

[0061] Computing CNN j The block inference delay generated by all the operation layers included is recorded as: tmd j , For any operating layer Ly i The computational complexity of .

[0062] Referring to FIG4 , the present invention proposes a method for cloud-edge-device collaborative CNN inference for global latency optimization, comprising the following steps:

[0063] S1. Based on the trained and N-layer CNN model, the N-layer CNN model is divided into n blocks: CNN1, CNN2, ...CNN j ,…CNN n , 1≤j≤n, n>1;

[0064] S2, offline learning phase: using the known edge server e I FS floating point computing capability I , the computational complexity of the CNN block being processed, the average computational complexity of the CNN block to be processed, and the number of operation layers contained in the CNN block to be processed are taken as input, and the corresponding edge server e I Calculate the actual block inference delay generated by the CNN block to be processed as output, and build and train a block inference delay prediction model; the operation layer includes a convolution layer, a fully connected layer, and a pooling layer;

[0065] The block reasoning delay prediction model is as follows:

[0066] Where, FS I Is the server e I Its own floating-point computing capabilities, It is the CNN block e that it is processing I _CNN j The computational complexity of _pt, is the CNN block to be processed j The average computational complexity of CNN is m. j The number of operation layers included, eI _tmd j _p is the edge server e I Computing CNN j The resulting theoretical block inference latency output.

[0067] S3. Using the known floating-point computing capability of the edge server, the computational complexity of the CNN block to be processed, the average computational complexity of the CNN block being processed, and the number of operation layers contained in the CNN block being processed as input, and using the actual delay loss caused by the CNN block to be processed to the CNN block being processed calculated by the corresponding edge server as output, a delay loss prediction model is constructed and trained; the delay loss prediction model is as follows:

[0068] Where, FS I is the floating point computing capability of the edge server, is the CNN block to be processed j The computational complexity of is the CNN block being processed The average computational complexity of m_pt is the number of operation layers contained in the CNN block being processed, e I _t loss _p is the edge server e I Computing CNN j For the e being processed I _CNN j The theoretical delay loss caused by _pt.

[0069] S4. Based on the physical terminal, the initial image data is processed into tensor data with the same resolution and data size, and the tensor data is sent to the edge server in the same local area network. The edge server collects the network bandwidth between itself and other edge servers in the local area network and calculates the theoretical data transmission delay according to the following formula:

[0070] t IJ =Td q-1 / bw IJ ,

[0071] In the formula, Td q-1 is tensor data, bw IJ is the network bandwidth between edge servers.

[0072] S5. Input the actual floating-point computing capability of the edge server, the computational complexity of the CNN block being processed, the average computational complexity of the CNN block to be processed, and the number of layers contained in the CNN block to be processed into the block inference delay prediction model obtained in step S2 to calculate the theoretical block inference delay generated by the edge server computing the CNN block to be processed.

[0073] The actual floating-point computing capability of the edge server, the computational complexity of the CNN block to be processed, the average computational complexity of the CNN block being processed, and the number of operation layers contained in the CNN block being processed are input into the delay loss prediction model obtained in step S3 to calculate the theoretical delay loss caused by the edge server calculating the CNN block to be processed on the CNN block being processed.

[0074] S6. Using the theoretical data transmission delay obtained in S4, the theoretical block inference delay generated by the edge server calculating the CNN block to be processed obtained in step S5, and the theoretical delay loss generated by the edge server calculating the CNN block to the CNN block being processed as input, and the edge server theoretical global delay as output, a global delay prediction model is constructed; the global delay prediction model is as follows:

[0075] S7. Calculate the theoretical global delay of each edge server using the global delay prediction model;

[0076] S8. Determine whether the theoretical global delay of the current edge server is less than or equal to the theoretical global delay of other edge servers. If so, the current edge server calculates the CNN block to be processed, and then executes step S9; otherwise, the CNN block to be processed is transmitted to other edge servers for calculation. The selected other edge servers complete the calculation of the CNN block to be processed, and then execute step S9 to complete the decision on the next CNN block.

[0077] S9. Determine whether all calculations of the CNN block are completed. If so, the edge server sends the calculation results of the CNN block to the physical terminal that issued the task request. Otherwise, return to step S7 to complete the decision on the next CNN block.

[0078] On the other hand, the present invention proposes a cloud-edge-end collaborative CNN inference system for global latency optimization, comprising: a cloud server c, at least two edge servers within the communication range of the cloud server c, the edge servers being deployed on a WIFI access point or base station, and at least one physical terminal being provided within the communication range of each edge server; for any edge server e within the communication range of the cloud server c, I , edge server e I The other o edge servers within the communication range whose physical distance is less than the preset distance x are recorded as: e1, e2, ...e J ,…e o , 1≤J≤o, o>1, and these o edge servers are also within the communication range of cloud server c, and these o edge servers are the same as edge server e I Together they form an edge cluster; the cloud server includes: a convolutional neural network, a model training center, and a communication module;

[0079] The edge server includes: a situation awareness center, a convolutional neural network, a strategy generation center, and a communication module;

[0080] The physical terminal includes: a communication module;

[0081] The model training center is used to train convolutional neural networks, inference delay prediction models, and delay loss prediction models;

[0082] The convolutional neural network is used for a CNN that has been fully trained and serves intelligent applications and is divided into n CNN blocks. j N-layer CNN;

[0083] The communication module is used to send and receive data between the cloud server, edge server and physical terminal;

[0084] The situation awareness center includes a workload awareness module and a network telemetry module; the workload awareness module is used to collect the floating point computing capacity FS of the edge server itself. I , the CNN block that is responsible for reasoning I _CNN j The computational complexity of _pt And the average computational complexity Other O edge servers e J Floating point computing capability FS I , other o edge servers e J The CNN block responsible for inference J _CNN j The computational complexity of _pt And the average computational complexity

[0085] The network telemetry module is used to collect edge server e I and edge servers J The network bandwidth bw IJ ; Referring to Figure 3, the strategy generation center includes a block reasoning delay prediction module, a transmission delay calculation module, a global delay prediction module, an offline sample data storage module and a decision information generation module;

[0086] The block reasoning delay prediction module is used to predict the time delay of the edge server using the known edge server e I FS floating point computing capability I , the computational complexity of the CNN block being processed, the average computational complexity of the CNN block to be processed, and the number of layers contained in the CNN block to be processed are taken as input, and the corresponding edge server e ICalculate the actual block inference latency generated by the CNN block to be processed as output, and build and train a block inference latency prediction model;

[0087] The transmission delay calculation module is used to process the image data into tensor data with the same resolution and data size based on the physical terminal, and send the tensor data to the edge server in the same local area network; the edge server collects the network bandwidth between itself and other edge servers in the local area network and calculates the theoretical data transmission delay;

[0088] The global delay prediction module is used to construct a global delay prediction model with the theoretical data transmission delay, the theoretical block inference delay generated by the edge server calculating the CNN block to be processed, and the theoretical delay loss generated by the edge server calculating the CNN block to the CNN block being processed as input, and the edge server theoretical global delay as output;

[0089] The offline sample storage module is used to collect and store edge server e I The floating point computing capability is FS I , the CNN block e that it is processing I _CNN j The computational complexity of _pt is When the number of computational layers is m and the average computational complexity is CNN block CNN j The actual block inference delay e I _tmd j ; Collection and storage edge server e I The floating point computing capability is FS I , the number of operation layers being processed is m_pt and the average computational complexity is CNN block I _CNN j _pt, the processing computation complexity is CNN block CNN j For the e being calculated I _CNN j The actual delay loss t caused by _pt loss ;

[0090] The decision information generation module is used to determine whether the theoretical global delay of the current edge server is less than or equal to the theoretical global delay of other edge servers. If so, the current edge server calculates the CNN block to be processed, and then determines whether all the CNN blocks are calculated. If so, the edge server sends the calculation result of the CNN block to the physical terminal that issued the task request; otherwise, the CNN block to be processed is transmitted to other edge servers for calculation until all the CNN blocks are calculated.

[0091] While the present invention has been described above with reference to preferred embodiments, this is not intended to limit the present invention. Persons skilled in the art will readily appreciate that various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A cloud-edge-device collaborative CNN inference method for global latency optimization, characterized by: The steps include: S1. Based on the trained and N-layer CNN model, the N-layer CNN model is divided into n blocks: CNN1, CNN2, ...CNN j , …CNN n , 1≤j≤n, n>1; S2, offline learning phase: using the known edge server e I FS floating point computing capability I , the computational complexity of the CNN block being processed, the average computational complexity of the CNN block to be processed, and the number of operation layers contained in the CNN block to be processed are taken as input, and the corresponding edge server e I Calculate the actual block inference delay generated by the CNN block to be processed as output, build and train a block inference delay prediction model; the operation layer includes a convolution layer, a fully connected layer and a pooling layer; S3, taking the known floating-point computing capability of the edge server itself, the computational complexity of the CNN block to be processed, the average computational complexity of the CNN block being processed, and the number of operation layers contained in the CNN block being processed as input, and using the corresponding edge server to calculate the actual delay loss caused by the CNN block to be processed to the CNN block being processed as output, to build and train a delay loss prediction model; S4, based on the physical terminal, processing the initial image data into tensor data with the same resolution and data size, and sending the tensor data to the edge server in the same local area network; The edge server collects the network bandwidth between itself and other edge servers in the local area network and calculates the theoretical data transmission delay; S5, input the actual floating-point computing capability of the edge server itself, the computational complexity of the CNN block being processed, the average computational complexity of the CNN block to be processed, and the number of layers contained in the CNN block to be processed into the block reasoning delay prediction model obtained in step S2, and calculate the theoretical block reasoning delay generated by the edge server calculating the CNN block to be processed; The actual floating-point computing capability of the edge server itself, the computational complexity of the CNN block to be processed, the average computational complexity of the CNN block being processed, and the number of operation layers contained in the CNN block being processed are input into the delay loss prediction model obtained in step S3, and the theoretical delay loss caused by the edge server calculating the CNN block to be processed to the CNN block being processed is calculated; S6, using the theoretical data transmission delay obtained in S4, the theoretical block reasoning delay generated by the edge server calculating the CNN block to be processed obtained in step S5, the theoretical delay loss generated by the edge server calculating the CNN block to the CNN block being processed as input, and the theoretical global delay of the edge server as output, to build a global delay prediction model; S7. Calculate the theoretical global delay of each edge server using the global delay prediction model; S8, determining whether the theoretical global delay of the current edge server is less than or equal to the theoretical global delay of other edge servers, if yes, the current edge server calculates the CNN block to be processed, and then executes step S9; Otherwise, the CNN block to be processed is transmitted to other edge servers for calculation, and the selected other edge servers complete the calculation of the CNN block to be processed, and then step S9 is executed to complete the decision on the next CNN block; S9. Determine whether all calculations of the CNN block are completed. If yes, the edge server sends the calculation results of the CNN block to the physical terminal that issued the task request. Otherwise, return to execute step S7 to complete the decision on the next CNN block.

2. The cloud-edge-end collaborative CNN inference method for global latency optimization according to claim 1 is characterized in that: In step S2, the block reasoning delay prediction model is as follows: Where, FS I Is the edge server e I Its own floating point computing capabilities, It is the CNN block that it is processing. I _CNN j The computational complexity of _pt is is the CNN block to be processed j The average computational complexity of CNN is j The number of operation layers included, e I _tmd j _p is the edge server e I Computing CNN j The resulting theoretical block inference latency output.

3. The cloud-edge-end collaborative CNN inference method for global latency optimization according to claim 2 is characterized in that: In step S3, the delay loss prediction model is as follows: Where, FS I is the floating point computing capability of the edge server, is the CNN block to be processed j The computational complexity of is the CNN block being processed I _CNN j _pt is the average computational complexity of the CNN block being processed, m_pt is the number of operation layers contained in the CNN block being processed, and e I _t loss _p is the edge server e I Computing CNN j For the e being processed I _CNN j The theoretical delay loss caused by _pt.

4. The cloud-edge-end collaborative CNN inference method for global latency optimization according to claim 3 is characterized in that: In step S4, the theoretical data transmission delay is calculated according to the following formula: IJ =Td q-1 / bw IJ , In the formula, Td q-1 is the tensor data, bw IJ is the network bandwidth between edge servers.

5. The cloud-edge-end collaborative CNN inference method for global latency optimization according to claim 4 is characterized in that: In step S6, the global delay prediction model is as follows:

6. The cloud-edge-end collaborative CNN inference method for global latency optimization according to claim 1, characterized in that: In step S1, CNN j Contains at least two consecutive tensor data. For any CNN containing m tensor data j In terms of tensor data and operation layers, it contains: Td j-1 , Td j-2 ,…Td j-m , Ly j-1 , Ly j-2 ,…Ly j-m , m>1, where Td j-1 The computational complexity is greater than that of CNN j The computational complexity of any other tensor data in ; for any two adjacent CNN blocks, that is, CNN p and CNN q :CNN1,…CNN p , CNN q , …CNN n , 1≤p<q≤n, qp=1, CNN p The last tensor data and CNN in q The first tensor data in is the same.

7. The cloud-edge-end collaborative CNN inference method for global latency optimization according to claim 1, characterized in that: In step S4, the initial image data is processed into tensor data with the same resolution and data size based on the physical terminal, and the tensor data is sent to the edge server in the same local area network. Specifically, the initial image data is processed into image feature data with the same data size and resolution, and the image feature data is used as the tensor data Td 1-1 Input CNN1, Td 1-1 After the operation layer calculation in CNN1, tensor data Td is generated 2-1 And pass it to CNN2, that is: in CNN j In the tensor data Td j-1 After the operation layer Ly j-1 , Ly j-2 ,…Ly j-m After calculation, tensor data Td is generated j-m And pass it to the next CNN block, CNN block CNN n The generation of the last tensor data in marks the end of CNN reasoning.

8. The cloud-edge-end collaborative CNN inference method for global latency optimization according to claim 1, characterized in that: CNN BlockCNN j The total computational complexity of all operation layers in is: Denoted as CNN j The computational complexity of CNN j The average computational complexity is recorded as: Computing CNN j The block inference delay generated by all the operation layers included is tmd j , For any operating layer Ly i The computational complexity of .

9. The cloud-edge-end collaborative CNN inference method for global latency optimization according to claim 8, characterized in that: CNN each operation layer Ly i The computational complexity is: In the formula, the operating layer Ly i is the convolutional layer, and its computational complexity is Operation layer Ly i is a fully connected layer, and its computational complexity is H, W, C in Represents the input operation layer Ly i Tensor data Td i The height, width, and number of channels of K represent the input operation layer Ly i The side length of the convolution kernel, C out Represents the output operation layer Ly i Tensor data Td i+1 The number of channels, I and Q represent the input operation layer Ly i Tensor data Td i The dimension of the output operation layer Ly i Tensor data Td i+1 The dimension of .

10. A cloud-edge-device collaborative CNN inference system for global latency optimization, characterized by: include: Cloud server c, at least two edge servers are arranged within the communication range of cloud server c, the edge servers are deployed on WIFI access points or base stations, and at least one physical terminal is arranged within the communication range of each edge server; for any edge server e within the communication range of cloud server c I , edge server I The other o edge servers within the communication range whose physical distance is less than the preset distance x are recorded as: e1, e2, ...e J , …e o , 1≤J≤o, o>1, and these O edge servers are also within the communication range of cloud server c. These o edge servers are the same as edge server e I Together they form edge clusters; The cloud server includes: a convolutional neural network, a model training center, and a communication module; The edge server includes: a situation awareness center, a convolutional neural network, a strategy generation center, and a communication module; The physical terminal includes: a communication module; The model training center is used to train convolutional neural networks, inference delay prediction models and delay loss prediction models; The convolutional neural network is used for a fully trained CNN block that serves intelligent applications and is divided into n blocks. CNN j n-layer CNN; The communication module is used for sending and receiving data between the cloud server, the edge server and the physical terminal; The situation awareness center includes a workload awareness module and a network telemetry module; the workload awareness module is used to collect the floating point computing capacity FS of the edge server itself. I , the CNN block that is responsible for reasoning I _CNN j The computational complexity of _pt And the average computational complexity Other o edge servers e J Floating point computing capability FS J , other o edge servers e J The CNN block responsible for inference J _CNN j The computational complexity of _pt And the average computational complexity The network telemetry module is used to collect edge server e I and edge servers J The network bandwidth between IJ ; The strategy generation center includes a block reasoning delay prediction module, a transmission delay calculation module, a global delay prediction module, an offline sample data storage module and a decision information generation module; The block reasoning delay prediction module is used to predict the time delay of the edge server based on the known edge server e I FS floating point computing capability I , the computational complexity of the CNN block being processed, the average computational complexity of the CNN block to be processed, and the number of layers contained in the CNN block to be processed are taken as input, and the corresponding edge server e I Calculate the actual block inference latency generated by the CNN block to be processed as output, build and train the block inference latency prediction model; The transmission delay calculation module is used to process the image data into tensor data with the same resolution and data size based on the physical terminal, and send the tensor data to the edge server in the same local area network; the edge server collects the network bandwidth between it and other edge servers in the local area network, and calculates the theoretical data transmission delay; The global delay prediction module is used to construct a global delay prediction model with theoretical data transmission delay, theoretical block reasoning delay generated by edge server calculation of CNN block to be processed, theoretical delay loss generated by edge server calculation of CNN block to CNN block being processed as input, and theoretical global delay of edge server as output; The offline sample storage module is used to collect and store edge server samples. I The floating point computing capability is FS I , the CNN block it is processing I _CNN j The computational complexity of _pt is When , the number of computational layers is m and the average computational complexity is CNN BlocksCNN j Actual block inference latency e I _tmd j ; Collection and storage edge server I The floating point computing capability is FS I , the number of operation layers being processed is m_pt and the average computational complexity is CNN blocks I _CNN j _pt, the processing calculation complexity is CNN BlocksCNN j For the e being calculated I _CNN j The actual delay loss t caused by _pt loss ; The decision information generation module is used to determine whether the theoretical global delay of the current edge server is less than or equal to the theoretical global delay of other edge servers. If so, the current edge server calculates the CNN block to be processed, and then determines whether all the CNN blocks are calculated. If so, the edge server sends the calculation result of the CNN block to the physical terminal that issued the task request; otherwise, the CNN block to be processed is transmitted to other edge servers for calculation until all the CNN blocks are calculated.

Images