A rapid response method for latency-sensitive intelligent services based on cloud-edge collaboration

Abstract

This invention discloses a rapid response method for latency-sensitive intelligent services based on cloud-edge collaboration. Based on a dual-latency deep deterministic strategy algorithm, it achieves joint optimization of convolutional kernel segmentation and server resource allocation. The core idea is to partition the computing resources in each edge server, with resources in each region used only for inference of a specific part of a convolutional layer in a convolutional neural network. All intelligent services based on the same type of convolutional neural network in a scenario are placed in a queue, and the inference order of different services is determined by priority. In the convolutional neural network corresponding to each intelligent service, each convolutional layer is divided into multiple parts, enabling parallel inference of the convolutional neural network on different servers. The entire algorithm features reasonable convolutional kernel segmentation, accurate resource allocation, and priority processing of high-priority service requests, meeting the rapid response requirements of latency-sensitive intelligent services in high-task-load scenarios.

Description

Technical Field

[0001] This invention relates to the field of edge computing technology, and more specifically to a fast response method for latency-sensitive intelligent services based on cloud-edge collaboration. Background Technology

[0002] Deep learning (DL), as one of the most important deep neural networks (DNNs), and convolutional neural networks (CNNs) are widely used in many vision applications, such as augmented reality, object classification, and object recognition. While CNNs can help these intelligent applications produce accurate results, their high computational complexity makes it difficult to deploy models for these applications on edge devices, including mobile phones, tablets, and laptops, as well as edge servers. The traditional solution is to infer CNN models on cloud servers; however, considering the large distance between the edge and the cloud, data transmission latency in cloud computing is extremely high.

[0003] Existing technologies address this issue through model partitioning, such as edge-cloud collaborative methods using the maximum cut minimum flow algorithm to design dynamically adaptive model partitioning methods, and methods based on joint parameter quantization. However, in these existing technologies, different parts within the CNN still involve sequential inference, thus failing to minimize the response time of intelligent applications. Therefore, some existing technologies have proposed a novel approach called model parallelism, which groups the channels of each convolutional kernel in the CNN and performs parallel inference on different devices to achieve model parallelism. However, existing model parallelism methods lead to non-negligible accuracy loss or additional training costs. Furthermore, with the development of edge computing and the widespread adoption of intelligent applications, more and more tasks are offloading from terminal devices to edge servers or the cloud, placing a huge computational load on these devices (especially edge devices). Limited computing resources and a large number of tasks result in excessively long response times for user requests, particularly failing to meet the Quality of Service (QoS) requirements for latency-sensitive tasks such as autonomous driving and road monitoring. Therefore, in scenarios with extremely high server loads, there is an urgent need to propose a new inference acceleration method for latency-sensitive services. Summary of the Invention

[0004] The purpose of this invention is to provide a fast response method for latency-sensitive intelligent services based on cloud-edge collaboration. This method sets priorities for each task in the scenario, and for each CNN, it divides each convolutional layer into multiple parts along the height and deploys them on multiple edge servers to perform parallel inference using a certain amount of resources.

[0005] To achieve the above functions, this invention designs a fast response method for latency-sensitive intelligent services based on cloud-edge collaboration. In a user-dense edge-cloud collaborative system composed of intelligent service users, edge servers and controllers, and cloud servers, the edge and cloud devices cooperate with each other. For intelligent service requests of different priorities in the scenario, the edge-cloud collaborative system executes steps A-E to obtain a queue of intelligent service requests to be processed and a CNN distributed inference method, and responds according to the priority of the intelligent service requests.

[0006] Step A: For the target scenario, select the corresponding convolutional neural network category based on the intelligent service requests sent by each intelligent service user, obtain the convolutional layer segmentation strategy according to numerical optimization theory, segment each convolutional kernel of the convolutional neural network along the height, and assign the different parts of the segmented convolutional layer to different edge servers in the scenario for parallel inference.

[0007] Step B: For all intelligent service requests based on the same type of convolutional neural network in this scenario, establish a queue and determine their priority according to the sensitivity of each intelligent service request in the same queue to response latency, so as to determine their running order;

[0008] Step C: Based on the computational cost of each type of convolutional neural network in the scenario, partition the computing resources in each edge server, and the computing resources in each region can only be used for inference of a convolutional neural network of a certain corresponding type.

[0009] Step D: For each edge server, the computing resources in each resource partition divided in Step C are further divided using a deep reinforcement learning model to form multiple sub-regions. The computing resources in each sub-region are used only for inference of a specific convolutional layer in a convolutional neural network of a specific category.

[0010] Step E: Run the smart service requests in each queue sequentially until all smart service requests in the queue have been processed.

[0011] As a preferred technical solution of the present invention: the convolution kernel segmentation strategy in step A is executed as follows: steps A1 to A4, to obtain the parallel inference strategy of each convolution kernel corresponding to each edge server;

[0012] Step A1: The target scene has N server There are N edge servers, and for a specific category of convolutional neural network, there are a total of N. conEach convolutional layer is divided and inferred in parallel on each edge server; all pooling layers are inferred on the controller; all fully connected layers are inferred on the cloud server; for each convolutional layer, the inference results of each edge server are sent to the controller for aggregation, and the aggregation results are returned to each edge server for the next convolutional layer to infer in parallel, or the aggregation results are sent to the cloud server for the fully connected layer to infer.

[0013] Step A2: The response latency for the intelligent service request issued by the intelligent service user is as follows:

[0014]

[0015] In the formula, This indicates the segmentation ratio of the nth convolutional layer to the mth edge server, float. n This represents the floating-point operations of the nth convolutional layer. γ represents the amount of computational resources allocated to the nth convolutional layer in the mth edge server. m N represents the computing speed per unit of computing resources in the m-th edge server. con Indicates the total number of convolutional layers;

[0016] Step A3: During the parallel inference process of each convolutional layer, the inference latency of each edge server is exactly the same. The convolutional kernel segmentation problem is transformed into the following formula:

[0017]

[0018]

[0019] This leads to the convolutional layer segmentation strategy, namely in j∈{1,2,…,N server}, and N server This represents the total number of edge servers.

[0020] Step A4: For each edge server, perform steps A4.1-A4.1:

[0021] Step A4.1: Denote the kernel size allocated to the server as calculated in step A3 as... Then to Round down to get And calculate The extra decimal part; the corresponding decimal part for each edge server Add the extra decimal parts together and then add them to the total.

[0022] Step A4.2: Iterate through the inference latency corresponding to each edge server at this time. And find the minimum value among them. The size of the convolution kernel allocated to that server is denoted as...

[0023] Step A4.3: Execute and

[0024] Step A4.4: Repeat steps A4.2-A4.3 until each edge server... The maximum value in is not less than the value before the change.

[0025] Step A4.5: For each of the remaining servers Perform the above operation once for each case to obtain the convolutional layer segmentation strategy.

[0026] As a preferred technical solution of the present invention: In step B, the intelligent service requests sent by each intelligent service user are classified according to the type of convolutional neural network they are based on. A task queue is established for a type of convolutional neural network, and each intelligent service request based on the same type of convolutional neural network is placed into the corresponding task queue. The intelligent service requests are sorted in the task queue according to their priority. The priority of each intelligent service request is preset by the service provider.

[0027] As a preferred technical solution of the present invention: in step C, the computing resources in each edge server are partitioned proportionally according to the computational amount of all categories of convolutional neural networks in the scene, and the computing resources in each region are used only for inference of one category of convolutional neural network.

[0028] As a preferred technical solution of the present invention: In step D, the dual-delay deep deterministic strategy algorithm in the deep reinforcement learning model is adopted to execute the following steps D1 to D3, dividing the computing resource sub-regions in each server and allocating computing resources to each convolutional layer in each type of convolutional neural network in the scene.

[0029] Step D1: Using the dual-delay deep deterministic policy algorithm in the deep reinforcement learning model, the computational resource sub-region is divided, and inference is performed on the convolutional neural network for a specific category. The state space and action space are set as follows:

[0030]

[0031]

[0032] Among them, a i For the action space, s iFor state space; Assigning resource allocation ratios; C represents the amount of computing resources in a resource region that have not yet been allocated to subregions. i This represents the number of convolutional layers in the convolutional neural network that have not yet been inferred.

[0033] Step D2: Set the reward function. n The deep reinforcement learning model is trained as follows:

[0034]

[0035] In the formula, This represents the interaction latency between the m-th edge server and the controller;

[0036] Step D3: The trained deep reinforcement learning model is stored in a controller. During the actual operation of intelligent service requests, the deep reinforcement learning model dynamically adjusts the computing resource allocation strategy based on network bandwidth fluctuations, with the goal of reducing the total inference latency of the convolutional neural network.

[0037] As a preferred technical solution of the present invention: in step E, the intelligent service requests in each queue are run sequentially according to priority. During the running of each intelligent service request, each convolutional layer in its corresponding convolutional neural network performs parallel inference in different resource sub-regions. For the same type of convolutional neural network model, the inference process of different convolutional layers of different intelligent service requests is independent of each other.

[0038] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0039] This invention proposes a rapid response method for latency-sensitive intelligent services based on edge-cloud collaboration. It presents a distributed inference method using convolutional neural networks (CNNs), which combines mathematical optimization with the dual-delay deep deterministic strategy (TD3) algorithm from deep reinforcement learning to jointly optimize convolutional kernel segmentation and server resource allocation. The core idea is to partition the computing resources of each edge server, with resources in each region used only for inference of a specific part of a convolutional layer within a specific CNN. All intelligent services based on the same type of CNN in a scenario are placed in a queue, and the inference order of different services is determined by priority. Within the CNN corresponding to each intelligent service, each convolutional layer is divided into multiple parts, enabling parallel inference of the CNN across different servers. The entire algorithm features reasonable convolutional kernel segmentation, precise resource allocation, and priority processing of high-priority service requests, meeting the rapid response requirements of latency-sensitive intelligent services in high-load scenarios. It should be noted that the novel parallel inference method for convolutional layers in this invention, compared with traditional model segmentation and model parallelism, further reduces the inference latency of CNN while ensuring the accuracy of the model inference results. In addition, the reasonable allocation of computing resources in edge servers in this invention enables full and efficient utilization of computing resources in all edge servers in the scenario. Attached Figure Description

[0040] Figure 1 This is a flowchart of a fast response method for latency-sensitive intelligent services based on cloud-edge collaboration according to an embodiment of the present invention;

[0041] Figure 2 This is a diagram of a convolution kernel parallel inference framework provided according to an embodiment of the present invention;

[0042] Figure 3 This is a distributed reasoning scenario diagram of the intelligent service model provided in the embodiments of the present invention;

[0043] Figure 4 This is a framework diagram of the queue and multitasking alternation mechanism provided in an embodiment of the present invention. Detailed Implementation

[0044] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0045] This invention proposes a fast response method for latency-sensitive intelligent services based on edge-cloud collaboration, namely a distributed inference method for convolutional neural networks (CNNs) in multi-user scenarios. This method utilizes mathematical optimization techniques and the dual-latency deep deterministic strategy (TD3) algorithm in deep reinforcement learning to jointly optimize convolutional kernel segmentation and server resource allocation. Compared to other model segmentation and parallel model methods, this invention further reduces CNN inference latency while ensuring the accuracy of the model inference results. Simultaneously, the rational allocation of computing resources on edge servers ensures full and efficient utilization of computing resources within all edge servers in the scenario. Furthermore, considering the different priorities of different intelligent services in the scenario, this invention designs a queue and multi-task alternation mechanism, placing all intelligent services based on the same type of CNN in the same queue and determining their execution order according to their different priorities; and within the same queue, different convolutional layers of different services in the CNNs corresponding to preceding and following services can be inferred simultaneously. This minimizes the response time of latency-sensitive intelligent service requests in high-task-load scenarios.

[0046] Reference Figure 1 The fast response method for latency-sensitive intelligent services based on cloud-edge collaboration provided in this invention is based on a user-dense edge-cloud collaborative system composed of intelligent service users, edge servers and controllers, and cloud servers. The edge and cloud devices collaborate with each other, and for intelligent service requests of different priorities in the scenario, the edge-cloud collaborative system executes steps A-E to obtain a queue of intelligent service requests to be processed and a CNN distributed inference method, and responds according to the priority of the intelligent service requests.

[0047] Step A: For the target scenario, select the corresponding convolutional neural network category based on the intelligent service requests sent by each intelligent service user, obtain the convolutional layer segmentation strategy according to numerical optimization theory, segment each convolutional kernel of the convolutional neural network along the height, and assign the different parts of the segmented convolutional layer to different edge servers in the scenario for parallel inference.

[0048] In step A, the convolution kernel segmentation strategy is executed as follows: steps A1 to A4, to obtain the parallel inference strategy for each convolution kernel corresponding to each edge server.

[0049] Reference Figure 2 For each convolutional kernel, it is first segmented along its height into multiple distinct parts. Each part is then deployed on different edge servers for parallel inference. Finally, the inference results from each edge server are aggregated on the controller. Furthermore, the segmentation of the convolutional kernel necessitates the segmentation of the corresponding input data. c with w c x represents the height and width of the convolution kernel, respectively. cWith y c These represent the start and end points of the input segmentation by the convolution kernel, respectively; thus, the start and end points (x) of the input segmentation corresponding to this convolution kernel can be calculated. input Indicates the starting point, y input (Indicates the end point):

[0050] x input =x c

[0051] y input =h c -(w c -y c )

[0052] Therefore, it is calculated that for each convolution kernel, the size of the inference result of each edge server is exactly equal to the size of the result obtained by normal inference without segmentation of the convolution kernel; and the process of the controller aggregating the inference results of each edge server is simply the sum of the inference results of each server.

[0053] Step A1: Refer to Figure 3 Suppose there are N users in a specific user-intensive load scenario. server There are N available edge servers. For a specific category of convolutional neural networks, there are a total of N. con There are several convolutional layers. The convolutional layers are segmented and inferred in parallel on multiple edge servers. Since pooling layers have very low computational cost, all pooling layers are inferred directly on the controller. Because fully connected layers have massive computational cost and are difficult to segment, all fully connected layers are inferred on cloud servers. For each convolutional layer, the inference results from each server are sent to the controller for aggregation, and the aggregation results are returned to the edge servers for parallel inference of the next convolutional layer, or sent to the cloud server for inference of the fully connected layer. The computational cost of a neural network layer is measured by floating-point operations; here, the floating-point operations of the nth convolutional layer in a convolutional neural network are denoted as `float`. n The convolutional layer corresponds to the segmentation ratio of the m-th edge server as follows: The amount of computing resources allocated to this convolutional layer in the m-th edge server is: The computing speed per unit of computing resources in the m-th edge server is γ. m The inference time of the fully connected layer on the cloud server is denoted as t. full The data transmission delay between the cloud and the edge is denoted as For the inference result of the nth convolutional layer, the interaction latency between the mth edge server and the controller is denoted as . This leads to the following formula for the response latency of a certain intelligent service:

[0054]

[0055] The aggregation latency of the convolutional layer parallel inference results in the controller and the latency of the pooling operation are negligible.

[0056] Step A2: The focus of this invention is on parallel inference of convolutional layers, therefore the inference time t of the fully connected layer on the cloud server... full Data transmission latency between cloud and edge This can be removed from the formula. Furthermore, to simplify the problem, the data interaction latency between the edge server and the controller is temporarily ignored during the convolutional layer segmentation stage. Therefore, the simplified formula is obtained:

[0057]

[0058] In the formula, This indicates the segmentation ratio of the nth convolutional layer to the mth edge server, float. n This represents the floating-point operations of the nth convolutional layer. γ represents the amount of computational resources allocated to the nth convolutional layer in the mth edge server. m N represents the computing speed per unit of computing resources in the m-th edge server. con Indicates the total number of convolutional layers;

[0059] Step A3: During the parallel inference process of each convolutional layer, the inference latency of each edge server is exactly the same. The convolutional kernel segmentation problem is transformed into the following formula:

[0060]

[0061]

[0062] This leads to the convolutional layer segmentation strategy, namely in j∈{1,2,…,N server}, and N server This represents the total number of edge servers.

[0063] Step A4: For each edge server, perform steps A4.1-A4.1:

[0064] Step A4.1: Considering that the height of the partial convolutional kernel allocated to each edge after segmentation should actually be an integer, the above method needs further optimization. The convolutional kernel size allocated to the server, calculated in step A3, is denoted as... Then to Round down to get And calculate The extra decimal part; the corresponding decimal part for each edge server Add the extra decimal parts together and then add them to the total.

[0065] Step A4.2: Iterate through the inference latency corresponding to each edge server at this time. And find the minimum value among them. The size of the convolution kernel allocated to that server is denoted as...

[0066] Step A4.3: Execute and

[0067] Step A4.4: Repeat steps A4.2-A4.3 until each edge server... The maximum value in is not less than the value before the change.

[0068] Step A4.5: For each of the remaining servers Perform the above operation once for each case to obtain the convolutional layer segmentation strategy.

[0069] Step B: For all intelligent service requests based on the same type of convolutional neural network in this scenario, establish a queue and determine their priority according to the sensitivity of each intelligent service request in the same queue to response latency, so as to determine their running order;

[0070] In step B, the intelligent service requests sent by each intelligent service user are classified according to the type of convolutional neural network they are based on. A task queue is established for each type of convolutional neural network, and each intelligent service request based on the same type of convolutional neural network is placed into the corresponding task queue. The intelligent service requests are sorted in the task queue according to their priority. The priority of each intelligent service request is preset by the service provider.

[0071] Step C: Based on the computational cost of each type of convolutional neural network in the scenario, partition the computing resources in each edge server, and the computing resources in each region can only be used for inference of a convolutional neural network of a certain corresponding type.

[0072] In step C, the computing resources in each edge server are partitioned proportionally according to the computational cost of all categories of convolutional neural networks in the scene, and the computing resources in each region are used only for inference of one category of convolutional neural network.

[0073] Step D: For each edge server, the computing resources in each resource partition divided in Step C are further divided using a deep reinforcement learning model to form multiple sub-regions. The computing resources in each sub-region are used only for inference of a specific convolutional layer in a convolutional neural network of a specific category.

[0074] In step D, the dual-delay deep deterministic strategy algorithm in the deep reinforcement learning model is adopted to execute the following steps D1 to D3: divide the computing resource sub-regions within each server and allocate computing resources to each convolutional layer in each type of convolutional neural network in the scene.

[0075] The Dual-Delay Deep Deterministic Policy Gradient (TD3) algorithm is an improvement upon the Deep Deterministic Policy Gradient (DDPG) algorithm. It adds a Critic network and a target Critic network to the DDPG algorithm, and by selecting a smaller Q-value, it avoids overestimation, thus making a better decision. In the server resource allocation decision-making process of this invention, the amount of computing resources in the resource region that have not yet been allocated to sub-regions at each time step is taken as the state, and the resource allocation ratio (a decimal between 0 and 1) at each time step is taken as the action. It is easy to see that the state at the next time step is determined by the state at the previous time step and the action at the current time step, thus conforming to the Markov property. Deep reinforcement learning can be considered to solve this problem. Considering the continuity of the action space and the large number of discrete states in the state space, combined with the strong convergence of the TD3 algorithm, the TD3 algorithm is ultimately chosen for resource allocation decisions in edge servers.

[0076] Step D1: Using the dual-delay deep deterministic policy algorithm in the deep reinforcement learning model, the computational resource sub-region is divided, and inference is performed on the convolutional neural network for a specific category. The state space and action space are set as follows:

[0077]

[0078]

[0079] Among them, a i For the action space, s i For state space; Assigning resource allocation ratios; C represents the amount of computing resources in a resource region that have not yet been allocated to subregions. i This represents the number of convolutional layers in the convolutional neural network that have not yet been inferred.

[0080] Step D2: Set the reward function. n The deep reinforcement learning model is trained as follows:

[0081]

[0082] In the formula, This represents the interaction latency between the m-th edge server and the controller;

[0083] The reward function takes into account the inference latency of the edge server and the data transmission latency between the server and the controller, with the aim of minimizing the inference latency of the convolutional neural network.

[0084] Step D3: The trained deep reinforcement learning model is stored in a controller. During the actual operation of intelligent service requests, the deep reinforcement learning model dynamically adjusts the computing resource allocation strategy based on network bandwidth fluctuations, with the goal of reducing the total inference latency of the convolutional neural network.

[0085] Step E: Run the smart service requests in each queue sequentially until all smart service requests in the queue have been processed.

[0086] like Figure 4 As shown, in step E, the intelligent services to be processed in each queue are run sequentially according to priority. During the operation of each intelligent service, each convolutional layer in its corresponding convolutional neural network model performs parallel inference in different resource sub-regions. For convolutional neural network models of the same category, the inference processes of different convolutional layers of different intelligent services are independent of each other. Therefore, while the second convolutional layer of the first intelligent service to be processed in the queue is being inferred in parallel, the first convolutional layer of the next intelligent service to be processed in the queue can also be inferred in parallel, and so on, until all intelligent services in the queue have been processed.

[0087] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A rapid response method for latency-sensitive intelligent services based on cloud-edge collaboration, characterized in that, In a user-dense edge-cloud collaborative system comprised of intelligent service users, edge servers and controllers, and cloud servers, edge and cloud devices collaborate with each other. For intelligent service requests of different priorities in the scenario, the edge-cloud collaborative system executes steps A-E to obtain a queue of intelligent service requests to be processed and a CNN distributed inference method, and responds according to the priority of the intelligent service requests: Step A: For the target scenario, select the corresponding convolutional neural network category based on the intelligent service requests sent by each intelligent service user, obtain the convolutional layer segmentation strategy according to numerical optimization theory, segment each convolutional kernel of the convolutional neural network along the height, and assign the different parts of the segmented convolutional layer to different edge servers in the scenario for parallel inference. Step B: For all intelligent service requests based on the same type of convolutional neural network in this scenario, establish a queue and determine their priority according to the sensitivity of each intelligent service request in the same queue to response latency, so as to determine their running order; Step C: Based on the computational cost of each type of convolutional neural network in the scenario, partition the computing resources in each edge server, and the computing resources in each region can only be used for inference of a convolutional neural network of a certain corresponding type. Step D: For each edge server, the computing resources in each resource partition divided in Step C are further divided using a deep reinforcement learning model to form multiple sub-regions. The computing resources in each sub-region are used only for inference of a specific convolutional layer in a convolutional neural network of a specific category. Step E: Run the smart service requests in each queue sequentially until all smart service requests in the queue have been processed.

2. The fast response method for latency-sensitive intelligent services based on cloud-edge collaboration according to claim 1, characterized in that, In step A, the convolution kernel segmentation strategy is executed as follows: steps A1 to A4, to obtain the parallel inference strategy for each convolution kernel corresponding to each edge server. Step A1: The target scene has N server There are N edge servers, and for a specific category of convolutional neural network, there are a total of N. con Each convolutional layer is divided and inferred in parallel on each edge server; all pooling layers are inferred on the controller; all fully connected layers are inferred on the cloud server; for each convolutional layer, the inference results of each edge server are sent to the controller for aggregation, and the aggregation results are returned to each edge server for the next convolutional layer to infer in parallel, or the aggregation results are sent to the cloud server for the fully connected layer to infer. Step A2: The response latency for the intelligent service request issued by the intelligent service user is as follows: In the formula, This indicates the segmentation ratio of the nth convolutional layer to the mth edge server, float. n This represents the floating-point operations of the nth convolutional layer. γ represents the amount of computational resources allocated to the nth convolutional layer in the mth edge server. m N represents the computing speed per unit of computing resources in the m-th edge server. con Indicates the total number of convolutional layers; Step A3: During the parallel inference process of each convolutional layer, the inference latency of each edge server is exactly the same. The convolutional kernel segmentation problem is transformed into the following formula: This leads to the convolutional layer segmentation strategy, namely in j∈{1,2,…,N server }, and N server This represents the total number of edge servers. Step A4: For each edge server, perform steps A4.1-A4.1: Step A4.1: Denote the kernel size allocated to the server as calculated in step A3 as... Then to Round down to get And calculate The extra decimal part; the corresponding decimal part for each edge server Add the extra decimal parts together and then add them to the total. Step A4.2: Iterate through the inference latency corresponding to each edge server at this time. And find the minimum value among them. The size of the convolution kernel allocated to that server is denoted as... Step A4.3: Execute and Step A4.4: Repeat steps A4.2-A4.3 until each edge server... The maximum value in is not less than the value before the change. Step A4.5: For each of the remaining servers Perform the above operation once for each case to obtain the convolutional layer segmentation strategy.

3. The fast response method for latency-sensitive intelligent services based on cloud-edge collaboration according to claim 1, characterized in that, In step B, the intelligent service requests sent by each intelligent service user are classified according to the type of convolutional neural network they are based on. A task queue is established for each type of convolutional neural network, and each intelligent service request based on the same type of convolutional neural network is placed into the corresponding task queue. The intelligent service requests are sorted in the task queue according to their priority. The priority of each intelligent service request is preset by the service provider.

4. The fast response method for latency-sensitive intelligent services based on cloud-edge collaboration according to claim 1, characterized in that, In step C, the computing resources in each edge server are partitioned proportionally according to the computational cost of all categories of convolutional neural networks in the scene, and the computing resources in each region are used only for inference of one category of convolutional neural network.

5. The fast response method for latency-sensitive intelligent services based on cloud-edge collaboration according to claim 1, characterized in that, In step D, the dual-delay deep deterministic strategy algorithm in the deep reinforcement learning model is adopted to execute the following steps D1 to D3: divide the computing resource sub-regions within each server and allocate computing resources to each convolutional layer in each type of convolutional neural network in the scene. Step D1: Using the dual-delay deep deterministic policy algorithm in the deep reinforcement learning model, the computational resource sub-region is divided, and inference is performed on the convolutional neural network for a specific category. The state space and action space are set as follows: Among them, a i For the action space, s i For state space; Assigning resource allocation ratios; C represents the amount of computing resources in a resource region that have not yet been allocated to subregions. i This represents the number of convolutional layers in the convolutional neural network that have not yet been inferred. Step D2: Set the reward function. n The deep reinforcement learning model is trained as follows: In the formula, This represents the interaction latency between the m-th edge server and the controller; Step D3: The trained deep reinforcement learning model is stored in a controller. During the actual operation of intelligent service requests, the deep reinforcement learning model dynamically adjusts the computing resource allocation strategy based on network bandwidth fluctuations, with the goal of reducing the total inference latency of the convolutional neural network.

6. The fast response method for latency-sensitive intelligent services based on cloud-edge collaboration according to claim 1, characterized in that, In step E, the intelligent service requests in each queue are run sequentially according to priority. During the running of each intelligent service request, each convolutional layer in its corresponding convolutional neural network performs parallel inference in different resource sub-regions. For the same type of convolutional neural network model, the inference process of different convolutional layers of different intelligent service requests is independent of each other.

Images