Edge collaborative deep learning model dynamic splitting calculation method and system based on edge computing power load perception

Abstract

The application relates to an edge collaborative deep learning model dynamic splitting calculation method and system based on edge computing load perception, belonging to the technical field of edge computing and deep learning. In view of the problems that a fixed splitting point cannot adapt to dynamic load, model compression precision loss and large switching overhead exist when a deep learning model is deployed on an existing edge device, the application provides the following: model splitting is carried out after a MaxPool layer of a ResNet50, and an edge end only bears 2% of the calculation amount; the utilization rate of the computing power of an edge computing unit is collected in real time through a hardware bottom layer monitoring interface and compared with a threshold value, and automatic millisecond-level switching is carried out between a Full mode and a Split mode; binary serialization and Base64 encoding transmission of intermediate features are adopted in the Split mode; a Non-local attention module is embedded in a cloud model to keep recognition precision. The application realizes adaptive scheduling of local reasoning under low load of an edge end and collaborative unloading under high load, reduces the calculation load and energy consumption of an edge device significantly while ensuring that the precision of pedestrian re-identification and other tasks is lossless, is suitable for resource-limited platforms such as unmanned aerial vehicles and mobile robots, and is compatible with domestic or general edge AI computing power platforms.

Description

Technical Field

[0001] This invention relates to the fields of edge computing and deep learning technology, and in particular to a dynamic splitting calculation method and system for edge collaborative deep learning models based on edge computing load perception, applicable to resource-constrained edge devices (such as drones, mobile robots, etc.) and oriented towards real-time visual reasoning tasks. Background Technology

[0002] With the widespread application of deep learning in computer vision, visual tasks such as person re-identification (Re-ID) and object detection are playing an increasingly important role in smart cities, security monitoring, and drone inspections. However, compared with traditional data center deployments, deploying deep learning models on resource-constrained devices such as drones and edge smart boxes faces several unique challenges. First, modern deep neural networks typically have tens to hundreds of millions of parameters, resulting in high computational complexity. Edge computing units such as GPUs, NPUs, or other AI accelerators in edge devices have limited computing power, making it difficult to run large-scale deep learning models in real time. Second, mobile devices such as drones are extremely sensitive to power consumption; continuous high-load inference significantly shortens battery life, limiting task execution time. Furthermore, the bandwidth and latency of wireless networks between the edge and cloud are uncertain, making it difficult for pure cloud solutions to guarantee real-time performance. Edge devices may run multiple tasks simultaneously, and the utilization rate of edge computing units changes dynamically over time, further increasing the complexity of resource scheduling.

[0003] To address these challenges, several methods have been proposed. Traditional model compression techniques, including pruning, quantization, and knowledge distillation, reduce computational burden by decreasing the number of model parameters. However, compression often results in decreased model accuracy and limited compression ratios, failing to meet the needs of scenarios with extremely low computing power. Pure cloud-based inference solutions send all computational tasks to cloud servers for processing, but rely entirely on network connections, leading to high latency and bandwidth consumption, making them unsuitable for scenarios with high real-time requirements. Some researchers have proposed a fixed-split-point splitting computation method, statically dividing the model into edge and cloud parts for execution. However, this method's fixed split points cannot adapt to dynamically changing edge device loads and network conditions, resulting in insufficient flexibility in practical applications.

[0004] In existing technologies, CN110321586A discloses an inference method based on model splitting and edge-cloud collaboration. Its decision-making mechanism primarily relies on external network bandwidth fluctuations, making it difficult to cope with dynamic changes in internal computing load caused by multi-tasking on edge devices, and the splitting point selection is relatively fixed. In contrast, this application explicitly proposes a splitting point with an extremely low computing power ratio (located after the MaxPool layer) optimized for ResNet50 (Residual Network), allowing the edge to bear only about 2% of the computational load, significantly reducing the device burden. Furthermore, CN111027150A discloses a method for determining the traffic splitting strategy based on prediction inference time, but it relies on complex regression models for estimation, increasing the computational overhead of resource-constrained devices, and does not mention a precision compensation mechanism after splitting. This application, however, directly reads the real-time load status of the edge computing unit through the hardware's underlying monitoring interface, achieving ultra-low-overhead millisecond-level mode switching, and combined with a non-local attention maintenance mechanism, ensures lossless preservation of recognition accuracy in tasks such as pedestrian re-identification.

[0005] Therefore, a deep learning model splitting calculation method is needed that can dynamically adjust the strategy according to the real-time load status of the device, while taking into account both extremely low switching overhead and lossless recognition accuracy. Summary of the Invention

[0006] This invention aims to address the following technical problems existing in current deep learning edge deployment methods: 1. Existing model compression methods suffer significant accuracy loss and cannot achieve efficient inference on resource-constrained edge devices while ensuring recognition accuracy.

[0007] 2. Existing split computing methods use fixed split points, which cannot dynamically switch between local inference and split computing modes according to the real-time computing load of edge devices, resulting in insufficient flexibility.

[0008] 3. The lack of an effective split point selection strategy makes it difficult to achieve an optimal balance between computational and data transfer.

[0009] 4. The efficiency of intermediate feature transmission between the edge and the cloud needs to be improved, and the existing solutions have high latency.

[0010] To this end, this invention proposes a dynamic splitting computation method and system for edge collaborative deep learning models based on edge computing load awareness. Its overall framework includes core substructures such as intelligent model splitting architecture, dynamic offloading decision module, efficient feature transmission protocol and non-local attention preservation mechanism.

[0011] A dynamic splitting computation method for edge collaborative deep learning models based on edge computing load awareness includes sequentially executed steps: model splitting architecture construction, real-time edge computing load awareness and mode decision-making, split mode collaborative inference, full mode local inference, and adaptive loop steps, forming a closed-loop adaptive control process of "perception-decision-execution-re-perception". The specific steps are as follows: S1. Model Splitting Architecture Construction Steps During the inference phase, the pre-trained deep learning model is logically divided at preset shallow segmentation points to generate a lightweight edge model and a computationally intensive cloud model. The edge model is used to perform shallow feature extraction of the input image, and its computational cost accounts for only a preset low proportion of the total computational cost of the complete model; the cloud model is used to receive intermediate features and perform deep inference.

[0012] As a preferred embodiment, this invention uses the ResNet50 network as its backbone, with the preset shallow segmentation point specifically located after the MaxPool layer of the ResNet50 network. The edge model consists of an input layer, a first convolutional layer (Conv1, using a 7×7 convolutional kernel with a stride of 2), a batch normalization layer (BN1), and a max pooling layer (MaxPool, using a 3×3 pooling kernel with a stride of 2) sequentially connected in series. This layer maps the original RGB image (size B×3×256×128) into an intermediate feature map of size B×64×64×32. Actual measurements show that the computational cost of this edge model accounts for only 2% of the total computational cost of the complete ResNet50 model, significantly reducing the computational burden on edge devices. The cloud model consists of all residual blocks from Layer 1 to Layer 4 of the ResNet50 network sequentially connected in series. This cloud model receives intermediate features and performs deep inference, ultimately outputting a 2048-dimensional feature vector for tasks such as pedestrian re-identification.

[0013] S2 Edge Computing Load Real-time Perception and Pattern Decision-Making Steps During the inference phase, edge devices collect real-time computing power utilization values ​​of their local edge computing units through underlying hardware monitoring interfaces. Specifically, they can call utilization query interfaces provided by device drivers, runtime, or system monitoring services to directly read the current load status from the hardware or driver layer, without needing to build additional regression prediction models, achieving millisecond-level awareness with ultra-low overhead.

[0014] The collected computing power utilization rate is compared with the preset dynamic switching threshold (default value is 70%): If the current computing power utilization rate is greater than or equal to the threshold, the edge device is determined to be in a high load state, and the decision is made to execute the Split mode; If the current computing power utilization rate is less than the threshold, the edge device is determined to be in a low-load state, and the decision is made to execute Full mode.

[0015] The decision result is used to trigger different subsequent inference processes. To further enhance adaptability, the dynamic switching threshold supports adaptive dynamic optimization: the system records historical computing power utilization and end-to-end inference latency before and after each mode switch, calculates the median of computing power utilization within the window using a sliding window statistical method, and dynamically updates the threshold to the value constrained by the median and preset upper and lower limits (such as 50%~90%), in order to adapt to the computing power characteristics and task load changes of different edge devices.

[0016] S3. Split Mode Collaborative Reasoning Steps When step S2 above decides to use the Split mode, the following sub-steps are executed sequentially: S3.1 Shallow Inference at the Edge: The edge model feeds the input image into the edge model for forward inference to obtain an intermediate feature tensor. In the ResNet50 instance, this tensor has a size of B×64×64×32 and a data size of approximately 512KB (when B=1).

[0017] S3.2 Feature Serialization and Encoding: The intermediate feature tensor is migrated from the device-side accelerator memory to system memory, converted to NumPy array format, and NumPy's binary serialization function is used to generate a byte stream. Then, Base64 encoding is used to generate a text string. The encoded string and the original shape information of the feature tensor are encapsulated together into a JSON-formatted HTTP request body.

[0018] S3.3 Network Transmission: The edge device sends the request body to the preset inference interface of the cloud server via the HTTP protocol (preferably using the RESTful API architecture style).

[0019] S3.4 Cloud Decoding and Deep Inference: After receiving the request, the cloud server reverse-engineers Base64 decoding, binary deserialization, and NumPy array to PyTorch Tensor format conversion, and then migrates the tensor to the cloud computing device. The recovered feature tensor is input into the cloud model, and deep network inference is performed to obtain the final recognition result feature vector.

[0020] S3.5 Result Return: The cloud encodes the feature vector of the recognition result using the same Base64 encoding and JSON encapsulation format, and returns it to the edge terminal via HTTP response; the edge terminal decodes to obtain the final recognition result.

[0021] S4. Full Mode Local Inference Steps When the edge computing load real-time perception and pattern decision step decides to use Full mode, the edge device directly inputs the input image into the complete deep learning model (i.e., the original model without splitting, containing all convolutional layers, residual blocks, and attention modules), performs all forward inference computation locally, and directly obtains the recognition result feature vector. In this mode, there is no need to communicate with the cloud, avoiding network transmission latency.

[0022] S5. Adaptive Loop Steps After completing one inference in the Split or Full mode, the system returns to the real-time edge computing load perception and mode decision-making step, and repeats the computing utilization acquisition, threshold comparison and mode decision-making process for the next frame or the next batch of input images, thereby realizing the dynamic, closed-loop adaptive switching of the operating mode according to the real-time load status of the edge device without manual intervention.

[0023] To ensure that the recognition accuracy is not affected after model splitting, this invention embeds non-local attention modules into the cloud model. Specifically, two non-local attention modules are embedded in Layer 2 of the ResNet50 cloud model, and three non-local attention modules are embedded in Layer 3. These non-local attention modules are used to capture pixel-level dependencies between distant locations in the feature map, enabling the cloud model to extract global contextual discriminative features even after receiving shallow features. Since model splitting only involves the logical division of the structure and does not change any weight parameters, and the intermediate features are transmitted using lossless binary serialization, the recognition accuracy in Split mode is completely consistent with the recognition accuracy of the full model in Full mode.

[0024] This invention also provides a dynamic splitting computation system for an edge collaborative deep learning model based on edge computing load awareness, used to execute the above method. The system includes: Edge devices (such as drones, mobile robots, or embedded AI computing platforms) are equipped with: a lightweight edge model for performing shallow feature extraction; a computing load monitoring module for real-time acquisition of the computing power utilization of the local edge computing unit through a hardware-level monitoring interface; a mode decision module for comparing computing power utilization with a dynamic threshold and outputting a Split or Full mode selection signal; a feature encoding module for serializing intermediate feature tensors and encoding them into a JSON request body in Split mode; an HTTP communication client for sending requests to the cloud and receiving responses; and a full model inference module for performing local full inference in Full mode.

[0025] The cloud server (a server or cloud host equipped with high-performance AI acceleration resources) is deployed with: an HTTP server for receiving inference requests sent from the edge; a feature decoding module for decoding, deserializing, and migrating the received request body; a computationally intensive cloud model, which embeds a non-local attention module for receiving the decoded features and performing deep inference; and a result encoding module for encoding the inference results and returning them to the edge.

[0026] The edge device is connected to the cloud server via a wireless network, and the output signal of the mode decision module is used to select between the feature encoding module and the HTTP communication client (Split mode) or the full model inference module (Full mode).

[0027] Compared with the prior art, the present invention has the following beneficial effects: 1) By setting a split point after the MaxPool layer in the ResNet50 network, the edge only needs to perform about 2% of the total computation. Tested on a typical embedded edge AI computing platform, the single-frame processing time can be controlled in the millisecond range, the overall power consumption is significantly reduced, and the battery life of mobile devices such as drones is extended.

[0028] 2) Based on a dynamic decision-making mechanism utilizing real-time computing power utilization, the system can sense load changes within edge devices and automatically switch between Full and Split modes within milliseconds. Compared to the CN110321586A solution, which relies solely on external network bandwidth, this invention directly senses the computing power load within edge devices, effectively solving the problem of insufficient local computing power in multi-task concurrent scenarios. Compared to the CN111027150A solution, which relies on complex regression model prediction, this invention directly reads the load status through the underlying hardware monitoring interface, achieving a decision response with extremely low overhead (<1ms).

[0029] 3) By embedding a non-local attention module in the cloud model and combining it with lossless binary serialized feature transmission, the recognition accuracy of the split model is completely consistent with that of the original complete model.

[0030] 4) Employing standardized HTTP REST API and JSON data format, and using common open-source libraries such as PyTorch, NumPy, and FastAPI, this invention requires no dedicated hardware or customized software, facilitating integration with various edge devices and cloud platforms and reducing deployment and maintenance costs. Compared to pure cloud solutions, this invention reduces network dependence and transmission latency; compared to fixed-splitting solutions, it possesses real-time adaptive capabilities; and compared to model compression solutions, it achieves lossless accuracy. It demonstrates significant advantages in multiple dimensions, including edge computing power requirements, network dependence, recognition accuracy, adaptive capabilities, and response latency. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the overall architecture of the dynamic split computing system based on the edge-cloud collaborative model of edge computing load perception according to the present invention.

[0032] Figure 2 This is a schematic diagram illustrating the split points of the ResNet50 network and the division between the edge model and the cloud model in this invention.

[0033] Figure 3 This is a flowchart of the dynamic unloading decision module of the present invention.

[0034] Figure 4 This is a timing diagram of communication between the edge and the cloud in the Split mode of this invention. Detailed Implementation

[0035] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0036] This embodiment provides a dynamic splitting computation method for an edge-cloud collaborative deep learning model based on edge computing load awareness. The method flow is as follows: During the inference phase, the system first monitors the computing power utilization of the edge device in real time and dynamically decides whether to use local full inference (Full mode) or edge-cloud collaborative split inference (Split mode). When Split mode is selected, the edge model performs shallow feature extraction on the input image, transmits the intermediate features to the cloud after efficient serialization encoding, and the cloud model receives the features, performs subsequent deep network inference, and returns the final recognition result. The entire process requires no manual intervention; the system automatically adjusts the computation allocation strategy according to the real-time load status.

[0037] Intelligent model split architecture: This invention uses ResNet50 (Residual Network) as its backbone and divides the network into two parts at a shallow level: a lightweight edge model and a computationally intensive cloud model. Specifically, the edge model includes an input layer, a first convolutional layer (Conv1, using a 7×7 convolutional kernel with a stride of 2), a batch normalization layer (BN1), and a max pooling layer (MaxPool, using a 3×3 pooling kernel with a stride of 2). In current implementations, MaxPool is directly connected after BN1 without an additional independent ReLU activation layer. The reasons for choosing to divide the model after the MaxPool layer are as follows: the edge only performs about 2% of the total computation, significantly reducing the burden on edge devices; the feature map size output by MaxPool is 64×64×32, corresponding to about 512KB of single-frame data, which is acceptable for transmission latency under wireless network conditions; at the same time, the output of this layer still retains sufficient spatial resolution information for subsequent processing. The cloud model contains all the residual block structures from Layer 1 to Layer 4 of ResNet50 (Residual Network), as well as Non-local attention modules embedded in Layer 2 and Layer 3. Finally, after global average pooling and L2 normalization, it outputs a 2048-dimensional feature vector for the pedestrian re-identification task.

[0038] Dynamic unloading decision module: This module dynamically selects the operating mode by monitoring the computing power utilization of edge devices in real time. During the initialization phase, the system calls the underlying hardware monitoring interface to obtain the load status of the edge computing units, and periodically collects the current computing power utilization value during inference. When the computing power utilization exceeds a preset threshold (the default setting is 70%), the system automatically switches to Split mode, offloading the computing load to the cloud for execution; when the computing power utilization is below the threshold, the system returns to Full mode, independently executing the complete model inference at the edge. This load-aware dynamic decision-making mechanism enables the system to fully utilize local computing power when the edge device load is low, and automatically distribute the computing pressure to the cloud when the load is high, thereby achieving optimal utilization of computing resources. At the same time, the system records each mode switching event, which is convenient for subsequent performance analysis and strategy optimization.

[0039] High-efficiency feature transmission protocol: To achieve efficient communication between the edge and the cloud, this invention designs an intermediate feature serialization and network transmission scheme based on Base64 encoding. At the edge, the intermediate feature tensor output from forward inference is first migrated from the device's accelerator memory to system memory, then converted to NumPy array format. A byte stream is generated using NumPy's binary serialization function, and then Base64 encoded to generate a string that can be transmitted via HTTP (Hypertext Transfer Protocol). The edge encapsulates the encoded features along with the original tensor shape information into a JSON (JavaScript Object Notation) request body, which is then sent to the cloud inference service interface via the HTTP POST method. The cloud service, implemented using the FastAPI framework, receives the request and reverse-engineers the decoding, deserialization, format conversion, and device migration operations, restoring the feature tensor to a PyTorch Tensor on the cloud computing device, which is then input into the cloud model to complete subsequent inference. The inference result is returned to the edge using the same encoding process. This solution uses standardized HTTP (Hypertext Transfer Protocol) REST API (Representational State Transfer Application Programming Interface) and JSON (JavaScript Object Notation) data formats, ensuring strong compatibility and easy integration with various edge devices and cloud platforms without requiring dedicated hardware or software support.

[0040] Non-local attention retention mechanisms: To ensure that the recognition accuracy is not affected after model splitting, this invention fully preserves the non-local attention module in the cloud model. The non-local attention module can capture the dependencies between distant locations in the feature map, which is crucial for tasks requiring global contextual information, such as person re-identification. This invention embeds the non-local module in Layer 2 and Layer 3 of the ResNet50 (Residual Network) in a cascaded insertion manner while maintaining residual output: Layer 2 inserts one non-local module after the 3rd and 4th Bottleneck blocks, and Layer 3 inserts one non-local module after the 4th, 5th, and 6th Bottleneck blocks. Each non-local module does not replace the original Bottleneck structure; instead, it continues to receive the output after the corresponding Bottleneck completes its forward computation and fuses it with the original features in the residual form z = W(y) + x. This allows the cloud model to fully utilize the attention module to extract discriminative features even after receiving shallow features transmitted from the edge. This design ensures that the performance of the entire split computing system remains consistent with that of the complete model, achieving a good balance between computational efficiency and recognition accuracy.

[0041] During training, this invention employs the standard ResNet50 (Residual Network)-NL model for end-to-end training. After training, the model parameters are loaded into the edge model and the cloud model respectively according to the split point location. Since the split operation only involves the division of the model structure and does not change any weight parameters, there is no accuracy loss in the intermediate features during transmission, and the recognition performance of the split model is completely consistent with that of the complete model.

[0042] Example 1: like Figure 1As shown, the system of this invention consists of two parts: an edge device and a cloud server, which are connected via a wireless network. The edge device can be an unmanned aerial vehicle (UAV) computing platform with AI acceleration capabilities, an embedded edge computing module, or a domestically produced / general-purpose edge computing device. This device may include a GPU, NPU, or other AI accelerators, possessing the ability to run shallow network inference, load monitoring, and network communication, and supporting 4G / 5G or WiFi wireless network connections. The edge software environment can use Linux or other embedded operating systems, a Python runtime environment, deep learning frameworks such as PyTorch, and dependency libraries such as requests and numpy. The cloud server can be a server or cloud host equipped with high-performance AI acceleration resources, with at least 8GB of video memory or accelerator memory, a public IP address, or be located on the same intranet as the edge device. Its software environment includes Python, PyTorch, and dependency libraries such as fastapi, uvicorn, pydantic, and numpy.

[0043] like Figure 2 As shown, this embodiment uses the ResNet50 network as the backbone and performs segmentation at a shallow layer. Specifically, the segmentation point is located after the max pooling layer.

[0044] During system initialization, the edge-side model (ClientResNetNL) is loaded. This model includes a Conv1 convolutional layer (3 input channels, 64 output channels, 7×7 kernel size, stride 2, padding 3), a BN1 batch normalization layer, and a MaxPool max pooling layer (3×3 kernel size, stride 2, padding 1). The MaxPool is directly connected after the BN1 layer, without any additional independent ReLU activation layer. An RGB (Red Green Blue) image with an input size of [B, 3, 256, 128] is processed by the edge-side model and outputs an intermediate feature tensor with a size of [B, 64, 64, 32]. Simultaneously, a cloud-based model (ServerResNetNL) is loaded in the cloud. This model comprises Layer 1 (3 Bottleneck blocks), Layer 2 (4 Bottleneck blocks, with a non-local attention module concatenated after the 3rd and 4th Bottleneck blocks), Layer 3 (6 Bottleneck blocks, with a non-local attention module concatenated after the 4th, 5th, and 6th Bottleneck blocks), Layer 4 (3 Bottleneck blocks), as well as a global average pooling layer and an L2 normalization layer. The cloud model receives intermediate features transmitted from the edge as input and outputs a 2048-dimensional unit feature vector for pedestrian identity matching.

[0045] like Figure 3 As shown, during the runtime phase, the edge-side dynamic offloading decision module first initializes the computing load monitoring function: it checks whether the system supports the hardware monitoring interface of the corresponding edge computing unit. If supported, it calls the utilization query interface provided by the driver, runtime, or system service to obtain the device load status; if not supported, it sets a flag and uses simulated utilization for testing. Each time an inference request arrives, the decision module obtains the current computing utilization value and compares it with a preset threshold (default 70%). If the utilization exceeds the threshold, the system selects Split mode to execute edge-cloud collaborative inference; if the utilization is below the threshold, the system selects Full mode to independently execute complete model inference at the edge. The system also records each mode switching event, including the switching timestamp and the mode state before and after the switch, facilitating subsequent performance analysis. This forms a dynamic offloading decision process of "monitoring initialization - utilization acquisition - threshold comparison - mode switching - event recording".

[0046] When the system selects Full mode, the edge independently executes the complete ResNet50 (Residual Network)-NL model inference: the input image is processed sequentially through Conv1, BN1, MaxPool, Layer1, Layer2, Layer3, and Layer4, and finally a 2048-dimensional feature vector is obtained through global average pooling. After L2 normalization, the output is used for pedestrian re-identification matching.

[0047] like Figure 4 As shown, when the system selects the Split mode, the edge and cloud collaboratively perform split inference. The edge first feeds the input image into the ClientResNetNL model, sequentially performing Conv1 convolution (output size [B, 64, 128, 64]), BN1 batch normalization, and MaxPool max pooling (output size [B, 64, 64, 32]). Then, the edge migrates the output intermediate feature tensor from the device's accelerator memory to system memory, converts it to NumPy array format, serializes it into a binary byte stream using the numpy.save function, and then converts it to a text string using Base64 encoding. The edge encapsulates the encoded feature string and the original tensor shape information into a JSON (JavaScript Object Notation) request body and sends it to the / infer interface of the cloud server via the HTTP (HyperText Transfer Protocol) POST method.

[0048] The cloud service uses the FastAPI framework to implement the RESTful API interface. Upon receiving a POST request from the edge, the cloud parses the JSON (JavaScript Object Notation) request body, extracts the Base64 encoded feature string, decodes it to obtain a binary byte stream, deserializes it using the `numpy.load` function to restore it to a NumPy array, converts it to PyTorch Tensor format, and migrates it to the cloud computing device. Subsequently, the cloud inputs the feature tensor into the ServerResNetNL model, processing it sequentially through Layer 1, Layer 2 (including a Non-local attention module), Layer 3 (including a Non-local attention module), and Layer 4, obtaining a 2048-dimensional feature vector through global average pooling and L2 normalization. The cloud encapsulates the resulting feature vector into a JSON (JavaScript Object Notation) response body using the same encoding process (NumPy serialization, Base64 encoding) and returns it to the edge. Upon receiving the response, the edge performs a decoding operation to obtain the final recognition features for subsequent pedestrian matching tasks. The entire process constitutes a Split mode communication sequence of "shallow inference at the edge - feature serialization encoding - HTTP sending - cloud decoding and recovery - backend inference - result encoding and back transmission - edge decoding and use".

[0049] Example 2: When the method of this invention is deployed in actual drone inspection scenarios, the system can adaptively cope with dynamically changing computational loads. Taking urban security monitoring drones as an example: during normal daytime patrols, the onboard edge computing unit simultaneously runs multiple tasks such as target detection and image stabilization, resulting in high computational utilization. At this time, the dynamic offloading decision module detects that the utilization rate exceeds the 70% threshold and automatically switches the pedestrian re-identification computation task to Split mode. The edge only performs about 2% of shallow feature extraction computation, offloading the main computational load to the cloud to ensure that all drone tasks can run smoothly. When the drone performs fixed-point hovering monitoring, the load of other tasks decreases, and the computational utilization rate drops below the threshold. The system automatically switches back to Full mode, independently completing the complete inference at the edge, reducing network transmission dependence and improving response speed. The entire process requires no manual intervention. The system automatically adjusts the computational allocation strategy according to the real-time load status, achieving optimal utilization of computing resources.

[0050] In summary, the edge-cloud collaborative dynamic split computing method based on edge computing load awareness provided by this invention achieves efficient deployment of deep learning models on resource-constrained edge devices through intelligent split point selection, dynamic load decision-making, and efficient feature transmission mechanism, balancing recognition accuracy and computational efficiency.

Claims

1. An edge collaborative deep learning model dynamic splitting calculation method based on edge computing power load perception, characterized in that, The following steps are performed sequentially: Step S1: Model Splitting Architecture Construction Steps The pre-trained deep learning model is logically divided at preset shallow segmentation points to generate a lightweight edge model and a computationally intensive cloud model. The edge model is used to perform shallow feature extraction of the input image, and its computational cost accounts for only a preset low proportion of the total computational cost of the complete model. The cloud model is used to receive intermediate features and perform deep inference. Step S2: Real-time perception and pattern decision-making of edge computing load On edge devices, the computing power utilization rate of the local edge computing unit is collected in real time through the hardware-level monitoring interface, and this value is compared with a preset dynamic switching threshold. - If the current computing power utilization rate is greater than or equal to the threshold, the edge device is determined to be in a high load state, and the decision is made to execute the Split mode; - If the current computing power utilization rate is less than the threshold, the edge device is determined to be in a low load state, and the decision is made to execute the Full mode; Step S3: Split Mode Collaborative Reasoning Steps When step S2 decides to use the Split mode, the following sub-steps are executed sequentially: Step S3.1: Shallow inference at the edge: The edge model feeds the input image into the edge model for forward inference to obtain the intermediate feature tensor; Step S3.2: Feature Serialization and Encoding: The intermediate feature tensor is migrated from the device-side accelerator memory to the system memory, and after binary serialization and encoding, a text string is generated and encapsulated together with the tensor shape information into an HTTP request body; Step S3.3: Network Transmission: The edge device sends the request body to the cloud server via the HTTP protocol; Step S3.4: Cloud Decoding and Deep Inference: After receiving the data, the cloud server reverse-engineers the decoding and deserialization processes, inputs the recovered feature tensor into the cloud model, and completes deep inference to obtain the recognition result feature vector. Step S3.5: Result Return: The cloud encodes the feature vector of the recognition result and returns it to the edge terminal, which decodes it to obtain the final recognition result; Step S4: Full Mode Local Inference Steps When step S2 decides to be in Full mode, the edge end directly inputs the input image into the complete deep learning model, performs all forward inference calculations locally, and obtains the feature vector of the recognition result. Step S5: Adaptive Loop Step Return to step S2 and repeat the above edge computing load perception and mode decision process for the next frame or the next batch of input images to realize dynamic, closed-loop adaptive switching of the operating mode according to the real-time load status of the edge device.

2. The dynamic splitting calculation method for an edge collaborative deep learning model based on edge computing power load awareness according to claim 1, characterized in that, The preset shallow segmentation point is specifically located after the MaxPool layer of the ResNet50 network; the edge model is composed of the input layer, the first convolutional layer Conv1, the batch normalization layer BN1, and the max pooling layer MaxPool, which are used to perform the mapping from the original image to an intermediate feature map of size 64×64×32; the cloud model is composed of all residual block structures from Layer 1 to Layer 4 of the ResNet50 network, which are connected in series; the computational cost of the edge model accounts for 2% of the total computational cost of the complete ResNet50 model.

3. The dynamic splitting calculation method for an edge collaborative deep learning model based on edge computing power load awareness according to claim 1, characterized in that, The specific implementation method of collecting computing power utilization values ​​in real time through the hardware underlying monitoring interface in step S2 is as follows: call the utilization query function provided by the edge computing unit driver, runtime or system monitoring interface to directly read the current load status from the hardware or driver layer without establishing an additional regression prediction model; the default value of the dynamic switching threshold is 70%.

4. The dynamic splitting calculation method for an edge collaborative deep learning model based on edge computing power load awareness according to claim 1, characterized in that, The specific implementation of binary serialization and Base64 encoding in step S3.2 is as follows: convert the intermediate feature tensor into NumPy array format, call NumPy's binary serialization function to generate a byte stream, and then generate a text string through Base64 encoding; and encapsulate the encoded string and the original shape information of the feature tensor together into an HTTP request body in JSON format.

5. The dynamic splitting calculation method for an edge collaborative deep learning model based on edge computing power load awareness according to claim 1, characterized in that, The cloud model embeds non-local attention modules: two non-local attention modules are embedded in Layer 2 of the cloud model, and three non-local attention modules are embedded in Layer 3. The non-local attention modules are used to capture pixel-level dependencies between distant locations in the feature map, so that the cloud model can still extract global contextual discriminative features after receiving shallow features, thereby ensuring that the recognition accuracy in Split mode is completely consistent with the recognition accuracy of the full model in Full mode.

6. The dynamic splitting calculation method for an edge collaborative deep learning model based on edge computing power load awareness according to claim 1, characterized in that, The preset dynamic switching threshold mentioned in step S2 supports adaptive dynamic optimization: the system records the historical computing power utilization rate and the end-to-end inference latency before and after each mode switch, uses the sliding window statistical method to calculate the median of computing power utilization rate within the window, and dynamically updates the threshold to the value after the median and preset upper and lower limits are constrained, so as to adapt to the computing power characteristics and task load changes of different edge devices.

7. The dynamic splitting calculation method for an edge collaborative deep learning model based on edge computing power load awareness according to claim 1, characterized in that, In step S3.3, the HTTP protocol adopts the RESTful API architecture style, and the data transmission format is JSON; the cloud uses the FastAPI framework to implement the inference service interface; in step S3.5, the result return also adopts Base64 encoding and JSON encapsulation format to form a symmetrical encoding and decoding process.

8. A dynamic splitting computation system based on an edge-cooperative deep learning model with edge computing load awareness, used to execute the method of any one of claims 1 to 7, characterized in that, The system includes: Edge devices are deployed as follows: - Lightweight edge-end model for performing shallow feature extraction; - The computing load monitoring module is used to collect the computing power utilization rate of the local edge computing unit in real time through the hardware underlying monitoring interface; - The mode decision module is used to compare computing power utilization with dynamic thresholds and output Split or Full mode selection signals; - Feature encoding module, used to serialize and encode intermediate feature tensors into JSON request bodies in Split mode; - An HTTP communication client used to send requests to the cloud and receive responses; - Full model inference module, used to perform local full inference in Full mode; Cloud servers, deployed with: - HTTP server, used to receive inference requests sent from the edge; - Feature decoding module, used to decode, deserialize and migrate the received request body; - A computationally intensive cloud model that incorporates a non-local attention module to receive decoded features and perform deep inference; - Result encoding module, used to encode the inference results and return them to the edge; The edge device is connected to the cloud server via a wireless network, and the output signal of the mode decision module is used to select between the feature encoding module and the HTTP communication client (Split mode) or the full model inference module (Full mode).

9. The dynamic splitting computation system for edge collaborative deep learning models according to claim 8, characterized in that, The edge device is a drone, a mobile robot, or an embedded AI computing platform; the cloud server is a server or cloud host equipped with high-performance AI acceleration resources.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images