Cloud-edge collaborative data processing method, system, device and storage medium

Abstract

The embodiments of the present application provide a cloud-edge collaborative data processing method, system, device and storage medium, which relate to the field of communication technology. The method extracts the intermediate features of the data to be processed in the edge encoder, inputs the intermediate features into the lightweight edge decoder for decoding to obtain the edge decoding result, and when the feature uncertainty of the edge decoding result is less than or equal to the preset threshold, the edge decoding result is used as the target processing result. When the feature uncertainty is greater than the preset threshold, the intermediate features are compressed using the edge compression model and sent to the cloud server in the cloud-edge collaborative data processing system. The cloud-edge collaborative data processing model is segmented, and the intermediate features with smaller dimensions are compressed to further reduce the amount of data sent from the edge to the cloud and reduce communication overhead. The cloud-edge collaborative dynamic exit mechanism is used to selectively compress part of the data from the full amount of data for transmission, reducing the amount of data and communication overhead.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to cloud-edge collaborative data processing methods, systems, devices and storage media. Background Art

[0002] Cloud-edge collaborative inference is an edge computing architecture that combines the advantages of cloud computing and edge computing. By performing data processing and analysis separately in the cloud and at the edge, it enables more efficient and low-latency data processing and decision-making.

[0003] In related technologies, given that edge computing power is typically lower than that of the cloud, the edge typically performs simple preprocessing on input data before sending it to the cloud through data compression, where the cloud performs the bulk of the processing. However, this processing approach results in large data transmission volumes and high communication overhead during the collaborative process between the cloud and edge. Summary of the Invention

[0004] The main purpose of the embodiments of this application is to propose a cloud-edge collaborative data processing method, system, device and storage medium to reduce the communication overhead of cloud-edge collaborative processing.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application proposes a cloud-edge collaborative data processing method, which is applied to an edge server in a cloud-edge collaborative data processing system. An edge encoder, at least one lightweight edge decoder corresponding to different processing tasks, and an edge compression model are deployed on the edge server. The method includes:

[0006] Acquire data to be processed from a terminal, and send the data to be processed to the edge encoder for feature extraction to obtain intermediate features;

[0007] For each of the processing tasks, inputting the intermediate features into the lightweight edge decoder for decoding to obtain an edge decoding result, and calculating an entropy value of the edge decoding result as a feature uncertainty;

[0008] When the feature uncertainty is less than or equal to a preset threshold, the edge decoding result is used as the target processing result of the to-be-processed data corresponding to the processing task;

[0009] When the feature uncertainty is greater than the preset threshold, the edge compression model is used to compress the intermediate feature to obtain an intermediate compressed feature, and the intermediate compressed feature is sent to a cloud server in the cloud-edge collaborative data processing system;

[0010] Among them, a cloud decompression model and a cloud decoder that together with the edge encoder constitute a cloud-edge collaborative data processing model are deployed on the cloud server. The lightweight edge decoder corresponds to the cloud decoder. The cloud decompression model is used to decompress the intermediate compression features to obtain intermediate decompressed features. The cloud decoder is used to decode the intermediate decompressed features to obtain cloud decoding results, and use the cloud decoding results as the target processing results corresponding to the processing task.

[0011] In some embodiments, the edge compression model includes at least a quantization model, and the step of compressing the intermediate features using the edge compression model to obtain the intermediate compressed features includes:

[0012] quantizing the intermediate features using the quantization model to obtain intermediate quantized features;

[0013] The intermediate quantized features are arithmetically encoded to obtain the intermediate compressed features of the binary bit string.

[0014] In some embodiments, the edge compression model includes at least a side information encoder, a statistical parameter encoder, and a side information decoder. The using the edge compression model to perform feature compression on the intermediate features to obtain the intermediate compressed features includes:

[0015] Inputting the intermediate features into the side information encoder for feature correction to obtain statistical hidden features, and quantizing the statistical hidden features to obtain quantized hidden features;

[0016] Inputting the quantized hidden features into the statistical parameter encoder for encoding to obtain side information compression features;

[0017] Inputting the quantized hidden features into the side information decoder for decoding to obtain statistical parameters, and using the statistical parameters to correct the intermediate quantized features to obtain simulated compressed features;

[0018] The intermediate compression feature is obtained according to the simulation compression feature and the side information compression feature.

[0019] In some embodiments, the method further comprises:

[0020] Using at least one of the edge encoder, the lightweight edge decoder, and the edge compression model as a first basic network;

[0021] Obtain at least one of a first convolutional layer parameter of a convolutional layer or a first fully connected layer parameter of a fully connected layer in the first basic network, and perform singular value decomposition on the first convolutional layer parameter and / or the first fully connected layer parameter to obtain a first singular value matrix and a first default matrix;

[0022] Obtaining a preset first variation, obtaining a first compression matrix according to the first variation and the first singular value matrix, and obtaining first compression model parameters according to the first compression matrix and the first default matrix;

[0023] Parameters of the first basic network are updated based on the first compression model parameters to obtain an updated first basic network.

[0024] To achieve the above objectives, a second aspect of an embodiment of the present application proposes a cloud-edge collaborative data processing method, which is applied to a cloud server in a cloud-edge collaborative data processing system. The cloud server is deployed with a cloud decompression model and a cloud decoder that together with the edge encoder constitutes a cloud-edge collaborative data processing model. The method includes:

[0025] Receiving intermediate compression features sent from an edge server, where the intermediate compression features are obtained by the cloud-edge collaborative data processing method according to any one of the first aspects;

[0026] Decompressing the intermediate compressed features using the cloud decoding model to obtain intermediate decompressed features;

[0027] The corresponding cloud decoder is selected according to the processing task, the intermediate decompression feature is input into the cloud decoder for decoding, a cloud decoding result is obtained, and the cloud decoding result is used as the target processing result corresponding to the processing task.

[0028] In some embodiments, the cloud-based decoding model includes at least a cloud-based side information decoder and a statistical parameter decoder. When the intermediate compressed features are obtained from the analog compressed features and the side information compressed features, decompressing the intermediate compressed features using the cloud-based decoding model to obtain the intermediate decompressed features includes:

[0029] Obtaining the simulation compression feature and the side information compression feature from the intermediate compression feature;

[0030] Inputting the side information compression feature into the statistical parameter decoder for decoding to obtain a decoded hidden feature;

[0031] Inputting the decoded hidden features into the cloud-side information decoder for decoding to obtain cloud-side statistical parameters;

[0032] The simulated compression feature is arithmetically decoded using the cloud statistical parameters to obtain the intermediate decompression feature.

[0033] In some embodiments, the method further comprises:

[0034] Using at least one of the cloud-based decompression model or the cloud-based decoder as a second basic network;

[0035] Obtain at least one of a second convolutional layer parameter of the convolutional layer or a second fully connected layer parameter of the fully connected layer in the second basic network, and perform singular value decomposition on the second convolutional layer parameter and / or the second fully connected layer parameter to obtain a second singular value matrix and a second default matrix;

[0036] Obtaining a preset second variation, obtaining a second compression matrix according to the second variation and the second singular value matrix, and obtaining second compression model parameters according to the second compression matrix and the second default matrix;

[0037] Parameters of the second basic network are updated based on the second compression model parameters to obtain an updated second basic network.

[0038] To achieve the above objectives, a third aspect of the embodiments of the present application proposes a cloud-edge collaborative data processing system, including:

[0039] An edge server, wherein an edge encoder, at least one lightweight edge decoder corresponding to different processing tasks, and an edge compression model are deployed on the edge server;

[0040] A cloud server, wherein a cloud decompression model and a cloud decoder are deployed on the cloud server, the cloud decoder and the edge encoder together constitute a cloud collaborative data processing model, and the cloud decoder corresponds to the lightweight edge decoder;

[0041] The edge server is configured to use the edge encoder to perform feature extraction on the data to be processed from the terminal to obtain intermediate features, use the lightweight edge decoder to decode the intermediate feature input to obtain an edge decoding result, calculate the entropy value of the edge decoding result as a feature uncertainty, and when the feature uncertainty is less than or equal to a preset threshold, use the edge decoding result as a target processing result of the data to be processed; when the feature uncertainty is greater than the preset threshold, use the edge compression model to perform feature compression on the intermediate features to obtain intermediate compressed features, and send the intermediate compressed features to the cloud server;

[0042] The cloud server is used to decompress the intermediate compression features using the cloud decompression model to obtain intermediate decompression features, select the corresponding cloud decoder according to the processing task, input the intermediate decompression features into the cloud decoder for decoding, obtain a cloud decoding result, and use the cloud decoding result as the target processing result corresponding to the processing task.

[0043] In some embodiments, the training process of the edge encoder, the edge compression model, the cloud decompression model, and the cloud decoder includes the following steps:

[0044] Get input sample data;

[0045] During the training process, obtaining intermediate training data corresponding to the input sample data, obtaining transmission mutual information between the input sample data and the intermediate training data, and generating an information amount constraint based on the maximum information amount and the transmission mutual information;

[0046] Obtaining cloud-based inference results corresponding to different processing tasks corresponding to the intermediate training data, generating inference mutual information between each cloud-based inference result and the intermediate training data, and maximizing the inference mutual information based on a Lagrange multiplier corresponding to each processing task to obtain a compression target;

[0047] Obtaining an optimization target according to the transmission mutual information and the compression target;

[0048] Obtaining a loss function and a target upper bound corresponding to the optimization target based on the input sample data, the intermediate training data, and the cloud-based inference result;

[0049] Under the premise of satisfying the target upper bound and the information amount constraint, the loss value corresponding to the loss function is minimized, and the edge encoder, the edge compression model, the cloud decompression model and the cloud decoder are trained.

[0050] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the method described in the first or second aspect above when executing the computer program.

[0051] To achieve the above-mentioned purpose, the first aspect of an embodiment of the present application proposes a storage medium, which is a storage medium that stores a computer program. When the computer program is executed by a processor, it implements the method described in the first or second aspect above.

[0052] The cloud-edge collaborative data processing method, system, device and storage medium proposed in the embodiment of the present application obtain the data to be processed from the terminal, and send the data to be processed into the edge encoder for feature extraction to obtain intermediate features. Next, for each processing task, the intermediate features are input into the lightweight edge decoder for decoding to obtain the edge decoding result, and the entropy value of the edge decoding result is calculated as the feature uncertainty. When the feature uncertainty is less than or equal to the preset threshold, the edge decoding result is used as the target processing result of the processing task corresponding to the data to be processed. When the feature uncertainty is greater than the preset threshold, the intermediate features are compressed using the edge compression model to obtain intermediate compressed features, and the intermediate compressed features are sent to the cloud server in the cloud-edge collaborative data processing system. The intermediate compressed features are decompressed on the cloud server to obtain intermediate decompressed features, and then the intermediate decompressed features are decoded to obtain cloud decoding results, and the cloud decoding results are used as the target processing results corresponding to the processing task. In the embodiment of the present application, the cloud-edge collaborative data processing model is divided, and part of the processing process is implemented at the edge end, and the other part is implemented in the cloud. Through this segmentation process, the data to be processed is converted into intermediate features of smaller dimensions. These intermediate features are then compressed to further reduce the amount of data sent from the edge to the cloud, thereby reducing communication overhead. Furthermore, for different processing tasks, a lightweight model is pre-calculated to quickly generate a result. The entropy of this result is used to determine whether to process the result at the edge or in the cloud. This flexible cloud-edge collaborative dynamic exit mechanism allows for selective compression of partial data from the full data set for transmission, further reducing both data volume and communication overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Schematic diagram of the principle of cloud-edge collaborative reasoning in related technologies.

[0054] Figure 2 It is a schematic diagram of the principles of the cloud-edge collaborative data processing system provided in an embodiment of the present application.

[0055] Figure 3 It is a structural diagram of the cloud-edge collaborative data processing system provided in an embodiment of the present application.

[0056] Figure 4 This is a flowchart of the cloud-edge collaborative data processing method provided in an embodiment of the present application applied to an edge server.

[0057] Figure 5 This is a schematic diagram of the cloud-edge collaborative dynamic exit mechanism provided in an embodiment of the present application.

[0058] Figure 6 It is a flow chart of the cloud-edge collaborative data processing method provided in an embodiment of the present application.

[0059] Figure 7 This is a schematic diagram of an embodiment of the present application providing a method of compressing intermediate features using an edge compression model to obtain intermediate compressed features.

[0060] Figure 8 This is a flowchart of adaptively adjusting the compression rate of the edge end provided by an embodiment of the present application.

[0061] Figure 9 This is a schematic diagram of the compression rate adjustment of the first basic network provided in an embodiment of the present application.

[0062] Figure 10 This is an optional flowchart of the cloud-edge collaborative data processing method provided in an embodiment of the present application applied to a cloud server.

[0063] Figure 11 It is a flowchart of using the cloud decoding model to decompress the intermediate compressed features to obtain the intermediate decompressed features.

[0064] Figure 12 This is a flow chart of the training process of the edge encoder, edge compression model, cloud decompression model and cloud decoder in the cloud-edge collaborative data processing system provided by an embodiment of the present application.

[0065] Figure 13 These are the accuracy experimental results for the image classification and reconstruction tasks provided in the embodiments of this application.

[0066] Figure 14 It is the accuracy loss of the cloud-edge collaborative dynamic exit mechanism under different network availability provided in the embodiment of this application.

[0067] Figure 15 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0069] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the order in the flowchart.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0071] First, let’s analyze some of the terms used in this application:

[0072] Artificial intelligence (AI) is a new technical discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. A branch of computer science, AI seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thinking. It also encompasses the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0073] Cloud-edge collaborative inference is an edge computing architecture that combines the advantages of cloud computing and edge computing. By performing data processing and analysis separately in the cloud and at the edge, it enables more efficient and low-latency data processing and decision-making.

[0074] In related technologies, since the computing power of the edge is usually lower than that of the cloud, the edge usually performs simple preprocessing on the input data, and then sends the data to the cloud through data compression, and the cloud completes most of the processing work. Figure 1 , Figure 1 Schematic diagram of the principle of cloud-edge collaborative reasoning in related technologies. Figure 1 In the process, the input data is compressed at the edge, and then the compressed data is sent to the cloud. The cloud then performs a decompression process to obtain the input data, and the input data is input into the neural network model for processing to obtain the task result. Even if the neural network model is directly split and deployed on the edge and cloud respectively, and part of the neural network is offloaded from the edge to the cloud, communication overhead will still be introduced in the neural network inference process. This is mainly because when the edge transmits the intermediate feature data of the model segmentation point to the cloud, the size of the intermediate feature data and the quality of the communication connection directly affect the inference delay. Therefore, in the related technology, the communication overhead is relatively high in the collaborative process between the cloud and the edge.

[0075] Based on this, the embodiments of the present application provide a cloud-edge collaborative data processing method, system, device and storage medium, which divides the cloud-edge collaborative data processing model into two parts, one part of the processing process is implemented at the edge, and the other part is implemented at the cloud. Through this segmentation process, the data to be processed is converted into intermediate features with smaller dimensions, and then these intermediate features are compressed to further reduce the amount of data sent from the edge to the cloud, thereby reducing communication overhead. In addition, for different processing tasks, a lightweight model is used in advance to calculate a quick result, and the entropy value of the result is used to determine whether to select the processing result of the edge or the processing result of the cloud. This flexible cloud-edge collaborative dynamic exit mechanism is used to selectively compress part of the data from the full amount of data for transmission, which can further reduce the amount of data and communication overhead.

[0076] The embodiments of the present application provide a cloud-edge collaborative data processing method, system, device and storage medium, which are specifically illustrated by the following embodiments. First, the cloud-edge collaborative data processing method in the embodiments of the present application is described.

[0077] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0078] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0079] The cloud-edge collaborative data processing method provided in the embodiment of the present application relates to the field of communication technology. The cloud-edge collaborative data processing method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be a computer program running in a terminal or a server side. For example, a computer program can be a native program or software module in an operating system; it can be a local (Native) application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as a client that supports cloud-edge collaborative data processing, or it can be a small program, that is, a program that can be run only by downloading it to a browser environment; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plug-in. Among them, the terminal communicates with the server through a network. The cloud-edge collaborative data processing method can be executed by a terminal or a server, or by the terminal and the server in collaboration.

[0080] In some embodiments, the terminal can be a smartphone, tablet computer, laptop computer, desktop computer or smart watch, etc. In addition, the terminal can also be an intelligent vehicle-mounted device. The intelligent vehicle-mounted device applies the cloud-edge collaborative data processing method of this embodiment to provide related services to enhance the driving experience. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms; it can also be a service node in a blockchain system, and the service nodes in the blockchain system form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP). The terminal and the server can be connected via Bluetooth, Universal Serial Bus (USB) or network communication connection methods, which are not limited in this embodiment.

[0081] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0082] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0083] First, the cloud-edge collaborative data processing system provided by the embodiment of the present application is described. Figure 2 , Figure 2 This is a schematic diagram of the principles of the cloud-edge collaborative data processing system provided by an embodiment of the present application. The cloud-edge collaborative data processing model is divided into two parts, one of which is deployed on the edge server (hereinafter referred to as the edge), and the other is deployed on the cloud server (hereinafter referred to as the cloud). During the processing of the edge, the input data is processed into low-dimensional intermediate features, and then the intermediate features are compressed and the compressed data is transmitted to the cloud. The cloud decompresses the compressed data to obtain the intermediate features, and then uses the model of the other part to process the intermediate features to obtain the final task results.

[0084] In one embodiment, referring to Figure 3 , Figure 3 It is a structural diagram of the cloud-edge collaborative data processing system provided in an embodiment of the present application. Figure 3The terminal acts as a sensor to collect data and obtain different data to be processed. The data to be processed here can be collected pictures, audio or video, and each type of data to be processed can correspond to at least one different processing task. In other words, in the embodiment of the present application, multi-task processing is performed on the data to be processed. Taking pictures as an example, the processing tasks can be image recognition, image reconstruction, etc. The embodiment of the present application does not limit the processing tasks corresponding to the data to be processed.

[0085] In one embodiment, the terminal sends the collected data to be processed to the edge for processing via a local area network or other means. The edge processes the data to be processed to obtain intermediate features, compresses them to obtain intermediate compressed features, and then sends the intermediate compressed features to the cloud for subsequent processing. The cloud decompresses the intermediate compressed features to obtain corresponding intermediate features, and then processes the intermediate features according to different processing tasks to obtain cloud decoding results corresponding to different processing tasks. In addition, task processing in the cloud is a multi-tasking process, and each processing task corresponds to a cloud decoder. Therefore, the intermediate features are input into different cloud decoders for processing.

[0086] Among them, an edge encoder, at least one lightweight edge decoder corresponding to different processing tasks and an edge compression model are deployed on the edge end, and a cloud decompression model and a cloud decoder are deployed on the cloud server.

[0087] In one embodiment, the cloud-edge collaborative processing model is segmented to obtain an edge encoder and a cloud decoder, wherein the edge encoder is deployed on the edge and the cloud decoder is deployed on the cloud. The embodiment of the present application does not specifically limit the segmentation point, that is, the cloud-edge collaborative processing model can be segmented into two parts according to any segmentation method, and the part that needs to be deployed on the edge is called the edge encoder, and the part that needs to be deployed on the cloud is called the cloud decoder. In specific applications, the cloud-edge collaborative processing model is segmented, and then a lightweight edge decoder and an edge compression model are added after the segmented edge encoder, and deployed together on the edge. A cloud decompression model is added in front of the segmented cloud decoder, and they are deployed together on the cloud.

[0088] In one embodiment, an optimal split point can also be determined based on the computing power of the edge and cloud. The computational effort of the edge encoder obtained by segmentation according to the determined split point is smaller than that of the cloud decoder. Because an edge compression model and a cloud decompression model are provided, when determining the split point, the embodiment of the present application does not need to consider whether the data dimension of the intermediate features is smaller than the data to be processed. This allows for a wider range of split points, making it suitable for different cloud-edge collaborative processing models.

[0089] In addition, the embodiments of the present application also control the compression rate of different models in the cloud-edge collaborative data processing system, thereby further reducing the amount of data that needs to be processed and reducing communication overhead.

[0090] The following describes the cloud-edge collaborative data processing method in an embodiment of the present application in conjunction with the above figure.

[0091] Figure 4 This is an optional flow chart of the cloud-edge collaborative data processing method provided in the embodiment of the present application applied to the edge server. Figure 4 The method may include but is not limited to steps 110 to 140. It is also understood that this embodiment is Figure 4 The order of step 110 to step 140 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0092] Step 110: Obtain the data to be processed from the terminal, and send the data to be processed to the edge encoder for feature extraction to obtain intermediate features.

[0093] In one embodiment, combining Figure 3 At the edge, edge encoders are used to extract features from the data to be processed, converting the features to be processed into more compact and information-rich intermediate features. By reducing the data dimension through feature extraction, only key features are extracted for subsequent possible cloud computing, which can greatly reduce the bandwidth requirements for data transmission. The cloud does not need to process large amounts of raw data and can make decisions and respond faster.

[0094] Step 120: For each processing task, the intermediate features are input into a lightweight edge decoder for decoding to obtain an edge decoding result, and the entropy value of the edge decoding result is calculated as the feature uncertainty.

[0095] In one embodiment, due to the different computational complexities of multiple processing tasks and the differences in computing power between the cloud and the edge, the cloud has powerful computing power and large-capacity storage, which is suitable for processing complex analysis and large-scale data storage tasks; while the edge has lower processing latency, which is suitable for rapid response and processing of real-time data. If all processing tasks are performed in the cloud, it will not only cause a waste of computing power on the edge, but also affect the overall processing efficiency. Therefore, in the embodiment of the present application, a choice is made among the processing tasks of the intermediate features to determine which processing tasks are performed by the cloud and which are performed by the edge, so as to reasonably allocate the processing tasks on the cloud and the edge. This can optimize the computing power consumption in the cloud-edge collaboration process and maximize the overall budget efficiency.

[0096] In one embodiment, a lightweight edge decoder is deployed at the edge, identical to the cloud decoder in the cloud. This means that for every cloud decoder, there is a corresponding lightweight edge decoder. The difference between the lightweight edge decoder and the cloud decoder is that while they handle the same tasks, the lightweight edge decoder has a more lightweight model structure, resulting in higher processing efficiency and faster response.

[0097] Among them, for the tth processing task, the tth cloud decoder is recorded as in, represents the intermediate features after decompression on the cloud, θ (t) represents the model parameters of the cloud decoder. The corresponding t-th lightweight edge decoder is recorded as y represents the intermediate feature of the edge, η (t) represents the model parameters of the lightweight edge decoder, where η (t) The parameter scale is smaller than θ (t) .

[0098] First, the intermediate features are input into the lightweight edge decoder for decoding to obtain the edge decoding result Next, calculate the edge decoding result Entropy As feature uncertainty, where H(·) represents an entropy calculation function, such as Shannon entropy. The purpose of calculating the entropy value in the embodiment of the present application is to obtain the uncertainty of the edge decoding result to measure the amount of information, so that intermediate features with less information can be processed at the edge end, and intermediate features with more information can be sent to the cloud to extract information using the cloud decoder with stronger expressive power.

[0099] Step 130: When the feature uncertainty is less than or equal to a preset threshold, the edge decoding result is used as the target processing result of the processing task corresponding to the data to be processed.

[0100] In one embodiment, different processing tasks correspond to different preset thresholds. Figure 5 , Figure 5 This is a schematic diagram of the cloud-edge collaborative dynamic exit mechanism provided in an embodiment of the present application. The cloud-edge collaborative dynamic exit mechanism includes both edge exit and cloud exit. Edge exit means the entire processing flow ends at the edge, while cloud exit means the entire processing flow ends in the cloud, reducing the number of communications with the cloud.

[0101] For edge exit, edge decoding results are obtained based on intermediate features The corresponding entropy value Then, the entropy value Compare with the preset threshold. Assume that the preset threshold corresponding to the tth processing task is At this time, if: Then receive the result of edge exit, that is, the edge decoding result The target processing result corresponding to the tth processing task as the data to be processed.

[0102] Step 140: When the feature uncertainty is greater than a preset threshold, the intermediate feature is compressed using the edge compression model to obtain an intermediate compressed feature, and the intermediate compressed feature is sent to the cloud server in the cloud-edge collaborative data processing system.

[0103] In one embodiment, combining Figure 3 A cloud decompression model and a cloud decoder corresponding to each processing task are deployed on the cloud server. The cloud decompression model is used to decompress the intermediate compressed features to obtain the intermediate decompressed features. The cloud decoder is used to decode the intermediate decompressed features to obtain the cloud decoding results, and the cloud decoding results are used as the target processing results corresponding to the processing tasks.

[0104] Reference Figure 5 If the result of edge exit is not satisfied, that is, the feature uncertainty is greater than the preset threshold, then enter the cloud exit, use the edge compression model to compress the intermediate features, obtain the intermediate compressed features, and then send the intermediate compressed features to the cloud server in the cloud-edge collaborative data processing system. Data processing is performed in the cloud to obtain the cloud decoder corresponding to the t-th processing task for decoding, and the corresponding cloud decoding result is obtained as the target processing result.

[0105] The aforementioned cloud-edge collaborative dynamic exit mechanism can also cope with network fluctuations. Although the accuracy of lightweight edge decoders is not as good as cloud decoders, they can still exit at the edge when they provide qualified data to meet the processing task requirements, and exit in the cloud when more accurate results are needed, thus meeting the dual requirements of speed and accuracy.

[0106] Next refer to Figure 6 , Figure 6 This is a flow chart of the cloud-edge collaborative data processing method provided by the embodiment of this application. Figure 6 It can be seen that the edge compression model includes at least a quantization model or a correction model, wherein the correction model includes a side information encoder, a statistical parameter encoder, and a side information decoder. The purpose of the correction model in this embodiment is to increase the information content of the intermediate features and adjust their distribution so that different intermediate features have different distribution characteristics, thereby improving the accuracy of the data processing results. Therefore, whether to add the correction model is determined based on actual needs and the actual computing power of the edge end.

[0107] In one embodiment, taking a separate quantization model as an example, the edge compression model is used to compress the intermediate features, and the process of obtaining the intermediate compressed features specifically includes: quantizing the intermediate features using the quantization model to obtain intermediate quantized features, and arithmetically encoding the intermediate quantized features to obtain intermediate compressed features of the binary bit string.

[0108] In one embodiment, the quantization process used in the model training process can be a soft quantization process, that is, soft quantization is performed using uniform distribution, and an error is added to the intermediate feature y′ to obtain the intermediate quantized feature Expressed as: Among them, ε~U(-0.5,0.5). In the inference process of the model, hard quantization operation with rounding is used to obtain the intermediate quantization feature Expressed as: in, Indicates a rounding operation.

[0109] In one embodiment, the intermediate quantized features are then arithmetic-coded and compressed using default statistical distribution parameters, such as mean and standard deviation. The principle of arithmetic coding in this embodiment is to use long bytes to represent numbers with a low probability of appearing in the intermediate quantized features, and short bytes to represent numbers with a high probability of appearing in the intermediate quantized features. Therefore, the probability of occurrence of each number in the intermediate quantized features must be determined based on the statistical distribution parameters. After arithmetic coding, the resulting binary bit string constitutes the intermediate compressed features.

[0110] In one embodiment, taking the inclusion of a modified model as an example, the purpose of the modified model is to provide adaptive statistical parameters different from the default statistical distribution parameters for the intermediate quantitative features. Figure 7 , Figure 7 This is a schematic diagram of using an edge compression model to compress intermediate features to obtain intermediate compressed features, as provided in an embodiment of the present application. The method specifically includes the following steps:

[0111] Step 710: Input the intermediate features into the side information encoder for feature correction to obtain statistical latent features, and quantize the statistical latent features to obtain quantized latent features.

[0112] In one embodiment, combining Figure 6The side information encoder deployed at the edge is used to mine the hidden features corresponding to the true statistical parameters of the intermediate features. For example, the true statistical parameters can be the true mean or the true standard deviation. The statistical hidden features can reflect the distribution of different intermediate features. Then, after obtaining the statistical hidden features, they are quantized to obtain quantized hidden features. It can be understood that, with reference to the quantization process of the above-mentioned intermediate features, soft quantization can be used for quantization in the training phase, and hard quantization can be used for quantization in the inference phase. This embodiment does not limit this.

[0113] Step 720: Input the quantized hidden features into the statistical parameter encoder for encoding to obtain side information compression features.

[0114] In one embodiment, after obtaining the quantized hidden features, two parts of processing are required. The first part is to use the quantized hidden features to adapt the statistical parameters of the compression process of the intermediate quantized features. The other part is to compress the quantized hidden features and send them to the cloud so as to decode the intermediate compressed features obtained by the statistical parameter adaptation.

[0115] In one embodiment, the quantized hidden features are input into a statistical parameter encoder for arithmetically encoding to obtain side information compression features. The purpose of the statistical parameter encoder here is to compress the quantized hidden features by arithmetically encoding. By inputting a set of quantized hidden features, the corresponding side information compression features can be output.

[0116] Step 730: Input the quantized hidden features into the side information decoder for decoding to obtain statistical parameters, and use the statistical parameters to correct the intermediate quantized features to obtain simulated compressed features.

[0117] In one embodiment, information mining is performed from the quantized hidden features to obtain statistical parameters that can represent the true statistical parameters of the intermediate features. Specifically, the quantized hidden features are input into a side information decoder for decoding to obtain the corresponding statistical parameters.

[0118] Next, during arithmetic coding of the intermediate quantized features, unlike the previous compression using default statistical distribution parameters, the statistical parameters calculated above are used to modify the intermediate quantized features, resulting in simulated compressed features. The use of real statistical parameters modifies the probability of each symbol to be encoded during lossless entropy coding, ensuring that the simulated compressed features approach the Shannon limit as closely as possible.

[0119] Step 740: Obtain an intermediate compression feature based on the simulation compression feature and the side information compression feature.

[0120] In one embodiment, the simulated compressed features and the side information compressed features are combined to generate an overall intermediate compressed feature, which is then sent to the cloud for subsequent processing. By modifying the compression process, the compression rate of the intermediate feature and the accuracy of subsequent processing tasks are further improved.

[0121] In one embodiment, to adaptively adjust the compression rate at the edge, different correlation models are obtained by setting different variation amounts during training. Due to the different variation amounts, the output results of the resulting models will also vary, reflected in different compression rates of the final intermediate compressed features. Therefore, model training can be performed in advance based on different variation amounts to obtain correlation models corresponding to different compression rates.

[0122] Reference Figure 8 , Figure 8 This is a flow chart of adaptively adjusting the compression rate of the edge provided by an embodiment of the present application, which specifically includes the following steps:

[0123] Step 810: Use at least one of an edge encoder, a lightweight edge decoder, and an edge compression model as a first basic network.

[0124] In one embodiment, the edge encoder, lightweight edge decoder and edge compression model at the edge end can all adapt the compression rate in this way. It can be understood that the compression rate control processes of the edge encoder, lightweight edge decoder and edge compression model are independent, so one or more first basic networks can be selected according to actual needs.

[0125] Step 820: Obtain at least one of the first convolutional layer parameters of the convolutional layer or the first fully connected layer parameters of the fully connected layer in the first basic network, and perform singular value decomposition on the first convolutional layer parameters and / or the first fully connected layer parameters to obtain a first singular value matrix and a first default matrix.

[0126] In one embodiment, the first base network is a neural network, and parameters of the convolutional layer or the fully connected layer therein can be locked to obtain different training results. Therefore, at least one of the first convolutional layer parameters of the convolutional layer or the first fully connected layer parameters of the fully connected layer in the first base network is obtained as the relevant parameters to be parameter locked. Then, singular value decomposition is performed on the selected first convolutional layer parameters and / or first fully connected layer parameters to obtain a first singular value matrix and a first default matrix.

[0127] Reference Figure 9 , Figure 9 : This is a schematic diagram of the compression rate adjustment of the first basic network provided in the embodiment of the present application. The first convolutional layer parameter or the first fully connected layer parameter is selected as the adjustment parameter φ∈R in×out, the input data dimension is in, the output data dimension is out, and the singular value decomposition of the adjustment parameter φ is expressed as:

[0128] φ=U×diag(δ)×V T

[0129] Where U is an in×in unitary matrix, V is an out×out unitary matrix, and diag(δ) is a positive semidefinite in×out diagonal matrix. The two unitary matrices U and V are used as the first default matrices, and δ is used as the first singular value matrix.

[0130] Step 830: Obtain a preset first variation, obtain a first compression matrix according to the first variation and the first singular value matrix, and obtain first compression model parameters according to the first compression matrix and the first default matrix.

[0131] In one embodiment, the model parameters are updated according to the actual training process, and the parameter changes of the model are quantified to obtain at least one first change Δ. Then, a first compression matrix is ​​obtained based on the first change Δ and the first singular value matrix δ, which is expressed as: δ′=ReLU(δ+Δ), where ReLU is an activation function. Finally, the first compression model parameter φ′ is obtained based on the first compression matrix and the first default matrix, which is expressed as:

[0132] φ′=U×diag(δ′)×V T

[0133] Step 840: Update the parameters of the first basic network based on the first compression model parameters to obtain an updated first basic network.

[0134] In one embodiment, the parameters of the first basic network are updated using different first compression model parameters to obtain corresponding different, updated first basic networks. It is understandable that by decomposing the parameter space of the model, the change amount and the updated first basic network are stored accordingly, and when storing the relevant models of the compression ratio corresponding to different intermediate features, only the first basic model and a small number of updated parameters corresponding to the compression ratio need to be stored, thereby achieving flexible adjustment of the compression ratio while saving edge storage resources. At the same time, when the change amount is selected to switch the compression ratio according to actual needs, the updated first basic network is used to replace the original first basic network, thereby controllably adjusting the compression ratio of the intermediate compressed features.

[0135] In the embodiment of the present application, the cloud-edge collaborative data processing model is segmented, with one part of the processing process implemented at the edge and the other part implemented at the cloud. Through this segmentation process, the data to be processed is converted into intermediate features with smaller dimensions, and then these intermediate features are compressed to further reduce the amount of data sent from the edge to the cloud, thereby reducing communication overhead. In addition, for different processing tasks, a lightweight model is used in advance to calculate a quick result, and the entropy value of the result is used to determine whether to select the processing result of the edge or the processing result of the cloud. This flexible cloud-edge collaborative dynamic exit mechanism is used to selectively compress part of the data from the full amount of data for transmission, which can further reduce the amount of data and communication overhead.

[0136] The following describes the processing process of the cloud-edge collaborative data processing method in the cloud in an embodiment of the present application.

[0137] Figure 10 This is an optional flow chart of the cloud-edge collaborative data processing method provided in the embodiment of the present application applied to a cloud server. Figure 10 The method may include but is not limited to steps 1010 to 1030. It is also understood that this embodiment is for Figure 10 The order of step 1010 to step 1030 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0138] Step 1010: Receive intermediate compression features sent from the edge server.

[0139] Step 1020: Decompress the intermediate compressed features using the cloud decoding model to obtain intermediate decompressed features.

[0140] In one embodiment, combining Figure 4 ,The cloud uses the cloud decoding model to perform a decompression operation on the intermediate ,compressed features corresponding to the compression to obtain the intermediate ,decompressed features.

[0141] In one embodiment, because the intermediate compressed features may include the results of the correction module, the decompression operation is also divided into two steps. If the intermediate compressed features do not include the side information compression features provided by the correction module, the intermediate compressed features are directly subjected to reverse arithmetic decoding based on default statistical distribution parameters to obtain the intermediate decompressed features. It is understood that if transmission loss and decoding errors are ignored, the intermediate decompressed features are consistent with the intermediate features.

[0142] In one embodiment, if the intermediate compression feature contains the side information compression feature provided by the correction module, decompression based on the side information compression feature is required. Figure 6 , the cloud decoding model includes the corresponding cloud side information decoder and statistical parameter decoder. Figure 11 , Figure 11 This is a flowchart of using the cloud decoding model to decompress the intermediate compressed features to obtain the intermediate decompressed features, which specifically includes the following steps:

[0143] Step 1110: Obtain simulation compression features and side information compression features from the intermediate compression features.

[0144] In one embodiment, since the simulation compression features and the side information compression features are obtained by splicing, the cloud can directly obtain the corresponding simulation compression features and side information compression features from the received intermediate compression features.

[0145] Step 1120: Input the side information compression feature into the statistical parameter decoder for decoding to obtain the decoded hidden feature.

[0146] In one embodiment, since the statistical parameter decoder and the statistical parameter encoder perform opposite functions, the compressed side information features are input into the statistical parameter decoder for decoding to obtain the decoded latent features. It is understood that the decoded latent features are the quantized latent features obtained through decoding. The difference in names is intended to distinguish them and to reflect errors introduced during the transmission, encoding, and decoding processes.

[0147] Step 1130: Input the decoded hidden features into the cloud-side edge information decoder for decoding to obtain cloud-side statistical parameters.

[0148] In one embodiment, after obtaining the decoded hidden features, similar to step 730, the decoded hidden features are input into the cloud-side side information decoder for decoding to obtain cloud-side statistical parameters. It is understood that the cloud-side side information decoder and the edge-side side information decoder can also share parameters to reduce training costs.

[0149] Step 1140: Perform arithmetic decoding on the simulated compression features using the cloud statistical parameters to obtain intermediate decompression features.

[0150] In one embodiment, since the arithmetic coding process determines the byte lengths used for different numbers in the analog compression feature based on statistical parameters, the reverse arithmetic decoding process can determine the possible numbers corresponding to long bytes and short bytes respectively based on the analog compression parameters related to the statistical parameters, thereby completing the decoding process and obtaining the intermediate decompression feature.

[0151] Step 1030: Select the corresponding cloud decoder according to the processing task, input the intermediate decompression features into the cloud decoder for decoding, obtain the cloud decoding result, and use the cloud decoding result as the target processing result corresponding to the processing task.

[0152] In one embodiment, since different processing tasks exit at different times, if they exit at the edge, there is no need to perform the decoding process of the processing task in the cloud. Therefore, some processing tasks exit at the edge and some processing tasks exit in the cloud. In this case, a task identifier can be added during the transmission of the intermediate compressed features so that the cloud can know which processing tasks need to be executed and select the corresponding cloud decoder. After the corresponding cloud decoder is selected, the intermediate decompression features are input into the cloud decoder for decoding to obtain the cloud decoding result. At this time, these processing tasks are all exited by the cloud, so the corresponding cloud decoding result is used as the target processing result corresponding to the processing task.

[0153] In one embodiment, referring to the compression rate control process at the edge, the cloud can also perform compression rate control on the relevant model. The specific process includes the following steps: using at least one of the cloud decompression model or the cloud decoder as the second basic network, obtaining at least one of the second convolutional layer parameters of the convolutional layer or the second fully connected layer parameters of the fully connected layer in the second basic network, and performing singular value decomposition on the second convolutional layer parameters and / or the second fully connected layer parameters to obtain a second singular value matrix and a second default matrix, obtaining a pre-set second change amount, obtaining a second compression matrix based on the second change amount and the second singular value matrix, and obtaining second compression model parameters based on the second compression matrix and the second default matrix, and updating the parameters of the second basic network based on the second compression model parameters to obtain an updated second basic network.

[0154] In combination with the description of the above embodiments, in the cloud-edge collaborative data processing system in the embodiment of the present application, the edge server is used to use an edge encoder to extract features from the data to be processed from the terminal to obtain intermediate features, use a lightweight edge decoder to decode the intermediate feature input to obtain an edge decoding result, calculate the entropy value of the edge decoding result as the feature uncertainty, and when the feature uncertainty is less than or equal to a preset threshold, use the edge decoding result as the target processing result of the data to be processed; when the feature uncertainty is greater than the preset threshold, use the edge compression model to compress the intermediate features to obtain intermediate compressed features, and send the intermediate compressed features to the cloud server.

[0155] Next, the cloud server is used to decompress the intermediate compressed features using the cloud decompression model to obtain the intermediate decompressed features, select the corresponding cloud decoder according to the processing task, input the intermediate decompressed features into the cloud decoder for decoding, obtain the cloud decoding results, and use the cloud decoding results as the target processing results corresponding to the processing task.

[0156] In one embodiment, referring to Figure 12 , Figure 12This is a flow chart of the training process of the edge encoder, edge compression model, cloud decompression model, and cloud decoder in the cloud-edge collaborative data processing system provided by an embodiment of the present application, which specifically includes the following steps:

[0157] Step 1210: Obtain input sample data.

[0158] In one embodiment, information theory is used to train the model, so only input sample data similar to the data to be processed needs to be obtained.

[0159] Step 1220: During the training process, obtain intermediate training data corresponding to the input sample data, obtain transmission mutual information between the input sample data and the intermediate training data, and generate an information amount constraint based on the maximum information amount and the transmission mutual information.

[0160] Step 1230: Obtain cloud-based inference results corresponding to different processing tasks corresponding to the intermediate training data, generate inference mutual information between each cloud-based inference result and the intermediate training data, and maximize the inference mutual information based on the Lagrange multiplier corresponding to each processing task to obtain the compression target.

[0161] In one embodiment, assuming that the input sample data is x, the intermediate training data is y, and the cloud inference result obtained by the tth processing task is z (t) , the total number of processing tasks is T. At this time, the joint distribution of T types of processing tasks in the entire cloud-edge collaborative data processing process is expressed as:

[0162] p(x,y,z (1:T) )=p(x,y,z (1) ,z (2) ,..,z (T) )

[0163] Where p(·) is the probability density function.

[0164] Since the intermediate training data is a hidden representation of the input sample data, y cannot provide additional information to z when x is given. (t) , the process is modeled by mutual information as I(y; z (t) |x)=0, where I(·) is the mutual information calculation function.

[0165] Since mutual information is a measure of the interdependence between two random variables, this modeling process indicates that, given input sample data x, the mutual information between the intermediate training data y and the cloud-based inference result corresponding to any processing task (taking task t as an example) is zero. This is also equivalent to the three satisfying a Markov chain: Given x, y and z (t) They are independent of each other.

[0166] Next, we obtain the transmission mutual information between the input sample data and the intermediate training data, expressed as: I(x;y). The transmission mutual information is used to measure the degree of information sharing between the input sample data and the intermediate training data. At the same time, since the purpose of the embodiment of this application is to reduce the communication overhead between cloud-edge communication, it is necessary to limit the maximum amount of information flowing from x to y I c , the process is modeled as the maximum amount of information and the transmission mutual information to generate information constraints, the information constraint is expressed as: I(x; y) ≤ I c .

[0167] Next, to ensure that the cloud decoder can obtain the correct cloud inference result z (t) , the most effective information needs to be retained in y. First, the inference mutual information I(y; z) of the cloud inference result and the intermediate training data is obtained. (t) ), and then maximize the inference mutual information. The process is modeled as

[0168] In one embodiment, considering that different processing tasks have different weights in the cloud, a Lagrange multiplier β is assigned to each processing task. (t) , where β (t) >0 is used to ensure that the information contribution of each processing task is taken into account.

[0169] Then, based on the Lagrange multiplier, the compression objective obtained by maximizing the inference mutual information is expressed as:

[0170]

[0171] Step 1240: Obtain an optimization target according to the transmission mutual information and the compression target.

[0172] In one embodiment, the optimization objective is expressed as:

[0173]

[0174] Step 1250: Obtain the target upper bound corresponding to the loss function and the optimization target based on the input sample data, intermediate training data, and cloud inference results.

[0175] In one embodiment, the process of converting input sample data into intermediate training data at the edge is modeled as p(y|x), where p(·|·) is the conditional probability density function, and the process of converting intermediate training data into the cloud-based inference result of the t-th processing task in the cloud is modeled as p(z (t) |y). Then, based on the prior knowledge of the experiment, we use the probability distribution r(y) and q(z (t) |y) to approximate p(y) and p(z (t) |y), the upper bound of the optimization target is expressed as:

[0176]

[0177] The upper bound of the target consists of two integral terms. The first integral term represents the joint distribution of input sample data, intermediate training data, and cloud inference results, and the approximate conditional distribution q(z (t) |y), and the following risk represents the relationship between the distribution of input sample data, intermediate training data, and the approximate distribution r(y).

[0178] Step 1260: Under the premise of satisfying the target upper bound and information constraint, minimize the loss value corresponding to the loss function and train the edge encoder, edge compression model, cloud decompression model and cloud decoder.

[0179] In one embodiment, with the target upper bound and the information constraint, the model parameters can be adjusted so as not to exceed the target upper bound and to minimize the loss value corresponding to the loss function while satisfying the information constraint.

[0180] Among them, the loss function of T processing tasks is expressed as:

[0181]

[0182] Where r(·) and q(·) are probability density functions, ψ is the parameter of r(y), is the quantization result corresponding to the intermediate training data, represents the expectation under the probability distribution p(x). The first part of the above loss function is the expected log-likelihood term, which is used to ensure that the quantization results can be effectively generated from the input sample data. The second part ensures that the cloud-based inference results of each processing task can be accurately predicted from the quantization results.

[0183] In one embodiment, the edge processing process is modeled as follows:

[0184]

[0185] y=E(x;φ e ).

[0186] Among them, E(x;φ e ) is the parameter of the edge end φ e The edge encoder is used to map high-dimensional data to the latent space y and obtain the

[0187] In one embodiment, the cloud processing process is modeled as follows:

[0188] The classification tasks are:

[0189]

[0190] The reconstruction tasks are:

[0191]

[0192] in, The parameter is θ (t) Cloud decoder, I cls is the 0-1 characteristic function of category j, and N is the normal distribution.

[0193] It can be seen from the above training process that the compression process of the edge compression module of the embodiment of the present application is related to the results of the edge encoder training. The training target is obtained through the probabilistic modeling of information theory to compress the most useful data. For example, in the task of image classification, the background of the image is irrelevant information, and the object is useful information. Therefore, through the training process, the edge compression module is combined with the edge encoder to remove irrelevant information as much as possible to achieve the maximum compression effect. And this end-to-end training method under a unified framework, the cloud-edge collaborative data processing system can automatically learn the feature representation and compression strategy that is most suitable for a specific task while maintaining high efficiency.

[0194] The cloud-edge collaborative data processing method of the present application makes intermediate feature data compression-friendly through joint optimization. After quantization, the intermediate features are directly compressed and transmitted to the cloud through lossless entropy coding. The cloud decoder then performs inference based on the intermediate features to obtain the final result. This method retains the most effective information when compressing intermediate features, saving bandwidth resources for communication between the edge and the cloud and reducing inference latency.

[0195] The following describes the experimental verification results of the cloud-edge collaborative data processing method in the embodiment of the present application.

[0196] Reference Figure 13 , Figure 13 These are the accuracy experimental results for image classification and reconstruction tasks provided by the embodiments of this application. The embodiments of this application use the Imagenette dataset as an example to conduct experiments on image classification and reconstruction tasks. The experimental data shows that the uncompressed intermediate data has the largest amount of data. The experimental results show that, with an acceptable loss in task results, compressing the intermediate features saves a significant amount of communication overhead compared to the uncompressed process, and that the compression process with added side information achieves a higher degree of compression than the compression process without side information.

[0197] Reference Figure 14 , Figure 14It is the accuracy loss of the cloud-edge collaborative dynamic exit mechanism under different network availability rates provided by the embodiment of the present application. In this embodiment, the progress loss of the cloud-edge collaborative dynamic exit mechanism is verified under different network availability rates, such as 100%, 90%, 80%, and 70%. The dataset is still the Imagenette dataset. Taking the classification task as an example, the success rate of the task when further reasoning is performed through the cloud is verified. Since the classification accuracy of the lightweight edge decoder deployed at the edge is 15.38% lower than that of the cloud, the cloud-edge collaborative dynamic exit mechanism allows the relevant data that is easy to process at the edge to be retained at the edge and exit, and the data that is more difficult to process is sent to the cloud after compression. According to the experimental results, it can be seen that the communication overhead is reduced within the acceptable overall task accuracy loss.

[0198] The technical solution provided by the embodiments of the present application obtains data to be processed from a terminal and feeds it into an edge encoder for feature extraction to obtain intermediate features. Next, for each processing task, the intermediate features are input into a lightweight edge decoder for decoding to obtain an edge decoding result. The entropy value of the edge decoding result is calculated as the feature uncertainty. When the feature uncertainty is less than or equal to a preset threshold, the edge decoding result is used as the target processing result for the processing task corresponding to the data to be processed. When the feature uncertainty is greater than the preset threshold, the intermediate features are compressed using an edge compression model to obtain intermediate compressed features. These intermediate compressed features are then sent to a cloud server in a cloud-edge collaborative data processing system. The intermediate compressed features are decompressed on the cloud server to obtain intermediate decompressed features, which are then decoded to obtain cloud decoding results. The cloud decoding results are used as the target processing result for the processing task. In the embodiments of the present application, the cloud-edge collaborative data processing model is segmented, with one processing process implemented at the edge and the other at the cloud. Through this segmentation process, the data to be processed is converted into intermediate features of smaller dimensions, which are then compressed to further reduce the amount of data sent from the edge to the cloud, thereby reducing communication overhead. In addition, for different processing tasks, a lightweight model is used in advance to calculate a quick result, and the entropy value of the result is used to determine whether to select the processing result on the edge or the cloud. This flexible cloud-edge collaborative dynamic exit mechanism is used to selectively compress part of the data from the full data for transmission, which can further reduce the data volume and communication overhead.

[0199] An embodiment of the present application further provides an electronic device, including:

[0200] at least one memory;

[0201] at least one processor;

[0202] at least one program;

[0203] The program is stored in the memory, and the processor executes the at least one program to implement the cloud-edge collaborative data processing method described above. The electronic device can be any smart terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), an in-vehicle computer, etc.

[0204] See also Figure 15 , Figure 15 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0205] The processor 1501 may be implemented as a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0206] The memory 1502 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1502 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1502, and the processor 1501 calls and executes the cloud-edge collaborative data processing method of the embodiment of this application;

[0207] Input / output interface 1503, used to implement information input and output;

[0208] Communication interface 1504, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.); and

[0209] Bus 1505 , which transmits information between various components of the device (e.g., processor 1501 , memory 1502 , input / output interface 1503 , and communication interface 1504 );

[0210] The processor 1501 , the memory 1502 , the input / output interface 1503 and the communication interface 1504 are connected to each other in communication within the device via the bus 1505 .

[0211] An embodiment of the present application also provides a storage medium, which is a storage medium that stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned cloud-edge collaborative data processing method.

[0212] The memory, as a non-transient storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0213] The cloud-edge collaborative data processing method, system, device and storage medium proposed in the embodiment of the present application obtains the data to be processed from the terminal, and sends the data to be processed into the edge encoder for feature extraction to obtain intermediate features. Next, for each processing task, the intermediate features are input into the lightweight edge decoder for decoding to obtain the edge decoding result, and the entropy value of the edge decoding result is calculated as the feature uncertainty. When the feature uncertainty is less than or equal to the preset threshold, the edge decoding result is used as the target processing result of the processing task corresponding to the data to be processed. When the feature uncertainty is greater than the preset threshold, the edge compression model is used to compress the intermediate features to obtain intermediate compressed features, and the intermediate compressed features are sent to the cloud server in the cloud-edge collaborative data processing system. The intermediate compressed features are decompressed on the cloud server to obtain intermediate decompressed features, and then the intermediate decompressed features are decoded to obtain cloud decoding results, and the cloud decoding results are used as the target processing results corresponding to the processing task. In the embodiment of the present application, the cloud-edge collaborative data processing model is divided, and part of the processing process is implemented at the edge end, and the other part is implemented in the cloud. Through this segmentation process, the data to be processed is converted into intermediate features of smaller dimensions. These intermediate features are then compressed to further reduce the amount of data sent from the edge to the cloud, thereby reducing communication overhead. Furthermore, for different processing tasks, a lightweight model is pre-calculated to quickly generate a result. The entropy of this result is used to determine whether to process the result at the edge or in the cloud. This flexible cloud-edge collaborative dynamic exit mechanism allows for selective compression of partial data from the full data set for transmission, further reducing both data volume and communication overhead.

[0214] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0215] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0216] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0217] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0218] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0219] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0220] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0221] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0222] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0223] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0224] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A cloud-edge collaborative data processing method, characterized in that: An edge server applied to a cloud-edge collaborative data processing system, wherein an edge encoder, at least one lightweight edge decoder corresponding to different processing tasks, and an edge compression model are deployed on the edge server, and the method includes: Acquire data to be processed from a terminal, and send the data to be processed to the edge encoder for feature extraction to obtain intermediate features; For each of the processing tasks, inputting the intermediate features into the lightweight edge decoder for decoding to obtain an edge decoding result, and calculating an entropy value of the edge decoding result as a feature uncertainty; When the feature uncertainty is less than or equal to a preset threshold, taking the edge decoding result as the target processing result of the to-be-processed data corresponding to the processing task; When the feature uncertainty is greater than the preset threshold, the edge compression model is used to compress the intermediate feature to obtain an intermediate compressed feature, and the intermediate compressed feature is sent to a cloud server in the cloud-edge collaborative data processing system; Among them, a cloud decompression model and a cloud decoder that together with the edge encoder constitute a cloud-edge collaborative data processing model are deployed on the cloud server. The lightweight edge decoder corresponds to the cloud decoder. The cloud decompression model is used to decompress the intermediate compression features to obtain intermediate decompressed features. The cloud decoder is used to decode the intermediate decompressed features to obtain cloud decoding results, and use the cloud decoding results as the target processing results corresponding to the processing task.

2. The cloud-edge collaborative data processing method according to claim 1, characterized in that: The edge compression model includes at least a quantization model, and the step of compressing the intermediate features using the edge compression model to obtain the intermediate compressed features includes: quantizing the intermediate features using the quantization model to obtain intermediate quantized features; The intermediate quantized features are arithmetically encoded to obtain the intermediate compressed features of the binary bit string.

3. The cloud-edge collaborative data processing method according to claim 2, characterized in that: The edge compression model includes at least a side information encoder, a statistical parameter encoder, and a side information decoder. The method of using the edge compression model to perform feature compression on the intermediate features to obtain intermediate compressed features includes: Inputting the intermediate features into the side information encoder for feature correction to obtain statistical hidden features, and quantizing the statistical hidden features to obtain quantized hidden features; Inputting the quantized hidden features into the statistical parameter encoder for encoding to obtain side information compression features; Inputting the quantized hidden features into the side information decoder for decoding to obtain statistical parameters, and using the statistical parameters to correct the intermediate quantized features to obtain simulated compressed features; The intermediate compression feature is obtained according to the simulation compression feature and the side information compression feature.

4. The cloud-edge collaborative data processing method according to claim 3, characterized in that: The method further comprises: Using at least one of the edge encoder, the lightweight edge decoder, and the edge compression model as a first basic network; Obtain at least one of a first convolutional layer parameter of a convolutional layer or a first fully connected layer parameter of a fully connected layer in the first basic network, and perform singular value decomposition on the first convolutional layer parameter and / or the first fully connected layer parameter to obtain a first singular value matrix and a first default matrix; Obtaining a preset first variation, obtaining a first compression matrix according to the first variation and the first singular value matrix, and obtaining first compression model parameters according to the first compression matrix and the first default matrix; Parameters of the first basic network are updated based on the first compression model parameters to obtain an updated first basic network.

5. A cloud-edge collaborative data processing method, characterized in that: A cloud server in a cloud-edge collaborative data processing system is applied, wherein a cloud decompression model and a cloud decoder that together with the edge encoder constitute a cloud-edge collaborative data processing model are deployed on the cloud server, and the method includes: Receiving intermediate compression features sent from an edge server, wherein the intermediate compression features are obtained by the cloud-edge collaborative data processing method according to any one of claims 1 to 4; Decompressing the intermediate compressed features using the cloud-based decompression model to obtain intermediate decompressed features; The corresponding cloud decoder is selected according to the processing task, the intermediate decompression feature is input into the cloud decoder for decoding, a cloud decoding result is obtained, and the cloud decoding result is used as the target processing result corresponding to the processing task.

6. The cloud-edge collaborative data processing method according to claim 5, characterized in that: The cloud-based decoding model includes at least a cloud-based side information decoder and a statistical parameter decoder. When the intermediate compression feature is obtained from the analog compression feature and the side information compression feature, decompressing the intermediate compression feature using the cloud-based decoding model to obtain the intermediate decompressed feature includes: Obtaining the simulation compression feature and the side information compression feature from the intermediate compression feature; Inputting the side information compression feature into the statistical parameter decoder for decoding to obtain a decoded hidden feature; Inputting the decoded hidden features into the cloud-side information decoder for decoding to obtain cloud-side statistical parameters; The simulated compression feature is arithmetically decoded using the cloud statistical parameters to obtain the intermediate decompression feature.

7. The cloud-edge collaborative data processing method according to claim 6, characterized in that: The method further comprises: Using at least one of the cloud-based decompression model or the cloud-based decoder as a second basic network; Obtain at least one of a second convolutional layer parameter of the convolutional layer or a second fully connected layer parameter of the fully connected layer in the second basic network, and perform singular value decomposition on the second convolutional layer parameter and / or the second fully connected layer parameter to obtain a second singular value matrix and a second default matrix; Obtaining a preset second variation, obtaining a second compression matrix according to the second variation and the second singular value matrix, and obtaining second compression model parameters according to the second compression matrix and the second default matrix; Parameters of the second basic network are updated based on the second compression model parameters to obtain an updated second basic network.

8. A cloud-edge collaborative data processing system, characterized in that: include: An edge server, wherein an edge encoder, at least one lightweight edge decoder corresponding to different processing tasks, and an edge compression model are deployed on the edge server; A cloud server, wherein a cloud decompression model and a cloud decoder are deployed on the cloud server, the cloud decoder and the edge encoder together constitute a cloud collaborative data processing model, and the cloud decoder corresponds to the lightweight edge decoder; The edge server is configured to use the edge encoder to perform feature extraction on the data to be processed from the terminal to obtain intermediate features, use the lightweight edge decoder to decode the intermediate feature input to obtain an edge decoding result, calculate the entropy value of the edge decoding result as a feature uncertainty, and when the feature uncertainty is less than or equal to a preset threshold, use the edge decoding result as a target processing result of the data to be processed; when the feature uncertainty is greater than the preset threshold, use the edge compression model to perform feature compression on the intermediate features to obtain intermediate compressed features, and send the intermediate compressed features to the cloud server; The cloud server is used to decompress the intermediate compression features using the cloud decompression model to obtain intermediate decompression features, select the corresponding cloud decoder according to the processing task, input the intermediate decompression features into the cloud decoder for decoding, obtain a cloud decoding result, and use the cloud decoding result as the target processing result corresponding to the processing task.

9. The cloud-edge collaborative data processing system according to claim 8, characterized in that: The training process of the edge encoder, the edge compression model, the cloud decompression model, and the cloud decoder includes the following steps: Get input sample data; During the training process, obtaining intermediate training data corresponding to the input sample data, obtaining transmission mutual information between the input sample data and the intermediate training data, and generating an information amount constraint based on the maximum information amount and the transmission mutual information; Obtaining cloud-based inference results corresponding to different processing tasks corresponding to the intermediate training data, generating inference mutual information between each cloud-based inference result and the intermediate training data, and maximizing the inference mutual information based on a Lagrange multiplier corresponding to each processing task to obtain a compression target; Obtaining an optimization target according to the transmission mutual information and the compression target; Obtaining a loss function and a target upper bound corresponding to the optimization target based on the input sample data, the intermediate training data, and the cloud-based inference result; Under the premise of satisfying the target upper bound and the information amount constraint, the loss value corresponding to the loss function is minimized, and the edge encoder, the edge compression model, the cloud decompression model and the cloud decoder are trained.

10. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the cloud-edge collaborative data processing method described in any one of claims 1 to 7 when executing the computer program.

11. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the cloud-edge collaborative data processing method described in any one of claims 1 to 7 is implemented.

Images