Cloud edge collaborative data processing method, system and device and storage medium

Abstract

The embodiment of the invention provides a cloud edge collaborative data processing method, system and device and a storage medium, and relates to the technical field of communication. The method comprises the following steps: extracting an intermediate feature of to-be-processed data in an edge encoder, inputting the intermediate feature into a lightweight edge decoder for decoding to obtain an edge decoding result, taking the edge decoding result as a target processing result when the feature uncertainty of the edge decoding result is less than or equal to a preset threshold, and taking the edge decoding result as the target processing result when the feature uncertainty is greater than the preset threshold. And compressing the intermediate features by using an edge compression model, and sending the compressed intermediate features to a cloud server in the cloud edge collaborative data processing system. And the cloud edge collaborative data processing model is segmented, and intermediate features with smaller dimensions are compressed, so that the amount of data sent from the edge end to the cloud end is further reduced, and the communication overhead is reduced. And a cloud edge collaborative dynamic exit mechanism is utilized to selectively compress part of data from total data for transmission, so that the data volume and the communication overhead are reduced.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to cloud-edge collaborative data processing methods, systems, devices and storage media. Background Art

[0002] Cloud-edge collaborative reasoning is an architecture in the field of edge computing that combines the advantages of cloud computing and edge computing. It processes and analyzes data separately on the cloud and edge to achieve more efficient and low-latency data processing and decision-making processes.

[0003] In related technologies, since the computing power of the edge is usually lower than that of the cloud, the edge usually performs simple preprocessing on the input data, and then sends the data to the cloud through data compression, and the cloud completes most of the processing work. However, this processing method results in a large amount of data transmission and high communication overhead in the collaborative process between the cloud and the edge. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a cloud-edge collaborative data processing method, system, device and storage medium to reduce the communication overhead of cloud-edge collaborative processing.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a cloud-edge collaborative data processing method, which is applied to an edge server in a cloud-edge collaborative data processing system, wherein an edge encoder, at least one lightweight edge decoder corresponding to different processing tasks, and an edge compression model are deployed on the edge server, and the method includes:

[0006] Acquire the data to be processed from the terminal, and send the data to be processed to the edge encoder for feature extraction to obtain intermediate features;

[0007] For each of the processing tasks, inputting the intermediate features into the lightweight edge decoder for decoding to obtain an edge decoding result, and calculating an entropy value of the edge decoding result as a feature uncertainty;

[0008] When the feature uncertainty is less than or equal to a preset threshold, the edge decoding result is used as a target processing result of the to-be-processed data corresponding to the processing task;

[0009] When the feature uncertainty is greater than the preset threshold, the edge compression model is used to compress the intermediate feature to obtain an intermediate compressed feature, and the intermediate compressed feature is sent to a cloud server in the cloud-edge collaborative data processing system;

[0010] Among them, a cloud decompression model and a cloud decoder which together with the edge encoder constitute a cloud-edge collaborative data processing model are deployed on the cloud server. The lightweight edge decoder corresponds to the cloud decoder. The cloud decompression model is used to decompress the intermediate compression features to obtain intermediate decompression features. The cloud decoder is used to decode the intermediate decompression features to obtain cloud decoding results, and use the cloud decoding results as the target processing results corresponding to the processing task.

[0011] In some embodiments, the edge compression model includes at least a quantization model, and the step of compressing the intermediate features using the edge compression model to obtain the intermediate compressed features includes:

[0012] quantizing the intermediate features using the quantization model to obtain intermediate quantized features;

[0013] The intermediate quantized features are arithmetically encoded to obtain the intermediate compressed features of the binary bit string.

[0014] In some embodiments, the edge compression model includes at least a side information encoder, a statistical parameter encoder, and a side information decoder, and the step of compressing the intermediate features using the edge compression model to obtain the intermediate compressed features includes:

[0015] Inputting the intermediate feature into the side information encoder for feature correction to obtain a statistical hidden feature, and quantizing the statistical hidden feature to obtain a quantized hidden feature;

[0016] Inputting the quantized hidden features into the statistical parameter encoder for encoding to obtain side information compression features;

[0017] Inputting the quantized hidden features into the side information decoder for decoding to obtain statistical parameters, and using the statistical parameters to correct the intermediate quantized features to obtain simulated compressed features;

[0018] The intermediate compression feature is obtained according to the simulation compression feature and the side information compression feature.

[0019] In some embodiments, the method further comprises:

[0020] Using at least one of the edge encoder, the lightweight edge decoder and the edge compression model as a first basic network;

[0021] Obtain at least one of a first convolutional layer parameter of a convolutional layer or a first fully connected layer parameter of a fully connected layer in the first basic network, and perform singular value decomposition on the first convolutional layer parameter and / or the first fully connected layer parameter to obtain a first singular value matrix and a first default matrix;

[0022] Obtaining a preset first variation, obtaining a first compression matrix according to the first variation and the first singular value matrix, and obtaining a first compression model parameter according to the first compression matrix and the first default matrix;

[0023] The first basic network is updated based on the first compression model parameters to obtain an updated first basic network.

[0024] To achieve the above-mentioned purpose, the second aspect of the embodiment of the present application proposes a cloud-edge collaborative data processing method, which is applied to a cloud server in a cloud-edge collaborative data processing system, wherein a cloud decompression model and a cloud decoder that together with the edge encoder constitute a cloud-edge collaborative data processing model are deployed on the cloud server, and the method includes:

[0025] Receiving an intermediate compression feature sent from an edge server, wherein the intermediate compression feature is obtained by the cloud-edge collaborative data processing method described in any one of the first aspects;

[0026] Decompressing the intermediate compressed features using the cloud decoding model to obtain intermediate decompressed features;

[0027] The corresponding cloud decoder is selected according to the processing task, the intermediate decompression feature is input into the cloud decoder for decoding, a cloud decoding result is obtained, and the cloud decoding result is used as the target processing result corresponding to the processing task.

[0028] In some embodiments, the cloud decoding model includes at least a cloud side information decoder and a statistical parameter decoder. When the intermediate compression feature is obtained by the analog compression feature and the side information compression feature, the cloud decoding model is used to decompress the intermediate compression feature to obtain the intermediate decompression feature, including:

[0029] Acquire the simulation compression feature and the side information compression feature from the intermediate compression feature;

[0030] Inputting the side information compression feature into the statistical parameter decoder for decoding to obtain a decoded hidden feature;

[0031] Inputting the decoded hidden features into the cloud side information decoder for decoding to obtain cloud side statistical parameters;

[0032] The simulated compression feature is arithmetically decoded using the cloud statistical parameter to obtain the intermediate decompression feature.

[0033] In some embodiments, the method further comprises:

[0034] Using at least one of the cloud decompression model or the cloud decoder as a second basic network;

[0035] Obtain at least one of a second convolutional layer parameter of the convolutional layer or a second fully connected layer parameter of the fully connected layer in the second basic network, and perform singular value decomposition on the second convolutional layer parameter and / or the second fully connected layer parameter to obtain a second singular value matrix and a second default matrix;

[0036] Obtaining a preset second variation, obtaining a second compression matrix according to the second variation and the second singular value matrix, and obtaining second compression model parameters according to the second compression matrix and the second default matrix;

[0037] The second basic network is updated based on the second compression model parameters to obtain an updated second basic network.

[0038] To achieve the above objectives, a third aspect of an embodiment of the present application proposes a cloud-edge collaborative data processing system, including:

[0039] An edge server, wherein an edge encoder, at least one lightweight edge decoder corresponding to different processing tasks, and an edge compression model are deployed on the edge server;

[0040] A cloud server, wherein a cloud decompression model and a cloud decoder are deployed on the cloud server, wherein the cloud decoder and the edge encoder together constitute a cloud collaborative data processing model, and the cloud decoder corresponds to the lightweight edge decoder;

[0041] The edge server is used to use the edge encoder to extract features from the data to be processed from the terminal to obtain intermediate features, use the lightweight edge decoder to decode the intermediate feature input to obtain an edge decoding result, calculate the entropy value of the edge decoding result as a feature uncertainty, when the feature uncertainty is less than or equal to a preset threshold, use the edge decoding result as a target processing result of the data to be processed, when the feature uncertainty is greater than the preset threshold, use the edge compression model to perform feature compression on the intermediate features to obtain intermediate compressed features, and send the intermediate compressed features to the cloud server;

[0042] The cloud server is used to decompress the intermediate compression features using the cloud decompression model to obtain intermediate decompression features, select the corresponding cloud decoder according to the processing task, input the intermediate decompression features into the cloud decoder for decoding, obtain a cloud decoding result, and use the cloud decoding result as the target processing result corresponding to the processing task.

[0043] In some embodiments, the training process of the edge encoder, the edge compression model, the cloud decompression model, and the cloud decoder includes the following steps:

[0044] Get input sample data;

[0045] During the training process, intermediate training data corresponding to the input sample data is obtained, transmission mutual information between the input sample data and the intermediate training data is obtained, and an information amount constraint is generated according to the maximum amount of information and the transmission mutual information;

[0046] Obtain cloud-based inference results corresponding to different processing tasks corresponding to the intermediate training data, generate inference mutual information between each of the cloud-based inference results and the intermediate training data, and maximize the inference mutual information based on the Lagrange multiplier corresponding to each of the processing tasks to obtain a compression target;

[0047] Obtaining an optimization target according to the transmission mutual information and the compression target;

[0048] Obtaining a loss function and a target upper bound corresponding to the optimization target based on the input sample data, the intermediate training data, and the cloud inference result;

[0049] Under the premise of satisfying the target upper bound and the information amount constraint, the loss value corresponding to the loss function is minimized, and the edge encoder, the edge compression model, the cloud decompression model and the cloud decoder are trained.

[0050] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the method described in the first aspect or the second aspect when executing the computer program.

[0051] To achieve the above-mentioned purpose, the first aspect of an embodiment of the present application proposes a storage medium, which is a storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0052] The cloud-edge collaborative data processing method, system, device and storage medium proposed in the embodiment of the present application obtain the data to be processed from the terminal, and send the data to be processed to the edge encoder for feature extraction to obtain intermediate features. Next, for each processing task, the intermediate features are input into the lightweight edge decoder for decoding to obtain the edge decoding result, and the entropy value of the edge decoding result is calculated as the feature uncertainty. When the feature uncertainty is less than or equal to the preset threshold, the edge decoding result is used as the target processing result of the processing task corresponding to the data to be processed. When the feature uncertainty is greater than the preset threshold, the edge compression model is used to compress the intermediate features to obtain intermediate compressed features, and the intermediate compressed features are sent to the cloud server in the cloud-edge collaborative data processing system. The intermediate compressed features are decompressed on the cloud server to obtain intermediate decompressed features, and then the intermediate decompressed features are decoded to obtain cloud decoding results, and the cloud decoding results are used as the target processing results corresponding to the processing task. In the embodiment of the present application, the cloud-edge collaborative data processing model is divided, and part of the processing process is implemented at the edge end, and the other part is implemented in the cloud. Through this segmentation process, the data to be processed is converted into intermediate features with smaller dimensions, and then these intermediate features are compressed to further reduce the amount of data sent from the edge to the cloud, thereby reducing communication overhead. In addition, for different processing tasks, a lightweight model is used in advance to calculate a fast result, and the entropy value of the result is used to determine whether to choose the processing result at the edge or the processing result at the cloud. This flexible cloud-edge collaborative dynamic exit mechanism is used to selectively compress part of the data from the full amount of data for transmission, which can further reduce the amount of data and communication overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Schematic diagram of the principle of cloud-edge collaborative reasoning in related technologies.

[0054] Figure 2 It is a schematic diagram of the principles of the cloud-edge collaborative data processing system provided in an embodiment of the present application.

[0055] Figure 3 It is a structural diagram of the cloud-edge collaborative data processing system provided in an embodiment of the present application.

[0056] Figure 4 It is a flowchart of the cloud-edge collaborative data processing method provided in an embodiment of the present application applied to an edge server.

[0057] Figure 5 It is a schematic diagram of the cloud-edge collaborative dynamic exit mechanism provided in an embodiment of the present application.

[0058] Figure 6 It is a flow chart of the cloud-edge collaborative data processing method provided in an embodiment of the present application.

[0059] Figure 7 It is a schematic diagram of using an edge compression model to compress intermediate features to obtain intermediate compressed features provided in an embodiment of the present application.

[0060] Figure 8 This is a flow chart of adaptively adjusting the compression rate of the edge end provided in an embodiment of the present application.

[0061] Fig. 9 It is a schematic diagram of the compression rate adjustment of the first basic network provided in an embodiment of the present application.

[0062] Fig.10 It is an optional flowchart of the cloud-edge collaborative data processing method provided in an embodiment of the present application applied to a cloud server.

[0063] Fig.11 It is a flow chart of using the cloud decoding model to decompress the intermediate compressed features to obtain the intermediate decompressed features.

[0064] Fig.12 It is a flow chart of the training process of the edge encoder, edge compression model, cloud decompression model and cloud decoder in the cloud-edge collaborative data processing system provided in an embodiment of the present application.

[0065] Fig.13 This is the accuracy experimental result of the image classification and reconstruction task provided by the embodiment of the present application.

[0066] Fig.14 It is the accuracy loss of the cloud-edge collaborative dynamic exit mechanism under different network availability rates provided in the embodiment of the present application.

[0067] Fig.15 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0069] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0071] First, some nouns involved in this application are analyzed:

[0072] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0073] Cloud-edge collaborative reasoning is an architecture in the field of edge computing that combines the advantages of cloud computing and edge computing. It processes and analyzes data separately on the cloud and edge to achieve more efficient and low-latency data processing and decision-making processes.

[0074] In related technologies, since the computing power of the edge is usually lower than that of the cloud, the edge usually performs simple preprocessing on the input data, and then sends the data to the cloud through data compression, and the cloud completes most of the processing work. Figure 1 , Figure 1 Schematic diagram of the principle of cloud-edge collaborative reasoning in related technologies. Figure 1 In the process, the input data is compressed at the edge, and then the compressed data is sent to the cloud. The cloud then performs a decompression process to obtain the input data, and then inputs the input data into the neural network model for processing to obtain the task result. Even if the neural network model is directly segmented and deployed on the edge and cloud respectively, and part of the neural network is offloaded from the edge to the cloud, communication overhead will still be introduced in the neural network reasoning process. This is mainly because when the edge transmits the intermediate feature data of the model segmentation point to the cloud, the size of the intermediate feature data and the quality of the communication connection directly affect the reasoning delay. Therefore, in the related technology, the communication overhead is high in the collaborative process between the cloud and the edge.

[0075] Based on this, the embodiments of the present application provide a cloud-edge collaborative data processing method, system, device and storage medium, by dividing the cloud-edge collaborative data processing model, part of the processing process is implemented at the edge, and the other part is implemented in the cloud. Through this segmentation process, the data to be processed is converted into intermediate features with smaller dimensions, and then these intermediate features are compressed to further reduce the amount of data sent from the edge to the cloud, thereby reducing communication overhead. In addition, for different processing tasks, a lightweight model is used in advance to calculate a quick result, and the entropy value of the result is used to determine whether to select the processing result of the edge or the processing result of the cloud. This flexible cloud-edge collaborative dynamic exit mechanism is used to selectively compress part of the data from the full amount of data for transmission, which can further reduce the amount of data and communication overhead.

[0076] The embodiments of the present application provide a cloud-edge collaborative data processing method, system, device and storage medium, which are specifically illustrated by the following embodiments. First, the cloud-edge collaborative data processing method in the embodiments of the present application is described.

[0077] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.

[0078] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0079] The cloud-edge collaborative data processing method provided in the embodiment of the present application relates to the field of communication technology. The cloud-edge collaborative data processing method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be a computer program running in a terminal or a server side. For example, a computer program can be a native program or software module in an operating system; it can be a local (Native) application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as a client that supports cloud-edge collaborative data processing, or it can be a small program, that is, a program that can be run only by downloading it to a browser environment; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plug-in. Among them, the terminal communicates with the server through a network. The cloud-edge collaborative data processing method can be executed by a terminal or a server, or by a terminal and a server in collaboration.

[0080] In some embodiments, the terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer or a smart watch, etc. In addition, the terminal may also be an intelligent vehicle-mounted device. The intelligent vehicle-mounted device applies the cloud-edge collaborative data processing method of this embodiment to provide related services and enhance the driving experience. The server may be an independent server, or it may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms; it may also be a service node in a blockchain system, and each service node in the blockchain system forms a peer-to-peer (P2P, Peer To Peer, P2P) network, and the P2P protocol is an application layer protocol running on the Transmission Control Protocol (Transmission Control Protocol, TCP) protocol. The terminal and the server may be connected via Bluetooth, Universal Serial Bus (USB) or network communication connection methods, which are not limited in this embodiment.

[0081] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0082] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0083] First, the cloud-edge collaborative data processing system provided by the embodiment of the present application is described. Figure 2 , Figure 2 It is a principle schematic diagram of the cloud-edge collaborative data processing system provided by the embodiment of the present application. Among them, the cloud-edge collaborative data processing model is divided into two parts, one of which is deployed on the edge server (hereinafter referred to as the edge), and the other is deployed on the cloud server (hereinafter referred to as the cloud). The input data in the processing process of the edge is processed into low-dimensional intermediate features, and then the intermediate features are compressed, and the compressed data is transmitted to the cloud. The cloud decompresses the compressed data, obtains the intermediate features, and then uses the model of the other part to process the intermediate features to obtain the final task results.

[0084] In one embodiment, referring to Figure 3 , Figure 3 It is a structural diagram of the cloud-edge collaborative data processing system provided in an embodiment of the present application. Figure 3The terminal acts as a sensor to collect data and obtain different data to be processed. The data to be processed here can be collected pictures, audio or video, and each type of data to be processed can correspond to at least one different processing task. That is to say, in the embodiment of the present application, multi-task processing is performed on the data to be processed. Taking pictures as an example, the processing tasks can be image recognition, image reconstruction, etc. The embodiment of the present application does not limit the processing tasks corresponding to the data to be processed.

[0085] In one embodiment, the terminal sends the collected data to be processed to the edge end for processing via a local area network or other means. The edge end processes the data to be processed to obtain intermediate features, compresses the data to obtain intermediate compressed features, and then sends the intermediate compressed features to the cloud for subsequent processing. The cloud decompresses the intermediate compressed features to obtain corresponding intermediate features, and then processes the intermediate features according to different processing tasks to obtain cloud decoding results corresponding to different processing tasks. In addition, the task processing in the cloud is a multi-task processing process. Each processing task corresponds to a cloud decoder, so the intermediate features are input into different cloud decoders for processing.

[0086] Among them, an edge encoder, at least one lightweight edge decoder corresponding to different processing tasks and an edge compression model are deployed in the edge end, and a cloud decompression model and a cloud decoder are deployed on the cloud server.

[0087] In one embodiment, the cloud-edge collaborative processing model is segmented to obtain an edge encoder and a cloud decoder, wherein the edge encoder is deployed on the edge and the cloud decoder is deployed on the cloud. The embodiment of the present application does not specifically limit the segmentation point, that is, the cloud-edge collaborative processing model can be segmented into two parts according to any segmentation method, the part that needs to be deployed on the edge is called the edge encoder, and the part that needs to be deployed on the cloud is called the cloud decoder. In specific applications, the cloud-edge collaborative processing model is segmented, and then a lightweight edge decoder and an edge compression model are added behind the segmented edge encoder, and deployed together on the edge. A cloud decompression model is added in front of the segmented cloud decoder, and they are deployed together on the cloud.

[0088] In one embodiment, the most appropriate split point can also be determined according to the computing power of the edge and the cloud, and the computational complexity of the edge encoder obtained by segmentation according to the determined split point is less than the computational complexity of the cloud decoder. Since the edge compression model and the cloud decompression model are set, when determining the split point, the embodiment of the present application does not need to consider whether the data dimension of the intermediate feature is smaller than the data to be processed, so that the range of the split point becomes larger and is suitable for different cloud-edge collaborative processing models.

[0089] In addition, the embodiments of the present application also control the compression rate of different models in the cloud-edge collaborative data processing system, thereby further reducing the amount of data to be processed and reducing communication overhead.

[0090] The following describes the cloud-edge collaborative data processing method in an embodiment of the present application in conjunction with the above figure.

[0091] Figure 4 is an optional flow chart of the cloud-edge collaborative data processing method provided in an embodiment of the present application applied to an edge server. Figure 4 The method may include but is not limited to steps 110 to 140. It can also be understood that this embodiment Figure 4 The order of step 110 to step 140 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0092] Step 110: Obtain the data to be processed from the terminal, and send the data to be processed to the edge encoder for feature extraction to obtain intermediate features.

[0093] In one embodiment, in combination Figure 3 At the edge, edge encoders are used to extract features from the data to be processed, converting the features to be processed into more compact and information-rich intermediate features. The data dimension is reduced by feature extraction, and only key features are extracted for subsequent possible cloud computing, which can greatly reduce the bandwidth requirements for data transmission. The cloud does not need to process a large amount of raw data, and can make decisions and respond faster.

[0094] Step 120: For each processing task, the intermediate features are input into a lightweight edge decoder for decoding to obtain an edge decoding result, and the entropy value of the edge decoding result is calculated as the feature uncertainty.

[0095] In one embodiment, due to the different computational complexities of multiple processing tasks and the difference in computing power between the cloud and the edge, the cloud has powerful computing power and large-capacity storage, which is suitable for processing complex analysis and large data storage tasks; while the edge has lower processing latency, which is suitable for rapid response and processing of real-time data. If all processing tasks are performed in the cloud, it will not only cause a waste of computing power on the edge, but also affect the overall processing efficiency. Therefore, in the embodiment of the present application, a choice is made among the processing tasks of the intermediate features to determine which processing tasks are performed by the cloud and which are performed by the edge, so as to reasonably allocate the processing tasks of the cloud and the edge, so as to optimize the computing power consumption in the cloud-edge collaboration process and maximize the overall budget efficiency.

[0096] In one embodiment, a lightweight edge decoder consistent with the cloud decoder in the cloud is deployed at the edge, that is, there are corresponding lightweight edge decoders for as many cloud decoders as there are. The difference between the lightweight edge decoder and the cloud decoder is that the processing tasks are the same, but the model structure of the lightweight edge decoder is lighter, the processing efficiency is higher, and the response speed is faster.

[0097] Among them, for the tth processing task, the tth cloud decoder is recorded as in, represents the intermediate features after decompression on the cloud, θ (t) represents the model parameters of the cloud decoder. The corresponding t-th lightweight edge decoder is recorded as y represents the intermediate feature of the edge, η (t) represents the model parameters of the lightweight edge decoder, where η (t) The parameter scale is smaller than θ (t) .

[0098] First, the intermediate features are input into the lightweight edge decoder for decoding to obtain the edge decoding result Next, calculate the edge decoding result The entropy value As feature uncertainty, H(·) represents an entropy calculation function, such as Shannon entropy. The purpose of calculating the entropy value in the embodiment of the present application is to obtain the uncertainty of the edge decoding result to measure the amount of information, so that the intermediate features with less information can be processed at the edge end, and the intermediate features with more information can be sent to the cloud to extract information using a cloud decoder with stronger expression ability.

[0099] Step 130: When the feature uncertainty is less than or equal to a preset threshold, the edge decoding result is used as the target processing result of the to-be-processed data corresponding to the processing task.

[0100] In one embodiment, different processing tasks correspond to different preset thresholds. Figure 5 , Figure 5 Schematic diagram of the cloud-edge collaborative dynamic exit mechanism provided in the embodiment of the present application. The cloud-edge collaborative dynamic exit mechanism includes edge exit and cloud exit. Edge exit means that the entire processing flow ends at the edge, and cloud exit means that the entire processing flow ends at the cloud, reducing the number of communications with the cloud.

[0101] For edge exit, edge decoding results are obtained based on intermediate features The corresponding entropy value Then, the entropy value Compare with the preset threshold. Assume that the preset threshold corresponding to the tth processing task is At this time, if it meets: Then receive the edge exit result, that is, the edge decoding result The data to be processed corresponds to the target processing result of the tth processing task.

[0102] Step 140: When the feature uncertainty is greater than a preset threshold, the edge compression model is used to compress the intermediate features to obtain intermediate compressed features, and the intermediate compressed features are sent to the cloud server in the cloud-edge collaborative data processing system.

[0103] In one embodiment, in combination Figure 3 A cloud decompression model and a cloud decoder corresponding to each processing task are deployed on the cloud server. The cloud decompression model is used to decompress the intermediate compression features to obtain the intermediate decompressed features. The cloud decoder is used to decode the intermediate decompressed features to obtain the cloud decoding results, and the cloud decoding results are used as the target processing results corresponding to the processing tasks.

[0104] Reference Figure 5 , if the result of edge exit is not satisfied, that is, the feature uncertainty is greater than the preset threshold, then enter the cloud exit, use the edge compression model to compress the intermediate features, obtain the intermediate compressed features, and then send the intermediate compressed features to the cloud server in the cloud-edge collaborative data processing system, perform data processing in the cloud, and obtain the cloud decoder corresponding to the tth processing task for decoding, and obtain the corresponding cloud decoding result as the target processing result.

[0105] The above-mentioned cloud-edge collaborative dynamic exit mechanism can also cope with network fluctuations. Although the accuracy of the lightweight edge decoder is not as good as that of the cloud decoder, it can also exit at the edge when providing qualified data that meets the processing task requirements, and exit at the cloud when more accurate results are needed, thus meeting the dual requirements of speed and accuracy.

[0106] Next, refer to Figure 6 , Figure 6 Schematic diagram of the process of cloud-edge collaborative data processing provided by the embodiment of the present application. Figure 6 It can be seen that the edge compression model includes at least a quantization model or a correction model, wherein the correction model includes a side information encoder, a statistical parameter encoder, and a side information decoder. The purpose of the correction model in this embodiment is to increase the amount of information of the intermediate features and adjust their distribution so that different intermediate features have different distribution characteristics, thereby improving the accuracy of the data processing results. Therefore, the correction model is determined based on actual needs and the actual computing power of the edge end.

[0107] In one embodiment, taking a separate quantization model as an example, the edge compression model is used to compress the intermediate features, and the process of obtaining the intermediate compressed features specifically includes: quantizing the intermediate features using the quantization model to obtain intermediate quantized features, arithmetically encoding the intermediate quantized features, and obtaining intermediate compressed features of the binary bit string.

[0108] In one embodiment, the quantization process used in the model training process can be a soft quantization process, that is, soft quantization is performed using uniform distribution, and an error is added to the intermediate feature y′ to obtain the intermediate quantized feature It is expressed as: Among them, ε~U(-0.5,0.5). In the inference process of the model, the hard quantization operation of rounding is used to obtain the intermediate quantization feature It is expressed as: in, Indicates a rounding operation.

[0109] In one embodiment, the intermediate quantization features are then arithmetically encoded, and the intermediate quantization features are compressed using default statistical distribution parameters, wherein the default statistical distribution parameters include mean, standard deviation, and the like. The principle of arithmetic coding in this embodiment is: long bytes are used to represent numbers with a small probability of appearing in the intermediate quantization features, and short bytes are used to represent numbers with a large probability of appearing in the intermediate quantization features, so it is necessary to obtain the probability of occurrence of each number in the intermediate quantization features based on the statistical distribution parameters. After arithmetic coding, the intermediate compressed features are formed by the binary bit string obtained.

[0110] In one embodiment, taking the inclusion of a modified model as an example, the purpose of the modified model is to provide adaptive statistical parameters different from the default statistical distribution parameters for the intermediate quantitative features. Figure 7 , Figure 7 This is a schematic diagram of using an edge compression model to compress intermediate features to obtain intermediate compressed features, which specifically includes the following steps:

[0111] Step 710: input the intermediate features into the side information encoder for feature correction to obtain statistical hidden features, and quantize the statistical hidden features to obtain quantized hidden features.

[0112] In one embodiment, in combination Figure 6The side information encoder deployed at the edge is used to mine the hidden features corresponding to the real statistical parameters of the intermediate features. For example, the real statistical parameters can be the real mean or the real standard deviation, etc. The statistical hidden features can reflect the distribution of different intermediate features. Then, after obtaining the statistical hidden features, they are quantized to obtain quantized hidden features. It can be understood that, with reference to the quantization process of the above-mentioned intermediate features, soft quantization can be used for quantization in the training stage, and hard quantization can be used for quantization in the inference stage. This embodiment does not limit this.

[0113] Step 720: Input the quantized hidden features into the statistical parameter encoder for encoding to obtain the side information compression features.

[0114] In one embodiment, after obtaining the quantized hidden features, two parts of processing are required. The first part is to use the quantized hidden features to adapt the statistical parameters of the compression process of the intermediate quantized features. The other part is to compress the quantized hidden features and send them to the cloud so as to decode the intermediate compressed features obtained by the statistical parameter adaptation.

[0115] In one embodiment, the quantized hidden features are input into a statistical parameter encoder for arithmetically encoding to obtain side information compression features. The purpose of the statistical parameter encoder here is to compress the quantized hidden features by arithmetically encoding. By inputting a set of quantized hidden features, the corresponding side information compression features can be output.

[0116] Step 730: Input the quantized hidden features into the side information decoder for decoding to obtain statistical parameters, and use the statistical parameters to correct the intermediate quantized features to obtain simulated compressed features.

[0117] In one embodiment, information mining is performed from the quantized hidden features to obtain statistical parameters that can represent the true statistical parameters of the intermediate features. Specifically, the quantized hidden features are input into a side information decoder for decoding to obtain corresponding statistical parameters.

[0118] Next, when performing arithmetic coding on the intermediate quantization features, unlike the previous compression of the intermediate quantization features using the default statistical distribution parameters, the intermediate quantization features are corrected using the statistical parameters calculated above to obtain simulated compression features. Due to the use of real statistical parameters, the probability of each symbol to be encoded when used for lossless entropy coding can be corrected, so that the simulated compression features can reach the Shannon limit as much as possible.

[0119] Step 740: Obtain an intermediate compression feature according to the simulation compression feature and the side information compression feature.

[0120] In one embodiment, the obtained simulation compression features and side information compression features are spliced ​​to obtain the overall intermediate compression features, which are sent to the cloud for subsequent processing. By correcting the compression process, the compression rate of the intermediate features and the accuracy of subsequent processing tasks are further improved.

[0121] In one embodiment, in order to adaptively adjust the compression rate of the edge end, different related models are obtained by setting different variation amounts during the training process. Due to the different variation amounts, the output results of the obtained models will also be different, which is reflected in the different compression rates of the final intermediate compression features. Therefore, model training can be performed in advance according to different variation amounts to obtain related models corresponding to different compression rates.

[0122] Reference Figure 8 , Figure 8 This is a flow chart of adaptively adjusting the compression rate of the edge end provided by an embodiment of the present application, which specifically includes the following steps:

[0123] Step 810: Use at least one of an edge encoder, a lightweight edge decoder, and an edge compression model as a first basic network.

[0124] In one embodiment, the compression rate of the edge encoder, lightweight edge decoder and edge compression model at the edge can be adapted in this way. It can be understood that the compression rate control processes of the edge encoder, lightweight edge decoder and edge compression model are independent, so one or more first basic networks can be selected according to actual needs.

[0125] Step 820: Obtain at least one of the first convolutional layer parameters of the convolutional layer or the first fully connected layer parameters of the fully connected layer in the first basic network, and perform singular value decomposition on the first convolutional layer parameters and / or the first fully connected layer parameters to obtain a first singular value matrix and a first default matrix.

[0126] In one embodiment, the first basic network is a neural network, and the convolutional layer or the fully connected layer therein can be parameter locked to obtain different training results. Therefore, at least one of the first convolutional layer parameters of the convolutional layer or the first fully connected layer parameters of the fully connected layer in the first basic network is obtained as the relevant parameters to be parameter locked, and then the selected first convolutional layer parameters and / or first fully connected layer parameters are singular value decomposition is performed to obtain a first singular value matrix and a first default matrix.

[0127] Reference Fig. 9 , Fig. 9 φ∈R is a schematic diagram of the compression rate adjustment of the first basic network provided in the embodiment of the present application. The first convolutional layer parameter or the first fully connected layer parameter is selected as the adjustment parameter φ∈R in×out, the input data dimension is in, the output data dimension is out, and the singular value decomposition of the adjustment parameter φ is expressed as:

[0128] φ=U×diag(δ)×V T

[0129] Among them, U is an in×in unitary matrix, V is an out×out unitary matrix, diag(δ) is a semi-positive definite in×out diagonal matrix, the two unitary matrices U and V are taken as the first default matrices, and δ is taken as the first singular value matrix.

[0130] Step 830: Obtain a preset first variation, obtain a first compression matrix according to the first variation and a first singular value matrix, and obtain first compression model parameters according to the first compression matrix and a first default matrix.

[0131] In one embodiment, the model parameters are updated according to the actual training process, and the parameter changes of the model are quantified to obtain at least one first change amount Δ. Then, the first compression matrix is ​​obtained according to the first change amount Δ and the first singular value matrix δ, which is expressed as: δ′=ReLU(δ+Δ), where ReLU is an activation function. Finally, the first compression model parameter φ′ is obtained according to the first compression matrix and the first default matrix, which is expressed as:

[0132] φ′=U×diag(δ′)×V T

[0133] Step 840: Update the parameters of the first basic network based on the first compression model parameters to obtain an updated first basic network.

[0134] In one embodiment, different first compression model parameters are used to update the parameters of the first basic network to obtain a corresponding different and updated first basic network. It can be understood that by decomposing the parameter space of the model, the change amount and the updated first basic network are stored correspondingly, and when storing the relevant models of the compression rate corresponding to different intermediate features, only the first basic model and a small number of updated parameters corresponding to the compression rate need to be stored, thereby achieving flexible adjustment of the compression rate and saving edge storage resources. At the same time, when the change amount is selected to switch the compression rate according to actual needs, the updated first basic network is used to replace the original first basic network, so as to controllably adjust the compression rate of the intermediate compression feature.

[0135] In the embodiment of the present application, the cloud-edge collaborative data processing model is segmented, with one part of the processing process implemented at the edge and the other part implemented in the cloud. Through this segmentation process, the data to be processed is converted into intermediate features of smaller dimensions, and then these intermediate features are compressed to further reduce the amount of data sent from the edge to the cloud, thereby reducing communication overhead. In addition, for different processing tasks, a lightweight model is used in advance to calculate a quick result, and the entropy value of the result is used to determine whether to select the processing result of the edge or the processing result of the cloud. This flexible cloud-edge collaborative dynamic exit mechanism is used to selectively compress part of the data from the full amount of data for transmission, which can further reduce the amount of data and communication overhead.

[0136] The following describes the processing process of the cloud-edge collaborative data processing method in the cloud in an embodiment of the present application.

[0137] Fig.10 is an optional flow chart of the cloud-edge collaborative data processing method provided in an embodiment of the present application applied to a cloud server. Fig.10 The method in the embodiment may include but is not limited to steps 1010 to 1030. Fig.10 The order of step 1010 to step 1030 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0138] Step 1010: Receive intermediate compression features sent from the edge server.

[0139] Step 1020: Decompress the intermediate compressed features using the cloud decoding model to obtain intermediate decompressed features.

[0140] In one embodiment, in combination Figure 4 ,The cloud uses the cloud decoding model to perform a decompression operation on the intermediate compressed features corresponding to the compression, and obtains the intermediate decompressed features.

[0141] In one embodiment, since the intermediate compression feature may include the result of the correction module, the decompression operation is also divided into two parts. If the intermediate compression feature does not have the side information compression feature provided by the correction module, the intermediate compression feature is directly reversely arithmetic decoded based on the default statistical distribution parameters to obtain the intermediate decompression feature. It can be understood that if the transmission loss and decoding error are ignored, the intermediate decompression feature is consistent with the intermediate feature.

[0142] In one embodiment, if the intermediate compression feature contains a side information compression feature provided by the correction module, decompression based on the side information compression feature is required. Figure 6 , the cloud decoding model includes the corresponding cloud side information decoder and statistical parameter decoder. Fig.11 , Fig.11 It is a flow chart of using the cloud decoding model to decompress the intermediate compressed features to obtain the intermediate decompressed features, which specifically includes the following steps:

[0143] Step 1110: Obtain simulation compression features and side information compression features from the intermediate compression features.

[0144] In one embodiment, since the simulation compression feature and the side information compression feature are obtained by splicing, the cloud can directly obtain the corresponding simulation compression feature and the side information compression feature from the received intermediate compression feature.

[0145] Step 1120: Input the side information compression feature into the statistical parameter decoder for decoding to obtain the decoded hidden feature.

[0146] In one embodiment, since the statistical parameter decoder and the statistical parameter encoder have opposite functions, the side information compression feature is input into the statistical parameter decoder for decoding to obtain the decoded hidden feature. It can be understood that the decoded hidden feature is the quantized hidden feature obtained by decoding, and the inconsistency of the names of the two is for distinction and reflects the errors introduced during the transmission, encoding and decoding processes.

[0147] Step 1130: Input the decoded hidden features into the cloud-side edge information decoder for decoding to obtain cloud-side statistical parameters.

[0148] In one embodiment, after the decoded hidden features are obtained, similar to step 730, the decoded hidden features are input into the cloud-side edge information decoder for decoding to obtain cloud-side statistical parameters. It is understandable that the cloud-side edge information decoder and the edge-side edge information decoder can also share parameters to reduce training costs.

[0149] Step 1140: Use cloud statistical parameters to perform arithmetical decoding on the simulated compression features to obtain intermediate decompression features.

[0150] In one embodiment, since the arithmetic coding process determines the byte lengths used for different numbers in the analog compression feature based on statistical parameters, the reverse arithmetic decoding process can determine the possible numbers corresponding to the long bytes and short bytes respectively based on the analog compression parameters related to the statistical parameters, thereby completing the decoding process and obtaining the intermediate decompression feature.

[0151] Step 1030: Select a corresponding cloud decoder according to the processing task, input the intermediate decompression features into the cloud decoder for decoding, obtain a cloud decoding result, and use the cloud decoding result as the target processing result corresponding to the processing task.

[0152] In one embodiment, since different processing tasks have different exit timings, if it is an edge exit, there is no need to perform the decoding process of the processing task in the cloud, so some processing tasks exit at the edge and some processing tasks exit in the cloud. At this time, a task identifier can be added during the transmission of the intermediate compressed features, so that the cloud can know which processing tasks need to be executed, and thus select the corresponding cloud decoder. After selecting the corresponding cloud decoder, the intermediate decompression features are input into the cloud decoder for decoding to obtain the cloud decoding result. At this time, these processing tasks are all exited by the cloud, so the corresponding cloud decoding result is used as the target processing result corresponding to the processing task.

[0153] In one embodiment, referring to the compression rate control process of the edge end, the cloud end may also perform compression rate control on the relevant model. The specific process includes the following steps: using at least one of the cloud decompression model or the cloud decoder as the second basic network, obtaining at least one of the second convolutional layer parameters of the convolutional layer or the second fully connected layer parameters of the fully connected layer in the second basic network, and performing singular value decomposition on the second convolutional layer parameters and / or the second fully connected layer parameters to obtain a second singular value matrix and a second default matrix, obtaining a pre-set second change amount, obtaining a second compression matrix according to the second change amount and the second singular value matrix, and obtaining second compression model parameters according to the second compression matrix and the second default matrix, and updating the parameters of the second basic network based on the second compression model parameters to obtain an updated second basic network.

[0154] In combination with the description of the above embodiments, in the cloud-edge collaborative data processing system in the embodiments of the present application, the edge server is used to use an edge encoder to extract features from the data to be processed from the terminal to obtain intermediate features, use a lightweight edge decoder to decode the intermediate feature input to obtain an edge decoding result, and calculate the entropy value of the edge decoding result as the feature uncertainty. When the feature uncertainty is less than or equal to a preset threshold, the edge decoding result is used as the target processing result of the data to be processed. When the feature uncertainty is greater than the preset threshold, the intermediate features are compressed using an edge compression model to obtain intermediate compressed features, and the intermediate compressed features are sent to the cloud server.

[0155] Next, the cloud server is used to decompress the intermediate compressed features using the cloud decompression model to obtain the intermediate decompressed features, select the corresponding cloud decoder according to the processing task, input the intermediate decompressed features into the cloud decoder for decoding, obtain the cloud decoding results, and use the cloud decoding results as the target processing results corresponding to the processing task.

[0156] In one embodiment, referring to Fig.12 , Fig.12It is a flow chart of the training process of the edge encoder, edge compression model, cloud decompression model and cloud decoder in the cloud-edge collaborative data processing system provided by an embodiment of the present application, which specifically includes the following steps:

[0157] Step 1210: Obtain input sample data.

[0158] In one embodiment, information theory is used to train the model, so only input sample data similar to the data to be processed needs to be obtained.

[0159] Step 1220: During the training process, intermediate training data corresponding to the input sample data is obtained, transmission mutual information between the input sample data and the intermediate training data is obtained, and an information amount constraint is generated according to the maximum amount of information and the transmission mutual information.

[0160] Step 1230: Obtain cloud-based inference results corresponding to different processing tasks corresponding to the intermediate training data, generate inference mutual information between each cloud-based inference result and the intermediate training data, and maximize the inference mutual information based on the Lagrange multiplier corresponding to each processing task to obtain the compression target.

[0161] In one embodiment, assuming that the input sample data is x, the intermediate training data is y, and the cloud inference result obtained by the tth processing task is z (t) , the total number of processing tasks is T. At this time, the joint distribution of T types of processing tasks in the entire cloud-edge collaborative data processing process is expressed as:

[0162] p(x,y,z (1:T) )=p(x,y,z (1) ,z (2) ,..,z (T) )

[0163] Where p(·) is the probability density function.

[0164] Since the intermediate training data is a hidden representation of the input sample data, y cannot provide additional information to z given x. (t) , the process is modeled by mutual information as I(y; z (t) |x)=0, where I(·) is the function for calculating mutual information.

[0165] Since mutual information is a measure of the mutual dependence between two random variables, this modeling process means that under the condition of given input sample data x, the mutual information between the intermediate training data y and the cloud-based inference result corresponding to any processing task (taking task t as an example) is zero. It is also equivalent to the three satisfying the Markov chain: Given x, y and z (t) Are independent of each other.

[0166] Next, the transmission mutual information between the input sample data and the intermediate training data is obtained, which is expressed as: I(x; y). The transmission mutual information is used to measure the degree of information sharing between the input sample data and the intermediate training data. At the same time, since the purpose of the embodiment of the present application is to reduce the communication overhead between cloud-edge communications, it is necessary to limit the maximum amount of information flowing from x to y. c , the process is modeled as the maximum amount of information and the transmission mutual information to generate information constraints, the information constraint is expressed as: I(x; y) ≤ I c .

[0167] Next, to ensure that the cloud decoder can obtain the correct cloud inference result z (t) , the most effective information needs to be retained in y. First, the inference mutual information I(y; z) of the cloud inference result and the intermediate training data is obtained. (t) ), and then maximize the inference mutual information. The process is modeled as

[0168] In one embodiment, considering that different processing tasks have different weights in the cloud, a Lagrange multiplier β is assigned to each processing task. (t) , where β (t) >0 is used to ensure that the information contribution of each processing task is taken into account.

[0169] Then, based on the Lagrange multiplier, the compression target obtained by maximizing the inference mutual information is expressed as:

[0170]

[0171] Step 1240: Obtain an optimization target according to the transmission mutual information and the compression target.

[0172] In one embodiment, the optimization objective is expressed as:

[0173]

[0174] Step 1250: Obtain the target upper bound corresponding to the loss function and the optimization target based on the input sample data, the intermediate training data and the cloud inference results.

[0175] In one embodiment, the process of converting the input sample data into the intermediate training data at the edge is modeled as p(y|x), where p(·|·) is the conditional probability density function, and the process of converting the intermediate training data into the cloud inference result of the t-th processing task in the cloud is modeled as p(z (t) |y). Then, based on the prior knowledge from the experiment, we use the probability distribution r(y) and q(z (t) |y) to approximate p(y) and p(z (t) |y), the upper bound of the optimization target is expressed as:

[0176]

[0177] The upper bound of the target consists of two integral terms. The first integral term represents the joint distribution of the input sample data, the intermediate training data, and the cloud inference results and the approximate conditional distribution q(z (t) |y), and the following risk represents the relationship between the distribution of input sample data, intermediate training data and the approximate distribution r(y).

[0178] Step 1260: Under the premise of satisfying the target upper bound and information constraint, minimize the loss value corresponding to the loss function, and train the edge encoder, edge compression model, cloud decompression model and cloud decoder.

[0179] In one embodiment, with the target upper bound and the information constraint, the model parameters can be adjusted so that the loss value corresponding to the loss function is minimized while not exceeding the target upper bound and satisfying the information constraint.

[0180] Among them, the loss function of T processing tasks is expressed as:

[0181]

[0182] where r(·) and q(·) are probability density functions, ψ is the parameter of r(y), is the quantization result corresponding to the intermediate training data, Represents the expectation under the probability distribution p(x). The first part of the above loss function is the expected log-likelihood term, which is used to ensure that the quantization results can be effectively generated from the input sample data, and the second part ensures that the cloud inference results of each processing task can be accurately predicted from the quantization results.

[0183] In one embodiment, the processing process at the edge is modeled as:

[0184]

[0185] y=E(x;φ e ).

[0186] Among them, E(x; φ e ) is the parameter of the edge end φ e The edge encoder is used to map high-dimensional data to the latent space y and obtain the

[0187] In one embodiment, the processing process in the cloud is modeled as:

[0188] The classification tasks are:

[0189]

[0190] The reconstruction tasks are:

[0191]

[0192] in, The parameter is θ (t) Cloud decoder, I cls is the 0-1 indicative function of category j, and N is a normal distribution.

[0193] It can be seen from the above training process that the compression process of the edge compression module of the embodiment of the present application is related to the results of the edge encoder training. The training target is obtained through the probabilistic modeling of information theory to compress the most useful data. For example, in the task of image classification, the background of the image is irrelevant information, and the object is useful information. Therefore, through the training process, the edge compression module is combined with the edge encoder to remove irrelevant information as much as possible to achieve the maximum compression effect. And this end-to-end training method under a unified framework, the cloud-edge collaborative data processing system can automatically learn the feature representation and compression strategy that is most suitable for a specific task while maintaining high efficiency.

[0194] The cloud-edge collaborative data processing method of the embodiment of the present application makes the intermediate feature data compression-friendly through joint optimization, and directly compresses and transmits the intermediate features to the cloud through lossless entropy coding after quantization, and then the cloud decoder performs reasoning based on the intermediate features to obtain the final result. When compressing the intermediate features, the most effective information is retained, bandwidth resources for edge and cloud communication are saved, and reasoning delay is reduced.

[0195] The following describes the experimental verification results of the cloud-edge collaborative data processing method in the embodiment of the present application.

[0196] Reference Fig.13 , Fig.13 It is the accuracy experimental result under the image classification and reconstruction tasks provided by the embodiment of the present application. The embodiment of the present application takes the Imagenette dataset as an example to conduct relevant experiments on the classification and reconstruction tasks of the image. It can be seen from the experimental data that the data of the uncompressed intermediate data is the largest. According to the experimental results, under an acceptable loss of task results, compressing the intermediate features saves a lot of communication overhead compared to the uncompressed process, and the compression process with added side information has a higher degree of compression than the compression process without side information.

[0197] Reference Fig.14 , Fig.14It is the accuracy loss of the cloud-edge collaborative dynamic exit mechanism under different network availability rates provided by the embodiment of the present application. In this embodiment, the progress loss of the cloud-edge collaborative dynamic exit mechanism is verified respectively under different network availability rates, such as 100%, 90%, 80%, and 70%. The data set is still the Imagenette data set. Taking the classification task as an example, the task success rate when further reasoning through the cloud is verified. Since the classification accuracy of the lightweight edge decoder deployed at the edge is 15.38% lower than that of the cloud, the cloud-edge collaborative dynamic exit mechanism is used to allow the relevant data that is easy to process at the edge to be retained at the edge to exit, and the data that is more difficult to process is sent to the cloud after compression. According to the experimental results, it can be seen that the communication overhead is reduced within an acceptable overall task accuracy loss.

[0198] The technical solution provided by the embodiment of the present application obtains the data to be processed from the terminal, and sends the data to be processed into the edge encoder for feature extraction to obtain the intermediate features. Next, for each processing task, the intermediate features are input into the lightweight edge decoder for decoding to obtain the edge decoding result, and the entropy value of the edge decoding result is calculated as the feature uncertainty. When the feature uncertainty is less than or equal to the preset threshold, the edge decoding result is used as the target processing result of the processing task corresponding to the data to be processed. When the feature uncertainty is greater than the preset threshold, the edge compression model is used to compress the intermediate features to obtain the intermediate compressed features, and the intermediate compressed features are sent to the cloud server in the cloud-edge collaborative data processing system. The intermediate compressed features are decompressed on the cloud server to obtain the intermediate decompressed features, and then the intermediate decompressed features are decoded to obtain the cloud decoding results, and the cloud decoding results are used as the target processing results corresponding to the processing task. In the embodiment of the present application, the cloud-edge collaborative data processing model is segmented, and one part of the processing process is implemented at the edge end, and the other part is implemented at the cloud end. Through this segmentation process, the data to be processed is converted into intermediate features with smaller dimensions, and then these intermediate features are compressed to further reduce the amount of data sent from the edge end to the cloud, thereby reducing communication overhead. In addition, for different processing tasks, a lightweight model is used in advance to calculate a quick result, and the entropy value of the result is used to determine whether to select the processing result on the edge or the cloud. This flexible cloud-edge collaborative dynamic exit mechanism is used to selectively compress part of the data from the full amount of data for transmission, which can further reduce the amount of data and communication overhead.

[0199] The present application also provides an electronic device, including:

[0200] at least one memory;

[0201] at least one processor;

[0202] at least one program;

[0203] The program is stored in the memory, and the processor executes the at least one program to implement the cloud-edge collaborative data processing method implemented in the present application. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0204] See also Fig.15 , Fig.15 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0205] The processor 1501 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0206] The memory 1502 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1502 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1502, and the processor 1501 calls and executes the cloud-edge collaborative data processing method of the embodiment of this application;

[0207] Input / output interface 1503, used to implement information input and output;

[0208] Communication interface 1504, used to realize communication interaction between the device and other devices, which can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.); and

[0209] A bus 1505 that transmits information between the various components of the device (e.g., the processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504);

[0210] The processor 1501 , the memory 1502 , the input / output interface 1503 and the communication interface 1504 are connected to each other in communication within the device via the bus 1505 .

[0211] An embodiment of the present application also provides a storage medium, which is a storage medium that stores a computer program. When the computer program is executed by a processor, the above-mentioned cloud-edge collaborative data processing method is implemented.

[0212] As a non-transitory storage medium, the memory can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0213] The cloud-edge collaborative data processing method, system, device and storage medium proposed in the embodiment of the present application obtain the data to be processed from the terminal, and send the data to be processed to the edge encoder for feature extraction to obtain the intermediate features. Next, for each processing task, the intermediate features are input into the lightweight edge decoder for decoding to obtain the edge decoding result, and the entropy value of the edge decoding result is calculated as the feature uncertainty. When the feature uncertainty is less than or equal to the preset threshold, the edge decoding result is used as the target processing result of the processing task corresponding to the data to be processed. When the feature uncertainty is greater than the preset threshold, the edge compression model is used to compress the intermediate features to obtain the intermediate compressed features, and the intermediate compressed features are sent to the cloud server in the cloud-edge collaborative data processing system. The intermediate compressed features are decompressed on the cloud server to obtain the intermediate decompressed features, and then the intermediate decompressed features are decoded to obtain the cloud decoding results, and the cloud decoding results are used as the target processing results corresponding to the processing task. In the embodiment of the present application, the cloud-edge collaborative data processing model is divided, and part of the processing process is implemented at the edge end, and the other part is implemented in the cloud. Through this segmentation process, the data to be processed is converted into intermediate features with smaller dimensions, and then these intermediate features are compressed to further reduce the amount of data sent from the edge to the cloud, thereby reducing communication overhead. In addition, for different processing tasks, a lightweight model is used in advance to calculate a fast result, and the entropy value of the result is used to determine whether to choose the processing result at the edge or the processing result at the cloud. This flexible cloud-edge collaborative dynamic exit mechanism is used to selectively compress part of the data from the full amount of data for transmission, which can further reduce the amount of data and communication overhead.

[0214] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0215] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0216] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0217] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0218] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0219] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0220] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0221] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0222] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0223] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store programs.

[0224] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A cloud-edge collaborative data processing method, characterized in that: An edge server applied to a cloud-edge collaborative data processing system, wherein an edge encoder, at least one lightweight edge decoder corresponding to different processing tasks, and an edge compression model are deployed on the edge server, and the method includes: Acquire the data to be processed from the terminal, and send the data to be processed to the edge encoder for feature extraction to obtain intermediate features; For each of the processing tasks, inputting the intermediate features into the lightweight edge decoder for decoding to obtain an edge decoding result, and calculating an entropy value of the edge decoding result as a feature uncertainty; When the feature uncertainty is less than or equal to a preset threshold, the edge decoding result is used as a target processing result of the to-be-processed data corresponding to the processing task; When the feature uncertainty is greater than the preset threshold, the edge compression model is used to compress the intermediate feature to obtain an intermediate compressed feature, and the intermediate compressed feature is sent to a cloud server in the cloud-edge collaborative data processing system; Among them, a cloud decompression model and a cloud decoder which together with the edge encoder constitute a cloud-edge collaborative data processing model are deployed on the cloud server. The lightweight edge decoder corresponds to the cloud decoder. The cloud decompression model is used to decompress the intermediate compression features to obtain intermediate decompression features. The cloud decoder is used to decode the intermediate decompression features to obtain cloud decoding results, and use the cloud decoding results as the target processing results corresponding to the processing task.

2. The cloud-edge collaborative data processing method according to claim 1 is characterized in that: The edge compression model at least includes a quantization model, and the step of compressing the intermediate features using the edge compression model to obtain the intermediate compressed features includes: quantizing the intermediate features using the quantization model to obtain intermediate quantized features; The intermediate quantized features are arithmetically encoded to obtain the intermediate compressed features of the binary bit string.

3. The cloud-edge collaborative data processing method according to claim 2 is characterized in that: The edge compression model at least includes a side information encoder, a statistical parameter encoder and a side information decoder, and the using the edge compression model to perform feature compression on the intermediate feature to obtain the intermediate compressed feature includes: Inputting the intermediate feature into the side information encoder for feature correction to obtain a statistical hidden feature, and quantizing the statistical hidden feature to obtain a quantized hidden feature; Inputting the quantized hidden features into the statistical parameter encoder for encoding to obtain side information compression features; Inputting the quantized hidden features into the side information decoder for decoding to obtain statistical parameters, and using the statistical parameters to correct the intermediate quantized features to obtain simulated compressed features; The intermediate compression feature is obtained according to the simulation compression feature and the side information compression feature.

4. The cloud-edge collaborative data processing method according to claim 3 is characterized in that: The method further comprises: Using at least one of the edge encoder, the lightweight edge decoder and the edge compression model as a first basic network; Obtain at least one of a first convolutional layer parameter of a convolutional layer or a first fully connected layer parameter of a fully connected layer in the first basic network, and perform singular value decomposition on the first convolutional layer parameter and / or the first fully connected layer parameter to obtain a first singular value matrix and a first default matrix; Obtaining a preset first variation, obtaining a first compression matrix according to the first variation and the first singular value matrix, and obtaining a first compression model parameter according to the first compression matrix and the first default matrix; The first basic network is updated based on the first compression model parameters to obtain an updated first basic network.

5. A cloud-edge collaborative data processing method, characterized in that: A cloud server applied to a cloud-edge collaborative data processing system, wherein a cloud decompression model and a cloud decoder that together with the edge encoder constitute a cloud-edge collaborative data processing model are deployed on the cloud server, and the method comprises: Receiving an intermediate compression feature sent from an edge server, wherein the intermediate compression feature is obtained by the cloud-edge collaborative data processing method according to any one of claims 1 to 4; Decompressing the intermediate compressed features using the cloud decoding model to obtain intermediate decompressed features; The corresponding cloud decoder is selected according to the processing task, the intermediate decompression feature is input into the cloud decoder for decoding, a cloud decoding result is obtained, and the cloud decoding result is used as the target processing result corresponding to the processing task.

6. The cloud-edge collaborative data processing method according to claim 5, characterized in that: The cloud decoding model at least includes a cloud side information decoder and a statistical parameter decoder. When the intermediate compression feature is obtained by the analog compression feature and the side information compression feature, the cloud decoding model is used to decompress the intermediate compression feature to obtain the intermediate decompression feature, including: Acquire the simulation compression feature and the side information compression feature from the intermediate compression feature; Inputting the side information compression feature into the statistical parameter decoder for decoding to obtain a decoded hidden feature; Inputting the decoded hidden features into the cloud side information decoder for decoding to obtain cloud side statistical parameters; The simulated compression feature is arithmetically decoded using the cloud statistical parameter to obtain the intermediate decompression feature.

7. The cloud-edge collaborative data processing method according to claim 6, characterized in that: The method further comprises: Using at least one of the cloud decompression model or the cloud decoder as a second basic network; Obtain at least one of a second convolutional layer parameter of the convolutional layer or a second fully connected layer parameter of the fully connected layer in the second basic network, and perform singular value decomposition on the second convolutional layer parameter and / or the second fully connected layer parameter to obtain a second singular value matrix and a second default matrix; Obtaining a preset second variation, obtaining a second compression matrix according to the second variation and the second singular value matrix, and obtaining second compression model parameters according to the second compression matrix and the second default matrix; The second basic network is updated based on the second compression model parameters to obtain an updated second basic network.

8. A cloud-edge collaborative data processing system, characterized in that: include: An edge server, wherein an edge encoder, at least one lightweight edge decoder corresponding to different processing tasks, and an edge compression model are deployed on the edge server; A cloud server, wherein a cloud decompression model and a cloud decoder are deployed on the cloud server, wherein the cloud decoder and the edge encoder together constitute a cloud collaborative data processing model, and the cloud decoder corresponds to the lightweight edge decoder; The edge server is used to use the edge encoder to extract features from the data to be processed from the terminal to obtain intermediate features, use the lightweight edge decoder to decode the intermediate feature input to obtain an edge decoding result, calculate the entropy value of the edge decoding result as a feature uncertainty, when the feature uncertainty is less than or equal to a preset threshold, use the edge decoding result as a target processing result of the data to be processed, when the feature uncertainty is greater than the preset threshold, use the edge compression model to perform feature compression on the intermediate features to obtain intermediate compressed features, and send the intermediate compressed features to the cloud server; The cloud server is used to decompress the intermediate compression features using the cloud decompression model to obtain intermediate decompression features, select the corresponding cloud decoder according to the processing task, input the intermediate decompression features into the cloud decoder for decoding, obtain a cloud decoding result, and use the cloud decoding result as the target processing result corresponding to the processing task.

9. The cloud-edge collaborative data processing system according to claim 8, characterized in that: The training process of the edge encoder, the edge compression model, the cloud decompression model and the cloud decoder includes the following steps: Get input sample data; During the training process, intermediate training data corresponding to the input sample data is obtained, transmission mutual information between the input sample data and the intermediate training data is obtained, and an information amount constraint is generated according to the maximum amount of information and the transmission mutual information; Obtain cloud-based inference results corresponding to different processing tasks corresponding to the intermediate training data, generate inference mutual information between each of the cloud-based inference results and the intermediate training data, and maximize the inference mutual information based on the Lagrange multiplier corresponding to each of the processing tasks to obtain a compression target; Obtaining an optimization target according to the transmission mutual information and the compression target; Obtaining a loss function and a target upper bound corresponding to the optimization target based on the input sample data, the intermediate training data, and the cloud-based inference result; Under the premise of satisfying the target upper bound and the information amount constraint, the loss value corresponding to the loss function is minimized, and the edge encoder, the edge compression model, the cloud decompression model and the cloud decoder are trained.

10. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the cloud-edge collaborative data processing method described in any one of claims 1 to 7 when executing the computer program.

11. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the cloud-edge collaborative data processing method described in any one of claims 1 to 7 is implemented.

Images