Cloud edge data transmission control system, method, storage medium

Abstract

The present disclosure relates to the technical field of artificial intelligence, and discloses a cloud-edge data transmission control system and method, a storage medium and an electronic device. The system comprises a cloud layer and an edge layer. The cloud layer comprises a cloud data channel module connected with a cloud application, which is used to proxy data transmission between the cloud layer and the edge layer. The edge layer comprises a data channel interface module, an edge data channel module and a service discovery module. The data channel interface module is used to establish a connection between an edge application and the edge data channel module. The edge data channel module is used to proxy data transmission between the edge layer and the cloud layer or other edge layers and process transmission data. The service discovery module is used for routing when data is transmitted between modules in the cloud layer and the edge layer, and monitors the state of nodes where the cloud data channel module and the edge data channel module are located, and regulates transmission data according to the state. The present disclosure not only realizes transmission regulation of cloud-edge data, but also realizes transmission regulation of data between edge terminals.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more specifically, to a cloud-edge data transmission control system, a cloud-edge data transmission control method, and a storage medium. Background Technology

[0002] With the development of intelligent edge computing technology, data transmission from cloud to edge, edge to cloud, and even between edges is a common requirement for almost all edge business applications. However, with the large-scale deployment of intelligent identification in the 5G (5th Generation Mobile Communication Technology) era and the diversification and complexity of AI (Artificial Intelligence) applications, the number of network edge devices and the data they generate are growing rapidly. Centralized cloud computing solutions (that is, data aggregation, processing, and storage in the cloud) will face multiple challenges. Therefore, the next generation of edge-cloud collaborative general computing solutions needs to move computing power forward and process the business logic of user applications at the network edge, which is closer to the data source.

[0003] In related technologies, the system architecture includes a cloud center layer, an edge layer, and a device layer. A central scheduling application is deployed in the cloud center portion of the system architecture to manage edge devices and tasks. Furthermore, KubeEdge (an open-source system) Edge endpoints are installed at the edge layer to manage various loads between the cloud and the edge. However, in these technologies, data transmission between edges is also impossible, and there is a lack of scheduling and control over data traffic between the cloud and the edge.

[0004] It should be noted that the information in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a cloud-edge data transmission control system and method, a computer storage medium, and an electronic device, thereby overcoming, to at least a certain extent, the problems of data traffic scheduling and control being impossible in cloud-edge data transmission, and the inability to transmit data between edges. To solve the above problems, this disclosure adopts the following technical solution.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of this disclosure, a cloud-edge data transmission control system is provided, comprising a cloud layer and an edge layer. The cloud layer includes a cloud data channel module connected to a cloud application, used to proxy data transmission between the cloud layer and the edge layer. The edge layer includes a data channel interface module, an edge data channel module, and a service discovery module. The data channel interface module is used to establish a connection between an edge application and the edge data channel module. The edge data channel module is used to proxy data transmission between the edge layer and the cloud layer or other edge layers, and to process the transmitted data. The service discovery module is used for routing data transmission between modules in the cloud layer and the edge layer, monitoring the status of the nodes where the cloud data channel module and the edge data channel module are located, and regulating the transmitted data according to the status.

[0008] In one exemplary embodiment of this disclosure, the data channel interface module includes: an address acquisition unit, configured to send a service request to the service discovery module and receive the address of the edge data channel module returned by the service discovery module based on the service request; and a connection establishment unit, configured to send a connection request to the edge data channel module through the address to establish a connection between the edge application and the edge data channel module.

[0009] In one exemplary embodiment of this disclosure, the data channel interface module further includes: an error detection unit for detecting the state of the node where the edge data channel module is located; a caching unit for caching the data to be transmitted when the state is offline; and a connection establishment unit for establishing a connection with other edge data channel modules when the state is offline.

[0010] In one exemplary embodiment of this disclosure, the system further includes a plug-in management module, which is connected to the edge data channel module and the cloud data channel module respectively; the edge data channel module includes: a data receiving unit, used to receive data packets sent by the edge application via the data channel interface module; a data packet parsing unit, used to parse the data packets to obtain parsed data; and a data dispatching unit, used to dispatch the parsed data to the plug-in management module.

[0011] In one exemplary embodiment of this disclosure, the plug-in management module includes multiple plug-in sub-modules, which form a directed acyclic structure in parallel or serial manner. The multiple sub-plug-in modules include: a first sub-plug-in module for receiving parsed data dispatched by the data dispatch unit; a second sub-plug-in module for processing the parsed data and encapsulating the processed parsed data to obtain encapsulated data; and a third sub-plug-in module for uploading the encapsulated data to the cloud data channel module.

[0012] In one exemplary embodiment of this disclosure, the second sub-plugin module includes: a data aggregation unit for aggregating the parsed data; a traffic statistics unit for statistically analyzing the transmission traffic based on the parsed data; a transmission scheduling unit for scheduling the transmission of the parsed data based on the aggregation processing result and / or the transmission traffic statistics result; and an information leakage detection unit for detecting privacy information present in the parsed data, so as to regulate the transmission of the parsed data based on the detection result.

[0013] In one exemplary embodiment of this disclosure, the service discovery module includes: a health status detection unit, used to detect the health status of the nodes where the cloud data channel module and the edge data channel module are located; a cluster construction unit, used to form a node cluster with nodes having the same functional type according to the functional type of the nodes where the cloud data channel module or the edge data dynamic module is located; and a transmission data regulation unit, used to regulate the transmission of transmission data according to the health status and the formed node cluster.

[0014] In one exemplary embodiment of this disclosure, the cloud data channel module is deployed in a cluster manner, and the cloud data channel module uses a message queue as the data carrier to provide the cloud application with the acquisition and transmission of data.

[0015] According to one aspect of this disclosure, a cloud-edge data transmission control method is provided, applied to an edge data channel module in the cloud-edge data transmission control system described in any of the preceding claims. The method includes: acquiring and processing transmission data sent by an edge application through a data channel interface; sending the processed transmission data to a cloud data channel module in the cloud layer, so that the processed transmission data is sent to the cloud application via the cloud data channel module; or, sending the processed transmission data to other edge data channel modules in other edge layers, so that the processed transmission data is sent to corresponding other edge applications via the other edge data channel modules; acquiring and processing transmission data sent by the cloud application through the cloud data channel module, and sending the processed transmission data to the edge application via the data channel interface module; wherein, during the transmission data transmission process, routing between modules is performed through a service discovery module, and the status of the nodes where the cloud data channel module and the other edge data channel modules are located is monitored through the service discovery module, and the transmission data is regulated according to the status.

[0016] According to one aspect of this disclosure, a computer storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the cloud-edge data transmission control method described above.

[0017] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the cloud-edge data transmission control method described above by executing the executable instructions.

[0018] The cloud-edge data transmission control system in the exemplary embodiments of this disclosure includes a cloud layer and an edge layer. The cloud layer includes a cloud data channel module connected to a cloud application, used to proxy data transmission between the cloud layer and the edge layer. The edge layer includes a data channel interface module, an edge data channel module, and a service discovery module. The data channel interface module is used to establish a connection between the edge application and the edge data channel module. The edge data channel module is used to proxy data transmission between the edge layer and the cloud layer or other edge layers and to process the transmitted data. The service discovery module is used to monitor the status of the nodes where the cloud data channel module and the edge data channel module are located, and to regulate the transmitted data according to the status. On the one hand, by deploying data channel interface modules in the cloud layer and each edge layer respectively, acting as communication proxies for the cloud layer or edge layer, bidirectional full-duplex data transmission between the cloud and the edge, as well as between different edge LANs, can be achieved. On the other hand, by processing the transmitted data through the edge data channel module, the transmitted data processed at the edge layer can be sent to the cloud layer, thereby enabling large-scale data transmission. Furthermore, during data transmission between the cloud and the edge, or between edge layers, the node status discovery function of the service discovery module can detect and transmit data through healthy nodes, thereby controlling the transmission data, facilitating efficient use of cloud and edge bandwidth, and achieving load balancing.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0021] Figure 1 A schematic diagram of the structure of a cloud-edge data transmission control system according to an exemplary embodiment of the present disclosure is shown.

[0022] Figure 2 A schematic diagram of the structure of a data channel interface module according to an exemplary embodiment of the present disclosure is shown.

[0023] Figure 3A schematic diagram of the structure of a data channel interface module according to an exemplary embodiment of the present disclosure is shown.

[0024] Figure 4 A schematic diagram of the structure of a cloud-edge data transmission control system according to an exemplary embodiment is shown.

[0025] Figure 5 A schematic diagram of the structure of a plug-in management module according to an exemplary embodiment of the present disclosure is shown.

[0026] Figure 6 A schematic diagram of the structure of a plug-in management module with a directed acyclic structure according to an exemplary embodiment of the present disclosure is shown.

[0027] Figure 7 A schematic diagram illustrating the structure of a service discovery module according to an exemplary embodiment of this disclosure is shown.

[0028] Figure 8 A flowchart illustrating a cloud-edge data transmission control method according to an exemplary embodiment of the present disclosure is shown schematically;

[0029] Figure 9 A schematic flowchart illustrating the connection establishment interaction between an edge application and an edge data channel module according to an exemplary embodiment of this disclosure is shown.

[0030] Figure 10 An interactive flowchart illustrating the initialization of CEC-C and CEC-E according to an exemplary embodiment of this disclosure is shown schematically;

[0031] Figure 11 A schematic diagram of a storage medium according to an exemplary embodiment of the present disclosure is shown; and

[0032] Figure 12 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown schematically.

[0033] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0034] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and therefore their detailed description will be omitted.

[0035] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details described, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known structures, methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0036] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, or in one or more software-hardened modules, or in different network and / or processor devices and / or microcontroller devices.

[0037] With the large-scale deployment of intelligent identification in the 5G era and the diversification and complexity of AI applications, the number of network edge devices and the amount of data generated are growing rapidly. Centralized cloud computing solutions (which aggregate, process, and store data in the cloud) will face challenges in terms of real-time performance, bandwidth, energy consumption, and data privacy protection. This necessitates moving computing power closer to the data source at the network edge to process the business logic of user applications. However, this also brings new problems: first, edge and cloud services need further collaboration; second, the reliability of data transmission between the edge and cloud needs further improvement; third, there may be unnecessary data transmission issues; and fourth, the complex environment and diverse ecosystem at the edge present challenges for development and deployment.

[0038] In related technologies, Docker (an open-source application container engine) is a container virtualization technology used for developing, delivering, and running applications. Containers virtualize the operating system layer and eliminate differences in architectures through their own set of standards, thus allowing deployment anywhere regardless of architecture. The Kubernetes framework is an open-source system for automatically deploying, scaling, and managing containerized applications. It aims to provide a platform for automatically deploying, scaling, and running application containers across host clusters, supporting a range of container tools, including Docker. The KubeEdge framework is a framework based on the cloud container governance and orchestration capabilities provided by Kubernetes, integrating edge nodes and devices into the Kubernetes system to achieve unified edge-cloud node and device governance. KubeEdge defines an abstraction of edge devices, and then a generic mapper on the edge nodes monitors the device's running status, acquires device data, and transmits control data to the device, providing the device abstraction and control capabilities to the cloud in a proxy manner.

[0039] Based on the above technologies, the cloud-edge communication system framework includes a cloud center layer, an edge layer, and a device layer. The cloud center layer uses Docker+Kubernetes as its system infrastructure, employing Kubernetes to establish the AI-Edge cloud center component, deploying a central scheduling application, and managing edge devices and tasks. The edge layer installs a Docker environment, configures operating system parameters, and then installs the KubeEdge Edge client to receive and execute instructions from the cloud component, manage various loads, and synchronize the status and events of the edge load to the cloud component. Devices in the device layer can connect to edge nodes in the edge layer.

[0040] Correspondingly, the related technologies have the following drawbacks: First, when transmitting data from the edge to the cloud, these technologies can only transmit a small amount of formatted data, and cannot transmit large amounts of formatted or unformatted data, meaning they cannot achieve large-scale data transmission. Second, the related technologies do not implement a service discovery mechanism at the edge, so edge devices cannot obtain the addresses of other devices at the edge, and cannot transmit data between edge devices. Finally, due to the limited number and slow speed of channels between the edge and the cloud, if improper scheduling leads to repeated transmission of duplicate data between the edge and the cloud, and data packet fragmentation reduces channel efficiency, the value of edge computing power will be greatly reduced.

[0041] To address the shortcomings of related technologies, this exemplary embodiment first provides a cloud-edge data transmission control system. (Reference) Figure 1 As shown, the cloud-edge data transmission control system 100 includes a cloud layer 110 and an edge layer 120. The cloud layer 110 includes a cloud data channel module 1101 connected to cloud applications, which is used to proxy data transmission between the cloud layer 110 and the edge layer 120. The edge layer 120 includes a data channel interface module 1201, an edge data channel module 1202, and a service discovery module 1203.

[0042] Data channel interface module 1201 is used to establish a connection between edge applications and edge data channel module 1202;

[0043] Edge data channel module 1202 is used to proxy the data transmission and processing between edge layer 120 and cloud layer 110 or other edge layers;

[0044] The service discovery module 1203 is used for routing data transmission between modules in the cloud layer and the edge layer, and monitors the status of the nodes where the cloud data channel module 1101 and the edge data channel module 1201 are located, and adjusts the transmitted data according to the status.

[0045] According to the cloud-edge data transmission control system in this example embodiment, on the one hand, by deploying data channel interface modules in the cloud layer and each edge layer respectively, as communication proxies of the cloud layer or edge layer, bidirectional full-duplex data transmission between the cloud and edge and between different edge LANs can be realized; on the other hand, by processing the transmitted data through the edge data channel module, the transmitted data processed at the edge layer can be sent to the cloud layer, thereby realizing large-scale data transmission; furthermore, during the data transmission process between the cloud and edge or between edge layers, the node status discovery function of the service discovery module is used to transmit data through healthy nodes, thereby realizing the regulation of transmitted data, which is conducive to the efficient use of cloud-edge bandwidth and the realization of load balancing.

[0046] The cloud-edge data transmission control system in this example embodiment will be further described below.

[0047] In one exemplary embodiment of this disclosure, such as Figure 2 As shown, the data channel interface module 1201 may include:

[0048] Address acquisition unit 1211 is used to send a service request to service discovery module 1203 and receive the address of edge data channel module 1202 returned by service discovery module 1203 based on the service request;

[0049] The connection establishment unit 1212 is used to send a connection request to the edge data channel module 1202 through the address obtained by the address acquisition unit 1211, so as to establish a connection between the edge application and the edge data channel module 1202.

[0050] Optionally, the address of the edge data channel module 1202 returned by the service discovery module 1203 based on the service request can be the real address of the edge data channel module 1202; alternatively, the address of the edge data channel module 1202 returned by the service discovery module 1203 based on the service request can be a virtual address of the edge data channel module 1202, so as to establish a connection between the edge application and the edge data channel module 1202 based on the virtual address. Based on this, when the address of the edge data channel module 1202 changes, the virtual address will not change accordingly, and the data channel interface module 1201 does not need to track the change of the specific address, thereby improving the connection effectiveness and the stability of data transmission.

[0051] In some possible implementations, the service discovery module 1203 may also provide a mapping relationship between the list of edge data channel modules 1202 and virtual addresses. The address acquisition unit 1211 can select different virtual addresses of edge data channel modules 1202 from the mapping relationship based on a pre-set default selection mechanism. Alternatively, the user can select different virtual addresses of edge data channel modules 1202 from the mapping relationship, thereby achieving load balancing to a certain extent.

[0052] In one exemplary embodiment of this disclosure, such as Figure 3 As shown, the data channel interface module 1201 may further include:

[0053] Error detection unit 1213 is used to detect the status of the node where edge data channel module 1202 is located;

[0054] The buffer unit 1214 is used to buffer the data to be transmitted when the status is offline.

[0055] The connection unit 1215 is used to establish a connection with other edge data channel modules when the status is offline.

[0056] Specifically, the edge data channel module 1202 is a stateless proxy service module. When the edge data channel module 1202 is offline, the error detection unit 1213 in the data channel interface module 1201 connected to the edge data channel module 1202 will detect the offline state and cache the transmitted data through the caching unit 1214, or establish a connection with other edge data channel modules through the connection establishment unit 1215. Based on this, the edge data channel module of the edge layer can be guaranteed to be in a high-availability state, thereby ensuring the fault tolerance and stability of cloud-edge data transmission or edge-to-edge data transmission.

[0057] In some possible implementations, since the data channel interface module 1201 is deployed in the application container, when the edge data channel module 1202 is offline, the offline problem can also be detected by the upper-layer Kubernetes+KubeEdge container governance system, which can then choose to restart the node service or start the edge data channel module service on other nodes, thereby further ensuring the high availability of the edge data channel module.

[0058] In one exemplary embodiment of this disclosure, such as Figure 4As shown, the cloud-edge data transmission control system also includes a plug-in management module 1204, which is connected to the edge data channel module 1202 and the cloud data channel module 1101 respectively. The edge data channel module 1202 may include: a data receiving unit for receiving data packets sent by edge applications via the data channel interface module 1201; a data packet parsing unit for parsing the data packets to obtain parsed data; and a data dispatching unit for dispatching the parsed data to the plug-in management module 1204.

[0059] In an exemplary embodiment of this disclosure, the plug-in management module 1204 includes multiple plug-in sub-modules, which form a directed acyclic structure through parallel or serial methods; wherein, for example... Figure 5 Multiple sub-plugin modules include:

[0060] The first sub-plugin module 1241 is used to receive parsed data dispatched by the data dispatch unit;

[0061] The second sub-plugin module 1242 is used to process the parsed data and encapsulate the processed parsed data to obtain encapsulated data;

[0062] The third sub-plugin module 1243 is used to upload packet data to the cloud data channel module 1101.

[0063] Specifically, when an edge application sends transmission data to the edge data channel module 1202 through the data channel interface module 1201, the data packet is received by the data receiving unit, parsed by the data packet parsing unit to obtain parsed data, and finally dispatched to the plug-in management module 1204 by the data dispatching unit for processing. Specifically, the parsed data is received by the first sub-plug-in module 1241, processed by the second sub-plug-in module 1242, and then packetized. Finally, the packetized data is uploaded to the cloud data channel module 1101 by the third sub-plug-in module 1243.

[0064] In some possible implementations, the second sub-plugin module 1242 may include: a data aggregation unit for aggregating the parsed data; a traffic statistics unit for statistically analyzing the transmission traffic based on the parsed data; and a transmission scheduling unit for scheduling the transmission of the parsed data based on the aggregation processing results and / or the transmission traffic statistics results. An information leakage detection unit is also included to detect privacy information present in the parsed data and adjust the transmission of the parsed data according to the detection results; for example, feature value extraction and comparison methods can be used to detect potential privacy leaks in the parsed data.

[0065] It should be noted that the plug-in management module 1204 can analyze the transmitted data packets in real time. For example, it can process the data packets according to the pre-configured processing logic and perform processing, forwarding, or discarding of the data, thereby realizing customizable data scheduling.

[0066] Figure 6 A schematic diagram of the structure of a plug-in management module 1204 according to an exemplary embodiment of the present disclosure is shown, as follows: Figure 6 As shown, each sub-plugin module can be combined in various ways, such as parallel, serial, or a combination of parallel and serial. By combining sub-plugin modules with different basic functions, data processing can be achieved, including but not limited to data aggregation and scheduling.

[0067] It should be noted that the composition of the plug-in management module 1204 described in the exemplary embodiments of this disclosure is not limited to the first sub-plug-in module 1241, the second sub-plug-in module 1242 and the third sub-plug-in module 1243 mentioned above, and each sub-plug-in module may also be composed of different modules (or units). Any data processing function achieved through different sub-plug-in modules in different combinations is within the protection scope of this disclosure.

[0068] In one exemplary embodiment of this disclosure, such as Figure 7 Service discovery module 1203 includes:

[0069] The health status detection unit 1231 is used to detect the health status of the nodes where the cloud data channel 1101 module and the edge data channel module 1202 are located; the cluster construction unit 1232 is used to form a node cluster with nodes of the same functional type according to the functional type of the nodes where the cloud data channel module 1101 or the edge data channel module 1202 are located; and the data transmission control unit 1233 is used to control the transmission of data according to the health status and the formed node cluster.

[0070] Specifically, the health status detection unit 1231 detects the health status of the nodes containing the cloud data channel 1101 module and the edge data channel module 1202 according to a preset cycle, and uses the cluster construction unit 1232 to form a node cluster with the same name from multiple nodes with the same function. During data transmission, the data transmission control unit 1233 can control and distribute data transmission to all nodes in the same node cluster according to their health status, thereby achieving load balancing. In addition, in the event of abnormal situations such as temporary failure of node services, the transmitted data is cached to improve disaster recovery performance.

[0071] In an exemplary embodiment of this disclosure, the cloud data channel module 1101 is deployed in a cluster. This cloud data channel module 1101 uses a message queue as the data carrier to provide data acquisition and transmission for cloud applications. Each node in the message queue is deployed in a master-slave configuration. When the master node fails, a slave node is elected as the new master node. Because the data between the master and slave nodes is synchronized in real time, data loss is minimized. The cluster node storage service uses the persistent infrastructure provided by Kubernetes, enabling the forwarding of transmitted data after a node comes back online.

[0072] In some possible implementations, the cloud data channel module 1101 deletes the transmitted data in the message queue only after receiving a response message indicating that the transmitted data in the message queue has been consumed, without deleting unconsumed data, thus ensuring reliable transmission and disaster recovery capabilities of the transmitted data.

[0073] It should be noted that although several modules or units of the cloud-edge data transmission control system have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0074] According to the cloud-edge data transmission control system in this example embodiment, by deploying data channel interface modules in the cloud layer and each edge layer respectively, as communication proxies of the cloud layer or edge layer, bidirectional full-duplex data transmission between the cloud and edge and between different edge LANs can be realized. At the same time, by processing the transmitted data through the edge data channel module, the transmitted data processed at the edge layer can be sent to the cloud layer, thereby realizing large-scale data transmission. In addition, during the data transmission process between the cloud and edge or between edge layers, the node status discovery function of the service discovery module is used to transmit data through healthy nodes, thereby realizing the regulation of transmitted data, which is conducive to the efficient use of cloud and edge bandwidth and the realization of load balancing.

[0075] Furthermore, in an exemplary embodiment of this disclosure, a cloud-edge data transmission control method is also provided, applied to the edge data channel module in the aforementioned cloud-edge data transmission control system. (See reference...) Figure 8 As shown, the cloud-edge data transmission control method may include the following steps:

[0076] In step S810, the transmission data sent by the edge application through the data channel interface module is acquired and processed;

[0077] In step S820, the processed transmission data is sent to the cloud data channel module of the cloud layer, so that the processed transmission data is sent to the cloud application via the cloud data channel module;

[0078] Alternatively, the processed transmission data can be sent to other edge data channel modules in other edge layers, so that the processed transmission data can be sent to other corresponding edge applications via the other edge data channel modules;

[0079] In step S830, the transmission data sent by the cloud application through the cloud data channel module is obtained and processed, and the processed transmission data is sent to the edge application through the data channel interface module.

[0080] During the data transmission process, the service discovery module performs routing between modules, monitors the status of the nodes where the cloud data processing module and other edge data processing modules are located, and adjusts the transmitted data according to the status.

[0081] In one exemplary embodiment of this disclosure, the edge application needs to establish a connection with the edge data channel module through the data channel interface module; in some possible implementations, the edge application uses the SDK (Software Development Kit) provided by the data channel interface module to connect to the edge data channel module. Specifically, Figure 9 A flowchart illustrating the connection and interaction process between an edge application and an edge data channel module according to an illustrative embodiment of this disclosure is shown, where UAI is the data channel interface module, CEC-E is the edge data channel module, and CEC-C is the cloud data channel module. Figure 9 As shown, the process includes the following steps:

[0082] In step S910, the edge application uses the SDK provided by UAI to send a service request; and in step S920, it sends an address acquisition request to the service discovery module.

[0083] In step S930, the service discovery module receives the address returned by CEC-E;

[0084] In step S940, a connection establishment request is sent to CEC-E via the address;

[0085] In step S950, a connection success response message returned by CEC-E is received.

[0086] Based on this, when sending data, UAI sends the data to CEC-E, then uploads it to CEC-C in the cloud, and finally it reaches the cloud application. If a CEC-E connection is interrupted, UAI will temporarily cache the data and retransmit it after the connection is restored.

[0087] In one exemplary embodiment of this disclosure, CEC-C and CEC-E need to be initialized before cloud-edge data transmission control. Figure 10 An interactive flowchart illustrating the initialization of CEC-C and CEC-E according to an exemplary embodiment of this disclosure is shown, as follows: Figure 10 The process includes the following steps:

[0088] In step S1001, CEC-C is deployed in the cloud layer via the Kubernetes API Server (K8S Application Programming Interface Service);

[0089] In step S1002, CEC-E1 and CEC-E2 are deployed at the edge layer via the Kubernetes API Server. CEC-E1 and CEC-E2 can be deployed at the same edge layer or at different edge layers (i.e., one or more CEC-Es can be deployed at the same edge layer). The number of CEC-Es can be deployed at different edge layers according to the actual data transmission requirements. This disclosure does not impose any special limitations on this.

[0090] In step S1003, CEC-E1 and CEC-E2 respectively send registration requests to CEC-C;

[0091] In step S1004, CEC-E1 and CEC-E2 respectively receive the registration completion response information returned by CEC-C, and complete the registration of CEC-E1 and CEC-E2 to CEC-C;

[0092] In step S1005, the CEC-E service discovery service is deployed in the service discovery module through the Kubernetes API Server, so as to schedule the data transmission of CEC-E1 and CEC-E2 respectively through the service discovery module to achieve load balancing.

[0093] It should be noted that since the functional modules in the cloud-edge data transmission control method of the exemplary embodiments of this disclosure are the same as the functional modules (or units) in the above-described cloud-edge data transmission control system, they will not be described again here.

[0094] Furthermore, in exemplary embodiments of this disclosure, a computer storage medium capable of implementing the above-described methods is also provided. A program product capable of implementing the methods described in this specification is stored thereon. In some possible embodiments, various aspects of this disclosure can also be implemented as a program product including program code, which, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0095] refer to Figure 11 As shown, a program product 1100 for implementing the above-described method according to an exemplary embodiment of the present disclosure is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0096] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0097] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0098] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0099] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0100] Furthermore, in exemplary embodiments of this disclosure, an electronic device capable of implementing the above-described methods is also provided. Those skilled in the art will understand that various aspects of this disclosure can be implemented as systems, methods, or program products. Therefore, various aspects of this disclosure can be specifically implemented as entirely hardware embodiments, entirely software embodiments (including firmware, microcode, etc.), or embodiments combining hardware and software aspects, collectively referred to herein as "circuit," "module," or "system."

[0101] The following reference Figure 12 To describe an electronic device 1200 according to such an embodiment of the present disclosure. Figure 12 The electronic device 1200 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0102] like Figure 12 As shown, the electronic device 1200 is manifested in the form of a general-purpose computing device. The components of the electronic device 1200 may include, but are not limited to: at least one processing unit 1210, at least one storage unit 1220, a bus 1230 connecting different system components (including storage unit 1220 and processing unit 1210), and a display unit 1240.

[0103] The storage unit stores program code that can be executed by the processing unit 1210, causing the processing unit 1210 to perform the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of this disclosure.

[0104] Storage unit 1220 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 1221 and / or cache memory 1222, and may further include a read-only memory (ROM) 1223.

[0105] Storage unit 1220 may also include a program / utility 1224 having a set (at least one) of program modules 1225, such program modules 1225 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0106] Bus 1230 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0107] Electronic device 1200 can also communicate with one or more external devices 1300 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 1200, and / or with any device that enables electronic device 1200 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1250. Furthermore, electronic device 1200 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1260. As shown, network adapter 1260 communicates with other modules of electronic device 1200 via bus 1230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0108] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0109] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0110] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0111] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A cloud-edge data transmission control system, comprising a cloud layer and an edge layer, characterized in that, The cloud layer includes a cloud data channel module connected to cloud applications, used to proxy data transmission between the cloud layer and the edge layer; the edge layer includes a data channel interface module, an edge data channel module, and a service discovery module. The data channel interface module is used to establish a connection between the edge application and the edge data channel module; The edge data channel module is used to proxy data transmission and processing between the edge layer and the cloud layer or other edge layers; The service discovery module is used for routing data transmission between modules in the cloud layer and edge layer, and monitors the status of the nodes where the cloud data channel module and edge data channel module are located, and adjusts the transmitted data according to the status. The service discovery module includes: A health status detection unit is used to detect the health status of the nodes where the cloud data channel module and the edge data channel module are located. The cluster building unit is used to form a node cluster by combining nodes with the same functional type according to the functional type of the node where the cloud data channel module or the edge data channel module is located. A data transmission control unit is used to control the transmission of data based on the health status and the formed node cluster; wherein, the data transmission control unit is used to control the data transmission of all nodes under the same node cluster according to the health status, and to cache the data when an abnormal situation of temporary failure of node service occurs. The data channel interface module includes: The address acquisition unit is used to send a service request to the service discovery module and receive the address of the edge data channel module returned by the service discovery module based on the service request. A connection establishment unit is used to send a connection request to the edge data channel module via the address to establish a connection between the edge application and the edge data channel module; An error detection unit is used to detect the status of the node where the edge data channel module is located; A buffer unit is used to buffer the data to be transmitted when the state is offline. The connection establishment unit is used to establish a connection with other edge data channel modules when the state is offline.

2. The system according to claim 1, characterized in that, The system also includes a plug-in management module, which is connected to the edge data channel module and the cloud data channel module respectively; The edge data channel module includes: A data receiving unit is used to receive data packets sent by the edge application via the data channel interface module; A data packet parsing unit is used to parse the data packet to obtain parsed data; The data dispatch unit is used to dispatch the parsed data to the plug-in management module.

3. The system according to claim 2, characterized in that, The plug-in management module includes multiple plug-in sub-modules, which form a directed acyclic structure through parallel or serial methods; the multiple sub-plug-in modules include: The first sub-plugin module is used to receive the parsed data distributed by the data distribution unit; The second sub-plugin module is used to process the parsed data and encapsulate the processed parsed data to obtain encapsulated data; The third sub-plugin module is used to upload the packet data to the cloud data channel module.

4. The system according to claim 3, characterized in that, The second sub-plugin module includes: A data aggregation unit is used to aggregate the parsed data. A traffic statistics unit is used to count the transmission traffic based on the parsed data; A transmission scheduling unit is used to schedule the transmission of the parsed data based on the aggregation processing results and / or transmission traffic statistics. The information leakage detection unit is used to detect privacy information in the parsed data, and to regulate the transmission of the parsed data according to the detection results.

5. The system according to any one of claims 1 to 4, characterized in that, The cloud data channel module is deployed in a cluster and uses a message queue as the data carrier to provide the cloud application with the ability to acquire and transmit data.

6. A cloud-edge data transmission control method, applied to the edge data channel module in the cloud-edge data transmission control system according to any one of claims 1 to 5, characterized in that, include: Acquire and process the transmission data sent by edge applications through the data channel interface module; The processed transmission data is sent to the cloud data channel module of the cloud layer, so that the processed transmission data is sent to the cloud application via the cloud data channel module; or, the processed transmission data is sent to other edge data channel modules of other edge layers, so that the processed transmission data is sent to the corresponding other edge application via the other edge data channel modules. The system acquires and processes the transmission data sent by the cloud application through the cloud data channel module, and then sends the processed transmission data to the edge application via the data channel interface module. During the data transmission process, a service discovery module performs routing between modules and monitors the status of the nodes where the cloud data channel module and other edge data channel modules reside. Based on this status, the service discovery module regulates the transmitted data. Furthermore, the service discovery module detects the health status of the nodes where the cloud data channel module and edge data channel modules reside, and forms node clusters with nodes of the same functional type based on the functional type of the cloud data channel module or edge data channel module. Based on the health status and the formed node clusters, the transmission of data is regulated, including regulating data transmission across all nodes within the same node cluster according to their health status. In the event of a temporary failure of a node service, the data is cached. The process of establishing a connection between the edge application and the edge data channel module includes: sending a service request to the service discovery module and receiving the address of the edge data channel module returned by the service discovery module based on the service request; and sending a connection request to the edge data channel module through the address to establish a connection between the edge application and the edge data channel module. The method further includes: Detect the status of the node where the edge data channel module is located; When the state is offline, the data to be transmitted is buffered; When the state is offline, establish a connection with other edge data channel modules.

7. A storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the cloud-edge data transmission control method according to claim 6.

Images