Data transmission method and device based on edge cloud node deployment, electronic equipment and storage medium

Abstract

The application relates to the technical field of communication, and provides a data transmission method and device based on an edge cloud node deployment, electronic equipment and a storage medium, the method comprising the following steps: based on the authority or the number of users of an edge cloud node, dividing all the edge cloud nodes under the same cluster into regions to obtain a plurality of cloud node available domains; based on the information capacity of the edge cloud node, determining a superior processing end of the edge cloud node; based on the information capacity of the edge cloud node, determining a data upload frequency of the edge cloud node, so that the edge cloud node uploads a to-be-processed task to the superior processing end based on the data upload frequency. Through the division of the cloud node available domain, the distributed deployment of the plurality of edge cloud nodes is realized, the edge cloud nodes are conveniently managed and scheduled, the server with processing capacity is dynamically selected based on the information capacity of the edge cloud node for data processing, and the data upload frequency is dynamically determined, so that the data transmission delay is effectively reduced, and the data transmission efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a data transmission method, apparatus, electronic device and storage medium based on edge cloud node deployment. Background Technology

[0002] With the rapid development of digital information, the single-line approach of centralized data center processing can hardly meet the needs of timely processing of massive amounts of data. Therefore, the edge cloud model, which decentralizes tasks, has emerged.

[0003] In existing technologies, user terminals typically send their operational data to a data processing center for processing and to obtain corresponding data processing strategies. However, with the increasing number of user nodes, if multiple user nodes simultaneously send large amounts of data to the data processing center, it can easily cause network congestion or excessive data transmission bandwidth load, thereby reducing data transmission efficiency. Summary of the Invention

[0004] This application provides a data transmission method, apparatus, electronic device, and storage medium based on edge cloud node deployment to solve the problem of low data transmission efficiency in the prior art.

[0005] This application provides a data transmission method based on edge cloud node deployment, including:

[0006] Based on the permissions or number of users of the edge cloud nodes, the regions of all the edge cloud nodes under the same cluster are divided to obtain multiple cloud node availability domains;

[0007] Based on the information capacity of the edge cloud node, the upper-level processing end of the edge cloud node is determined; the upper-level processing end includes servers and central cloud nodes in the availability domain of the cloud node;

[0008] Based on the information capacity of the edge cloud node, the data upload frequency of the edge cloud node is determined, so that the edge cloud node uploads the task to be processed to the upper-level processing terminal based on the data upload frequency.

[0009] According to the data transmission method based on edge cloud node deployment provided in this application, the edge cloud nodes under the same cluster are divided into regions based on the permissions or number of users of the edge cloud nodes to obtain multiple cloud node availability domains, including:

[0010] Determine the permission level corresponding to the permissions of the edge cloud node;

[0011] Edge cloud nodes with the same permission level within the same cluster are grouped into the same region to obtain multiple availability domains for the cloud nodes; or...

[0012] Based on the number of users, determine the user quantity level for each edge cloud node;

[0013] Edge cloud nodes with the same number of users in the same cluster are divided into the same region to obtain multiple available domains for the cloud nodes.

[0014] According to the data transmission method based on edge cloud node deployment provided in this application, determining the upper-level processing terminal of the edge cloud node based on the information capacity of the edge cloud node includes:

[0015] If the information capacity of the edge cloud node is less than or equal to the data processing threshold of the cloud node's available domain, then the upstream processing end of the edge cloud node is determined to be a server in the cloud node's available domain.

[0016] If the information capacity of the edge cloud node is greater than the data processing threshold of the available domain of the cloud node, then the upper-level processing end of the edge cloud node is determined to be the central cloud node.

[0017] According to the data transmission method based on edge cloud node deployment provided in this application, determining the data upload frequency of the edge cloud node based on the information capacity of the edge cloud node includes:

[0018] Determine the capacity level of the information capacity;

[0019] Based on the matching result between the capacity level and the preset upload frequency table, the data upload frequency of the edge cloud node is determined; wherein, the preset upload frequency table stores the mapping information between the capacity level and the data upload frequency.

[0020] According to the data transmission method based on edge cloud node deployment provided in this application, after determining the data upload frequency of the edge cloud node based on its information capacity, the method further includes:

[0021] Send a scheduling policy to all edge cloud nodes in the same cloud node availability domain so that the edge cloud nodes can back up the scheduling policy;

[0022] When an abnormal edge cloud node is detected, a task processing request is sent to other edge cloud nodes in the available domain of the cloud node where the abnormal edge cloud node is located, so that the other edge cloud nodes execute the task processing request based on a pre-backed-up scheduling strategy.

[0023] According to the data transmission method based on edge cloud node deployment provided in this application, after determining the data upload frequency of the edge cloud node based on its information capacity, the method further includes:

[0024] Monitor the data transmission status of the edge cloud nodes;

[0025] The data transmission status is shared with all edge cloud nodes in the same cloud node availability domain.

[0026] According to the data transmission method based on edge cloud node deployment provided in this application, any of the cloud node availability domains includes multiple edge cloud nodes; the multiple cloud node availability domains are connected to a central cloud node in a long-term connection.

[0027] This application provides a data transmission device based on edge cloud node deployment, comprising:

[0028] The region partitioning module is used to partition all the edge cloud nodes under the same cluster based on the permissions or number of users of the edge cloud nodes, so as to obtain multiple cloud node availability domains.

[0029] The upper-level processing end determination module is used to determine the upper-level processing end of the edge cloud node based on the information capacity of the edge cloud node; the upper-level processing end includes servers and central cloud nodes in the availability domain of the cloud node;

[0030] The data upload frequency determination module is used to determine the data upload frequency of the edge cloud node based on the information capacity of the edge cloud node, so that the edge cloud node uploads the task to be processed to the upper-level processing terminal based on the data upload frequency.

[0031] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the data transmission method based on edge cloud node deployment as described above.

[0032] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data transmission method based on edge cloud node deployment as described above.

[0033] This application provides a data transmission method, apparatus, electronic device, and storage medium based on edge cloud node deployment. It divides all edge cloud nodes within the same cluster into multiple availability zones based on their permissions or the number of users. It determines the upstream processing terminal for each edge cloud node based on its information capacity and data upload frequency, enabling edge cloud nodes to upload tasks to the upstream processing terminal according to this frequency. By dividing cloud node availability zones, this application achieves distributed deployment of multiple edge cloud nodes, facilitating their management and scheduling. Furthermore, it dynamically selects servers with processing capabilities for data processing based on the edge cloud node's information capacity and dynamically determines the data upload frequency, effectively reducing data transmission latency and improving data transmission efficiency. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating the data transmission method based on edge cloud node deployment provided in this application;

[0036] Figure 2 This is a schematic diagram of the data transmission device deployed based on edge cloud nodes provided in this application;

[0037] Figure 3 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] It should be noted that in the description of the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terms "upper," "lower," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Unless otherwise expressly specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly, for example, they can be fixed connections, detachable connections, or integral connections; they can be mechanical connections or electrical connections; they can be direct connections or indirect connections through an intermediate medium; and they can be internal connections between two elements. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0040] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0041] The following is combined Figures 1-3 This application describes a data transmission method, apparatus, electronic device, and storage medium based on edge cloud node deployment.

[0042] Specifically, this application provides a data transmission method based on edge cloud node deployment, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the data transmission method based on edge cloud node deployment provided in this application.

[0043] The data transmission method based on edge cloud node deployment provided in this application includes:

[0044] Step 100: Based on the permissions or number of users of the edge cloud nodes, divide all the edge cloud nodes under the same cluster into regions to obtain multiple cloud node availability domains.

[0045] The central cloud management system divides all edge cloud nodes under the same cluster into regions based on their permissions or the number of users. Then, based on the region division results, different levels of cloud node availability domains are obtained. Optionally, cloud node availability domain division can also be based on different criteria, such as the geographical location of the edge cloud node, network topology, computing resources, and data processing requirements, to better manage and control each edge cloud node.

[0046] As can be understood, edge cloud nodes refer to computing nodes distributed in an edge network, located near users or devices, used to process and store data, and provide edge computing services. Edge cloud nodes can take the form of physical servers, virtual machines, containers, etc. A cloud node availability domain is a concept used to divide edge cloud nodes; it represents a set of edge cloud nodes with the same or similar characteristics within a cluster or network. By dividing cloud node availability domains, distributed deployment of multiple edge cloud nodes can be achieved, facilitating the management and scheduling of edge cloud nodes.

[0047] Each cloud node availability domain comprises multiple edge cloud nodes. Each cloud node availability domain has unique user identification information, such as an IP address, used to identify the edge cloud nodes within that domain. Long-lived connections are established between the central cloud node and each of the various cloud node availability domains. This means all edge cloud nodes within each domain are permanently connected to the central cloud node. Based on this, real-time communication and data transmission can be achieved. These long-lived connections maintain a continuous communication channel for timely data synchronization and interaction. Simultaneously, a one-to-many connection relationship can be formed between the central cloud node and several edge cloud node availability domains, enabling distributed deployment of multiple edge cloud nodes.

[0048] Step 200: Based on the information capacity of the edge cloud node, determine the upper-level processing terminal of the edge cloud node.

[0049] The upper-level processing end of the edge cloud nodes is divided based on the information capacity of their metadata. It's understood that edge cloud nodes store a large amount of metadata information, including file attributes, permissions, location information, etc. The metadata information capacity of each edge cloud node can be measured by statistically analyzing the total amount of metadata stored in the entire system or the metadata capacity of a single edge cloud node. Optionally, the upper-level processing end is responsible for managing and processing the metadata information of the edge cloud nodes, performing operations such as processing, indexing, querying, and backing up the metadata to improve data access efficiency and availability. Simultaneously, the upper-level processing end can also optimize based on the characteristics of the metadata, for example, caching frequently accessed data on high-speed storage media to accelerate access speed.

[0050] Specifically, if the information capacity of the edge cloud node is less than or equal to the data processing threshold of the cloud node's available domain, then the upstream processing end of the edge cloud node is determined to be a server in the cloud node's available domain; if the information capacity of the edge cloud node is greater than the data processing threshold of the cloud node's available domain, then the upstream processing end of the edge cloud node is determined to be the central cloud node.

[0051] For example, suppose an availability domain containing an edge cloud node has a certain processing capacity, capable of handling information up to 100GB. If the size of some metadata is within 100GB, according to the partitioning strategy, the receiving upstream of this metadata will be set to a server within the availability domain, because servers in the availability domain have sufficient processing capacity to handle information within this range. If the size of some metadata exceeds the availability domain's processing threshold, such as reaching 200GB, according to the partitioning strategy, the receiving upstream of this metadata will be set to a central cloud node. The central cloud node has more powerful processing and storage capabilities, and can handle large volumes of information and tasks.

[0052] Step 300: Based on the information capacity of the edge cloud node, determine the data upload frequency of the edge cloud node, so that the edge cloud node uploads the task to be processed to the upper-level processing terminal based on the data upload frequency.

[0053] Based on the information capacity of edge cloud nodes, the data upload frequency of these nodes is determined, with larger data volumes requiring longer upload intervals. For example, if one edge cloud node generates 100GB of data while another generates 50GB, the upload interval for the 100GB data will be longer than that for the 50GB data before being uploaded to the central cloud node. This ensures that data is transmitted in time slots during the upload process, avoiding network congestion and excessive processing pressure on the central cloud node caused by uploading large amounts of data at once. This approach effectively manages the data upload process, reduces network congestion, lowers the data processing pressure on the central cloud node, and balances the data transmission load between edge and central cloud nodes, improving the overall system efficiency and stability. Furthermore, this strategy helps optimize the operation of the edge computing system and improve data processing efficiency and performance.

[0054] The method for determining the specific data upload frequency is as follows: determine the capacity level of the information, and then determine the data upload frequency of the edge cloud node based on the matching result between the capacity level and the preset upload frequency table; wherein the preset upload frequency table stores the mapping information between the capacity level and the data upload frequency.

[0055] For example, suppose there are three capacity levels: small, medium, and large, with the corresponding information capacity ranges shown in Table 1. At the same time, the mapping information between capacity levels and data upload frequency is shown in Table 1.

[0056] Table 1

[0057]

[0058]

[0059] Suppose an edge cloud node generates 20GB of data, which falls under the medium capacity category. Based on the preset data upload frequency (as shown in Table 1), medium capacity corresponds to uploading data every 10 minutes. Therefore, for the 20GB of data generated by this edge cloud node, the upload frequency is once every 10 minutes.

[0060] The data transmission method based on edge cloud node deployment provided in this application involves dividing all edge cloud nodes in the same cluster into multiple cloud node availability domains based on their permissions or the number of users. The method determines the upper-level processing endpoint for each edge cloud node based on its information capacity. The upper-level processing endpoint includes servers within the cloud node availability domain and a central cloud node. The method also determines the data upload frequency of each edge cloud node based on its information capacity, enabling edge cloud nodes to upload tasks to the upper-level processing endpoint according to this frequency. This application achieves distributed deployment of multiple edge cloud nodes by dividing them into cloud node availability domains, facilitating the management and scheduling of these nodes. Furthermore, it dynamically selects servers with corresponding processing capabilities based on the information capacity of each edge cloud node and dynamically determines the data upload frequency, effectively reducing data transmission latency and improving data transmission efficiency.

[0061] Based on the above embodiments, the step of dividing all edge cloud nodes under the same cluster into multiple cloud node availability domains based on the permissions or number of users of the edge cloud nodes includes:

[0062] Step 111: Determine the permission level corresponding to the permissions of the edge cloud node;

[0063] Step 112: Divide edge cloud nodes with the same permission level within the same cluster into the same region to obtain multiple availability domains for the cloud nodes; or,

[0064] Step 113: Based on the number of users, determine the user quantity level for each edge cloud node;

[0065] Step 114: Divide edge cloud nodes with the same number of users in the same cluster into the same region to obtain multiple cloud node availability domains.

[0066] To divide all edge cloud nodes in the same cluster into regions based on their permissions, the following steps can be taken:

[0067] 1.1) Determine permission levels: First, it is necessary to effectively manage the permissions of edge cloud nodes, ensuring that each edge cloud node has a clearly defined permission level or identifier. For example, a hierarchical structure of permission levels can be determined according to specific needs, such as ordinary users, advanced users, and administrators.

[0068] 1.2) Divide cloud node availability domains: Divide edge cloud nodes into different availability domains based on permission levels. For example, group edge cloud nodes with the same permission level into the same cloud node availability domain to facilitate unified management of their permission settings, access control policies, etc., simplifying the permission management process. Different cloud node availability domains at different levels are allocated different resources, such as different data processing volumes.

[0069] Optionally, corresponding access control policies can be configured for different levels of cloud node availability domains. For example, access permissions for different levels of cloud node availability domains can be restricted through authentication, access control lists, or access tokens. Higher-level cloud node availability domains have higher privileges and more resource access rights. Simultaneously, corresponding management and monitoring mechanisms can be established to ensure effective management and monitoring of different levels of cloud node availability domains. For example, the usage and performance metrics of each level of availability domain can be monitored in real time through logging, event triggering, and other methods.

[0070] For example, suppose an edge computing scenario has three permission levels: ordinary edge cloud nodes, advanced edge cloud nodes, and management edge cloud nodes. Dividing the regions according to permission levels can divide the edge cloud nodes into three different levels of cloud node availability domains:

[0071] Standard Availability Domain: This availability domain is suitable for ordinary edge cloud nodes. It has the lowest level of privileges and can only access and operate edge cloud nodes and resources belonging to this availability domain. For example, it can perform some basic operations, such as uploading files and accessing specific applications.

[0072] Advanced Availability Domain: Advanced edge cloud nodes have higher privileges than regular edge cloud nodes, allowing them to access and operate edge cloud nodes and resources belonging to this availability domain. For example, they can perform advanced operations such as configuring network settings and managing containers.

[0073] Manage Availability Domains: This feature grants the highest privileges to manage edge cloud nodes, allowing access to and management of all edge cloud nodes and resources. For example, it enables advanced operations such as troubleshooting, system configuration, and access control, as well as monitoring the overall operation of the edge computing environment.

[0074] By dividing the system into regions and setting permissions, edge cloud nodes at different levels can operate within their respective cloud node availability domains, ensuring system security and flexibility, and effectively managing and controlling the use of edge computing resources.

[0075] To divide all edge cloud nodes in the same cluster into regions based on the number of users on each edge cloud node, the following steps can be taken:

[0076] 2.1) User Count Statistics: First, the number of users on the edge cloud nodes is counted to determine the scale of users at different levels, such as a small number of users, a medium number of users, and a large number of users.

[0077] 2.2) Divide cloud node availability zones: Divide edge cloud nodes with the same number of users in the same cluster into the same region. For example, based on the number of users, divide edge cloud nodes into groups with a small number of users, a medium number of users, and a large number of users. Then, based on the regional division results of edge cloud nodes, obtain availability zones for a small number of users, medium number of users, and a large number of users, ensuring that each availability zone contains edge cloud nodes with the same number of users.

[0078] Optionally, corresponding resource sharing policies can be configured for cloud node availability domains with different user numbers to ensure that users of different numbers can share the corresponding resources, avoiding over-allocation or under-allocation of resources. Simultaneously, corresponding management and monitoring mechanisms can be established to ensure effective management and monitoring of cloud node availability domains with different user numbers. For example, resource utilization monitoring and user access logs can be used to monitor the resource usage and performance indicators of each availability domain level in real time.

[0079] By configuring regional division and resource sharing strategies, the needs of edge cloud nodes with different numbers of users can be effectively met, improving system availability and performance, while also achieving rational utilization and management of resources.

[0080] This application's embodiments divide cloud node availability zones based on edge cloud node permissions or the number of users, enabling distributed deployment of multiple edge cloud nodes. This achieves better load balancing, resulting in a more even load distribution among edge cloud nodes in different regions. Simultaneously, it also improves task processing efficiency.

[0081] Based on the above embodiments, after determining the data upload frequency of the edge cloud node based on its information capacity, the method further includes:

[0082] Step 311: Send the scheduling policy to all edge cloud nodes in the same cloud node availability domain so that the edge cloud nodes can back up the scheduling policy;

[0083] Step 312: When an abnormal edge cloud node is detected, a task processing request is sent to other edge cloud nodes in the available domain of the cloud node where the abnormal edge cloud node is located, so that the other edge cloud nodes execute the task processing request based on the pre-backed-up scheduling strategy.

[0084] In the central cloud management system, the central cloud node sends scheduling policies to all edge cloud nodes in the same cloud node availability domain through a distributed management framework. Each edge cloud node, upon receiving the scheduling policy, backs it up to ensure it can execute tasks according to the policy when needed. The system periodically monitors the status of each edge cloud node. If an edge cloud node exhibits an anomaly (such as downtime or unresponsiveness), the system reacts promptly. When the central cloud node detects an anomaly in an edge cloud node, it sends task processing requests to other edge cloud nodes in the cloud node availability domain where the anomaly edge cloud node resides. Other normally functioning edge cloud nodes then execute the task processing requests according to the pre-backed-up scheduling policy.

[0085] It is understandable that edge cloud nodes within the same cloud node availability domain have backup data scheduling policies. This means that even if one edge cloud node fails, other edge cloud nodes within the same cloud node availability domain can still access and utilize the backup scheduling policies. The backup scheduling policies contain information such as task distribution and execution, so other normal edge cloud nodes can continue to execute tasks that were originally performed by the failed node based on this information.

[0086] For example, suppose there is a distributed edge computing system consisting of one central cloud node and three edge cloud nodes, communicating via a single-line connection. The central cloud node formulates a scheduling policy, which needs to be distributed to all edge cloud nodes for execution. Using a distributed management framework built with Dubbo and Zookeeper, the central cloud node distributes the scheduling policy to each edge cloud node via remote invocation. Simultaneously, the distributed management framework backs up the scheduling policy to the availability domain of the cloud nodes where the edge cloud nodes reside. Suppose one edge cloud node fails and cannot execute the scheduling policy normally. However, since there are other normally functioning edge cloud nodes in the availability domain of the same cloud node, these nodes can take over the tasks of the failed node through the backup scheduling policy in the distributed management framework, ensuring continuous system operation and timely recovery. Based on this, the distributed management framework ensures the multi-point deployment of edge cloud nodes, improving system availability and fault tolerance, while also guaranteeing data synchronization within the same cloud node availability domain.

[0087] Once the abnormal / faulty edge cloud node is repaired, the system can rebalance task allocation to ensure that the system returns to normal working status.

[0088] This application embodiment utilizes a distributed management framework to perform hot backup of the scheduling strategy for edge cloud nodes, reducing the impact of single-line failures on these nodes. Simultaneously, by promptly transferring task processing requests, it improves system stability and reliability. The backup scheduling strategy and distributed management framework ensure system flexibility and rapid response capabilities, enabling timely transfer of tasks to other healthy nodes when an edge cloud node fails, thereby preventing overall system performance degradation.

[0089] Based on the above embodiments, after determining the data upload frequency of the edge cloud node based on its information capacity, the method further includes:

[0090] Step 321: Monitor the data transmission status of the edge cloud node;

[0091] Step 322: Share the data transmission status to all edge cloud nodes in the same cloud node availability domain.

[0092] The system monitors data transmission between edge cloud nodes and their availability zones, between cloud node availability zones and central cloud nodes, and between edge cloud nodes and central cloud nodes through a pre-defined monitoring mechanism to ensure the stability and reliability of data transmission. The system then determines the data transmission status of the edge cloud nodes based on the monitoring results.

[0093] The system utilizes a distributed management framework to share workflows among all edge cloud nodes within the same cloud node availability domain. This includes sharing the monitored data transmission status of edge cloud nodes and allowing other edge cloud nodes to read the workflow status of other edge cloud nodes within the same availability domain. This enables status information sharing and collaboration among nodes, allowing for better adjustment of their own status. For example, when an edge cloud node detects a change in data transmission status or a problem, it can share this information with other edge cloud nodes through the distributed management framework, informing them of the current situation. Other edge cloud nodes can then take appropriate action, such as adjusting their workflows to adapt to the new data transmission status or assisting in handling problematic data transmission tasks.

[0094] The StateSynchronizer component synchronizes read and write operations across multiple edge cloud nodes within the same cloud node's availability domain, ensuring data state consistency and preventing data conflicts and inconsistencies. The StateSynchronizer component is a technical component used to synchronize data state in distributed systems. In edge computing systems, when multiple edge cloud nodes need to share data and maintain consistency, the StateSynchronizer component ensures data state consistency across all edge cloud nodes.

[0095] For example, suppose in an edge computing system, there are three edge cloud nodes (A, B, and C) located in the same cloud node availability domain and connected to the central cloud node. The system uses a pre-defined monitoring mechanism to monitor data transmission between them. When node A fails during task execution, the distributed management framework notifies nodes B and C to take over node A's tasks. Nodes B and C can read node A's working status, such as task progress and data processing results, through a worker process sharing mechanism, and adaptively adjust their own states as needed to complete unfinished tasks. During this process, the StateSynchronizer component ensures data state synchronization between nodes B and C to maintain data consistency. For example, if node B has already processed part of the task, and node C needs to take over that part, StateSynchronizer ensures that node C obtains the latest task data and status information, thus avoiding data corruption or duplicate processing. Through this mechanism, the system can guarantee that when an edge cloud node fails or its state changes, other edge cloud nodes can respond promptly and maintain data transmission consistency, thereby improving the system's reliability and stability.

[0096] This application embodiment enables various edge cloud nodes to work collaboratively by sharing data transmission status, maintaining data transmission consistency, and promptly handling anomalies, ultimately improving the reliability and efficiency of the entire system. Based on this, the resources and information of each node in the system can be effectively utilized to achieve better data transmission monitoring and management.

[0097] To further explain the data transmission method based on edge cloud node deployment proposed in this application, please refer to the following embodiments.

[0098] This application embodiment uses a distributed management framework built with Dubbo+Zookeeper to perform distributed scheduling of edge cloud nodes, and combines edge computing technology to deploy massive amounts of metadata in a hierarchical manner, thereby effectively reducing data transmission latency and reducing network congestion.

[0099] Understandably, Dubbo is an open-source, high-performance, lightweight distributed service framework that is dedicated to providing reliable Remote Procedure Call (RPC) communication, enabling different applications to call each other over the network and achieving efficient communication and collaboration between distributed systems.

[0100] ZooKeeper is an open-source distributed coordination service framework designed to provide highly reliable distributed coordination capabilities for building and managing distributed systems.

[0101] By combining Dubbo's RPC capabilities with Zookeeper's service registration and discovery functions, communication and scheduling between nodes in a distributed system can be achieved, enabling dynamic allocation and scheduling of resources. Simultaneously, it allows for monitoring and management of nodes within the system, real-time detection of node status changes, and timely adjustments and actions to ensure system stability and high availability.

[0102] This application specifically proposes a method for deploying edge cloud nodes and transmitting data, which mainly includes the following:

[0103] (1) Based on the permissions or number of users of the edge cloud nodes, all edge cloud nodes under the same cluster are divided into regions to obtain different levels of cloud node availability domains. Based on the unique user identification information such as the IP address of the cloud node availability domain, each cloud node availability domain is connected to the central cloud node, thereby forming a one-to-many connection relationship between the central cloud node and the cloud node availability domains of several edge cloud nodes, realizing the distributed deployment of multiple edge cloud nodes.

[0104] (2) Based on the information capacity of the metadata of the edge cloud nodes, the processing superiors of the edge cloud nodes are divided. For example, if the information capacity is within the processing threshold range of the available domain of the cloud node, the receiving superior of the metadata is set to a server in the available domain of the cloud node; if the information capacity exceeds the processing threshold range of the available domain of the cloud node, the receiving superior of the metadata is set to the central cloud node. Based on this, the layered collaborative work of the edge cloud nodes is realized, and the processing tasks of the servers are reasonably allocated. At the same time, the data upload time interval is set according to the information capacity, such as the larger the information capacity, the longer the data upload interval, thereby reducing the data processing pressure of the central cloud node. By using time-sharing data transmission, network congestion is reduced, thereby alleviating the data transmission bandwidth pressure between the central cloud node and the edge cloud node.

[0105] (3) A distributed management framework is built using Dubbo+Zookeeper. For example, a distributed management framework is built based on the single-line connection between each edge cloud node and the central cloud node. When the central cloud node distributes the scheduling policy, the distributed management framework distributes the scheduling policy to the corresponding edge cloud node. At the same time, the scheduling policy is pushed to the cloud node availability domain where the edge cloud node is located for backup. This allows for multi-point deployment of multiple edge cloud nodes in the same cloud node availability domain. When an edge cloud node goes down, it can be remedied in time by other edge cloud nodes in the same cloud node availability domain, preventing single point failure from affecting the operation of the entire system and improving the data synchronization of the same cloud node availability domain.

[0106] (4) A pre-defined monitoring mechanism is used to monitor data transmission throughout its entire lifecycle between edge cloud nodes and cloud node availability domains, between cloud node availability domains and central cloud nodes, and between edge cloud nodes and central cloud nodes. The data transmission status is determined based on the monitoring results. At the same time, a distributed management framework is used to share the working processes of all edge cloud nodes under the same cloud node availability domain. This allows other edge cloud nodes to read the working status of other edge cloud nodes under the same cloud node availability domain, thereby adaptively adjusting their own status based on the reading results to achieve data transmission consistency. For example, the StateSynchronizer component is used to synchronize the reading and writing of processes of multiple edge cloud nodes under the same cloud node availability domain.

[0107] This application's embodiments utilize a distributed management framework to dynamically deploy central cloud nodes, cloud node availability domains, and edge cloud nodes. Based on the actual information capacity of the edge cloud nodes, servers with corresponding processing capabilities are dynamically selected for data processing, effectively reducing data transmission latency. Simultaneously, by monitoring the entire lifecycle of data transmission, the data transmission progress of each edge cloud node is controlled, and the data transmission progress of other edge cloud nodes within the same cloud node availability domain is adjusted, thereby ensuring data consistency within the same cloud node availability domain.

[0108] Figure 2 This is a schematic diagram of the data transmission device deployed based on edge cloud nodes provided in this application, with reference to... Figure 2 The embodiments of this application provide a data transmission device based on edge cloud node deployment, including a region division module 201, an upper-level processing end determination module 202, and a data upload frequency determination module 203.

[0109] The region partitioning module 201 is used to partition all the edge cloud nodes under the same cluster based on the permissions or number of users of the edge cloud nodes, so as to obtain multiple cloud node availability domains.

[0110] The upper-level processing end determination module 202 is used to determine the upper-level processing end of the edge cloud node based on the information capacity of the edge cloud node; the upper-level processing end includes servers and central cloud nodes in the availability domain of the cloud node;

[0111] The data upload frequency determination module 203 is used to determine the data upload frequency of the edge cloud node based on the information capacity of the edge cloud node, so that the edge cloud node uploads the task to be processed to the upper-level processing terminal based on the data upload frequency.

[0112] The data transmission device based on edge cloud node deployment provided in this application embodiment divides all edge cloud nodes under the same cluster into multiple cloud node availability domains based on the permissions or number of users of the edge cloud nodes. It determines the upper-level processing end of each edge cloud node based on its information capacity; the upper-level processing end includes servers and central cloud nodes within the cloud node availability domains. Based on the information capacity of each edge cloud node, it determines the data upload frequency of each edge cloud node, enabling the edge cloud nodes to upload tasks to be processed to the upper-level processing end according to the data upload frequency. This application embodiment achieves distributed deployment of multiple edge cloud nodes by dividing cloud node availability domains, facilitating the management and scheduling of edge cloud nodes. Simultaneously, it dynamically selects servers with corresponding processing capabilities for data processing based on the information capacity of the edge cloud nodes and dynamically determines the data upload frequency, effectively reducing data transmission latency and improving data transmission efficiency.

[0113] In one embodiment, the region division module 201 is further configured to:

[0114] Determine the permission level corresponding to the permissions of the edge cloud node;

[0115] Edge cloud nodes with the same permission level within the same cluster are grouped into the same region to obtain multiple availability domains for the cloud nodes; or...

[0116] Based on the number of users, determine the user quantity level for each edge cloud node;

[0117] Edge cloud nodes with the same number of users in the same cluster are divided into the same region to obtain multiple available domains for the cloud nodes.

[0118] In one embodiment, the upper-level processing terminal determining module 202 is further configured to:

[0119] If the information capacity of the edge cloud node is less than or equal to the data processing threshold of the cloud node's available domain, then the upstream processing end of the edge cloud node is determined to be a server in the cloud node's available domain.

[0120] If the information capacity of the edge cloud node is greater than the data processing threshold of the available domain of the cloud node, then the upper-level processing end of the edge cloud node is determined to be the central cloud node.

[0121] In one embodiment, the data upload frequency determination module 203 is further configured to:

[0122] Determine the capacity level of the information capacity;

[0123] Based on the matching result between the capacity level and the preset upload frequency table, the data upload frequency of the edge cloud node is determined; wherein, the preset upload frequency table stores the mapping information between the capacity level and the data upload frequency.

[0124] In one embodiment, the data upload frequency determination module 203 is further configured to:

[0125] Send a scheduling policy to all edge cloud nodes in the same cloud node availability domain so that the edge cloud nodes can back up the scheduling policy;

[0126] When an abnormal edge cloud node is detected, a task processing request is sent to other edge cloud nodes in the available domain of the cloud node where the abnormal edge cloud node is located, so that the other edge cloud nodes execute the task processing request based on a pre-backed-up scheduling strategy.

[0127] In one embodiment, the data upload frequency determination module 203 is further configured to:

[0128] Monitor the data transmission status of the edge cloud nodes;

[0129] The data transmission status is shared with all edge cloud nodes in the same cloud node availability domain.

[0130] In one embodiment, any of the cloud node availability domains includes multiple edge cloud nodes; the multiple cloud node availability domains are permanently connected to a central cloud node.

[0131] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a data transmission method based on edge cloud node deployment, the method including:

[0132] Based on the permissions or number of users of the edge cloud nodes, the regions of all the edge cloud nodes under the same cluster are divided to obtain multiple cloud node availability domains;

[0133] Based on the information capacity of the edge cloud node, the upper-level processing end of the edge cloud node is determined; the upper-level processing end includes servers and central cloud nodes in the availability domain of the cloud node;

[0134] Based on the information capacity of the edge cloud node, the data upload frequency of the edge cloud node is determined, so that the edge cloud node uploads the task to be processed to the upper-level processing terminal based on the data upload frequency.

[0135] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0136] On the other hand, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the data transmission method based on edge cloud node deployment provided by the above methods, the method comprising:

[0137] Based on the permissions or number of users of the edge cloud nodes, the regions of all the edge cloud nodes under the same cluster are divided to obtain multiple cloud node availability domains;

[0138] Based on the information capacity of the edge cloud node, the upper-level processing end of the edge cloud node is determined; the upper-level processing end includes servers and central cloud nodes in the availability domain of the cloud node;

[0139] Based on the information capacity of the edge cloud node, the data upload frequency of the edge cloud node is determined, so that the edge cloud node uploads the task to be processed to the upper-level processing terminal based on the data upload frequency.

[0140] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data transmission method based on edge cloud node deployment, characterized in that, The method comprises: based on the edge cloud node permission or the number of users, all the edge cloud nodes under the same cluster are regionally divided to obtain a plurality of cloud node available domains; based on the information capacity of the edge cloud node, the superior processing end of the edge cloud node is determined; the superior processing end comprises a server and a central cloud node in the cloud node available domain; based on the information capacity of the edge cloud node, the data upload frequency of the edge cloud node is determined, so that the edge cloud node uploads the to-be-processed task to the superior processing end based on the data upload frequency; based on the edge cloud node permission or the number of users, all the edge cloud nodes under the same cluster are regionally divided to obtain a plurality of cloud node available domains, which comprises: determining the permission level corresponding to the permission of the edge cloud node; the edge cloud nodes with the same permission level under the same cluster are divided into the same region to obtain a plurality of cloud node available domains; or, based on the number of users, the user number level of each edge cloud node is determined; the edge cloud nodes with the same user number level under the same cluster are divided into the same region to obtain a plurality of cloud node available domains; based on the information capacity of the edge cloud node, the superior processing end of the edge cloud node is determined, which comprises: if the information capacity of the edge cloud node is less than or equal to the data processing threshold of the cloud node available domain, the superior processing end of the edge cloud node is determined as the server in the cloud node available domain; if the information capacity of the edge cloud node is greater than the data processing threshold of the cloud node available domain, the superior processing end of the edge cloud node is determined as the central cloud node. 2.The data transmission method based on edge cloud node deployment according to claim 1, characterized in that, based on the information capacity of the edge cloud node, the data upload frequency of the edge cloud node is determined, which comprises: determining the capacity level of the information capacity; based on the matching result of the capacity level and the preset upload frequency table, the data upload frequency of the edge cloud node is determined; wherein the preset upload frequency table stores the mapping information of the capacity level and the data upload frequency. 3.The data transmission method based on edge cloud node deployment according to claim 1, characterized in that, after determining the data upload frequency of the edge cloud node based on the information capacity of the edge cloud node, the method further comprises: sending a scheduling strategy to all the edge cloud nodes in the same cloud node available domain, so that the edge cloud nodes backup the scheduling strategy; when an abnormal edge cloud node is monitored, a task processing request is sent to other edge cloud nodes in the cloud node available domain where the abnormal edge cloud node is located, so that the other edge cloud nodes execute the task processing request based on the pre-backup scheduling strategy. 4.The data transmission method based on edge cloud node deployment according to claim 1, characterized in that, after determining the data upload frequency of the edge cloud node based on the information capacity of the edge cloud node, the method further comprises: monitoring the data transmission state of the edge cloud node; sharing the data transmission state to all the edge cloud nodes in the same cloud node available domain.

5. The data transmission method based on edge cloud node deployment according to claim 1, characterized in that, Any of the cloud node available domains comprises a plurality of edge cloud nodes; a plurality of cloud node available domains are in long connection with one central cloud node.

6. A data transmission apparatus based on an edge cloud node deployment, characterized by, The method comprises: The regional division module is configured to divide all the edge cloud nodes under the same cluster based on the permissions or the number of users of the edge cloud nodes to obtain a plurality of cloud node available domains; The superior processing end determination module is configured to determine a superior processing end of the edge cloud node based on the information capacity of the edge cloud node, wherein the superior processing end comprises a server and a central cloud node in the cloud node available domain; The data upload frequency determination module is configured to determine a data upload frequency of the edge cloud node based on the information capacity of the edge cloud node, so that the edge cloud node uploads a to-be-processed task to the superior processing end based on the data upload frequency. The regional division module is further configured to determine a permission level corresponding to the permission of the edge cloud node, and divide the edge cloud nodes with the same permission level under the same cluster into the same region to obtain the plurality of cloud node available domains. Alternatively, the regional division module is further configured to determine a user number level of each edge cloud node based on the number of users, and divide the edge cloud nodes with the same user number level under the same cluster into the same region to obtain the plurality of cloud node available domains. The superior processing end determination module is further configured to determine the superior processing end of the edge cloud node as the server in the cloud node available domain if the information capacity of the edge cloud node is less than or equal to a data processing threshold of the cloud node available domain, and determine the superior processing end of the edge cloud node as the central cloud node if the information capacity of the edge cloud node is greater than the data processing threshold of the cloud node available domain.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the data transmission method based on the edge cloud node deployment according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the data transmission method based on the edge cloud node deployment according to any one of claims 1 to 5.

Images