Hierarchical distributed computing communication system and method based on cloud edge end integration

Abstract

This invention belongs to the field of distributed computing and multi-level network architecture technology. It discloses a hierarchical distributed computing communication system and method based on cloud-edge-device integration. The system includes: a task classification module, used to divide action computing tasks into high-level, medium-level, and low-level tasks according to their urgency and complexity; and a node classification module, used to classify computing nodes into three categories based on their geographical location, computing power, and communication latency: remote nodes, edge nodes, and central nodes. This invention aims to improve the collaboration efficiency of remote and edge computing nodes and achieve efficient real-time action data processing and action decision-making. By dividing action computing tasks into different levels and coordinating processing among remote, edge, and central nodes, the real-time performance, accuracy, and reliability of action computing are ensured.

Description

Technical Field

[0001] This invention relates to the field of distributed computing and multi-level network architecture technology, and more specifically, to a hierarchical distributed computing communication system and method based on cloud-edge-device integration. Background Technology

[0002] With the development of science and technology, humanity has entered the era of the "Internet of Everything," leading to an explosive growth in the number of mobile devices and IoT devices requiring connectivity. Smart home devices, in-vehicle networks, wearable health devices, and smart manufacturing devices require complex calculations to be performed through cloud computing centers, which they themselves cannot perform. Significant data processing and data transmission latency exists between cloud computing centers and these smart devices. To address this issue, edge servers are added between cloud computing centers and on-site smart devices. This allows for low-latency and high-efficiency event processing. The Edge Computing Consortium (ECC) and the Industrial Internet Consortium (AII) jointly released the Edge Computing Reference Architecture 3.0, which has a three-layer architecture: a cloud computing layer, an edge computing layer, and an on-site device computing layer. According to IDC, global edge computing spending reached $176 billion in 2022, with an annual growth rate of 14.8%; the Chinese edge computing market reached a scale of 180.37 billion yuan in 2024.

[0003] However, existing mobile computing communication systems are not efficient enough in real-time mobile data processing and mobile decision-making, and it is also inconvenient to coordinate mobile computing tasks between remote, edge and central nodes to ensure the real-time performance, accuracy and reliability of mobile computing.

[0004] In view of this, the present invention proposes a hierarchical distributed computing communication system and method based on cloud-edge-device integration to solve the above problems. Summary of the Invention

[0005] To overcome the aforementioned shortcomings of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a hierarchical distributed computing communication system and method based on cloud-edge-device integration, comprising:

[0006] The task classification module is used to classify action computing tasks into high-level tasks, medium-level tasks, and low-level tasks based on their urgency and complexity.

[0007] The node partitioning module is used to classify computing nodes into three categories based on their geographical location, computing power, and communication latency: remote nodes, edge nodes, and central nodes.

[0008] The task allocation and collaborative computing module includes an initial task allocation unit and a task scheduling and collaboration unit. The initial task allocation unit is used to allocate computing nodes according to the task level, and the task scheduling and collaboration unit is used to dynamically adjust the task allocation by monitoring the computing load of each node and the urgency of the task.

[0009] The data synchronization and consistency maintenance module includes a local data processing unit, an asynchronous data synchronization unit, and a consistency maintenance unit.

[0010] The adaptive scheduling and fault tolerance mechanism module includes an adaptive scheduling unit and a fault tolerance mechanism unit. The adaptive scheduling unit is used to autonomously schedule tasks according to changes in the action environment, changes in task priority, and changes in node load. The fault tolerance mechanism unit is used to automatically reallocate tasks when some nodes fail.

[0011] The data transmission communication module is used to transmit the collected data to the edge device in a predetermined format, and then use the cloud computing center to analyze and process the data and transmit it to the end user.

[0012] Furthermore, in the task classification module:

[0013] Action computation tasks classified as high-level tasks are rapidly computed on edge nodes, action computation tasks classified as medium-level tasks are jointly processed by central nodes and edge nodes, and action computation tasks classified as low-level tasks are assigned to remote nodes for processing.

[0014] Furthermore, in the node partitioning module:

[0015] The remote nodes are used for low real-time tasks such as storing historical data, data mining, and strategy research.

[0016] The edge nodes are used to process high real-time tasks;

[0017] The central node is located between the edge nodes and the remote nodes and serves as a central coordination system, allocating tasks among the various parts of the system and handling the tasks of a certain level.

[0018] Furthermore, in the task allocation and collaborative computing module:

[0019] The initial task allocation unit allocates high-level tasks to edge nodes first, medium-level tasks between central and edge nodes, and low-level tasks to remote nodes according to the task level.

[0020] The task scheduling and coordination unit dynamically adjusts task allocation by monitoring the computing load and task urgency of each node. If the computing power of the edge node reaches a bottleneck, some high-level tasks will be transferred to the central node. If the central node is overloaded, medium-level tasks will be assigned to remote nodes first.

[0021] Furthermore, the data synchronization and consistency maintenance module:

[0022] The local data processing unit is used to preprocess and simplify calculations on edge nodes using data provided by local sensors and monitoring equipment.

[0023] The asynchronous data synchronization unit is used to asynchronously transmit part of the calculation results to the central and remote nodes after the edge nodes have completed the calculation.

[0024] The consistency maintenance unit is used to ensure data consistency among nodes through a distributed database and a consistency algorithm.

[0025] Furthermore, in the adaptive scheduling unit:

[0026] The changes in task priority include adjustments to objectives, changes in resources, external factors, risk assessments, stakeholder feedback, schedule updates, and technological changes;

[0027] The node load changes include changes in task allocation, data traffic fluctuations, dynamic resource adjustments, hardware performance differences, network condition changes, and faults.

[0028] Furthermore, the fault tolerance mechanism unit includes multi-replica storage, hot backup, and node failure detection mechanisms;

[0029] The multi-copy storage is used to store multiple copies of data in multiple locations or on multiple devices;

[0030] The hot backup is divided into online backup or dynamic backup. The hot backup refers to the backup performed when the database is running normally and accessible to users.

[0031] The node fault detection mechanism includes heartbeat detection, timeout retransmission and acknowledgment mechanism, log and monitoring information analysis, fault injection testing, and distributed consistency protocol detection.

[0032] Furthermore, the data transmission communication module includes: a data acquisition and transmission unit, a communication architecture and model construction unit, an edge device communication and intent recognition unit, and a data processing and matching unit;

[0033] The data acquisition and transmission unit is used to acquire data through sensors, and transmit the data to the edge device in a predetermined format via wireless transmission or other means. The data is then analyzed and processed by the cloud computing center, and the processed data is transmitted to the end user.

[0034] The communication architecture and model building unit constructs the model by adopting a distributed data transmission framework, wherein the communication architecture is used to realize communication between cloud computing, edge devices, and mobile terminals;

[0035] The edge device communication and intent recognition unit is used to analyze the communication behavior between edge devices to infer intent.

[0036] Furthermore, the data processing and matching unit is used to process the collected data and use the cloud computing center to control multiple edge devices to distribute the massive data collection work to different devices. In the secondary filtering of data, data that meets the requirements is selected according to specific rules or conditions.

[0037] A hierarchical distributed computing and communication method based on cloud-edge-device integration includes the following steps:

[0038] S1. Based on the urgency and complexity of the task, action calculation tasks are divided into high-level tasks, medium-level tasks, and low-level tasks.

[0039] S2. Based on the geographical location, computing power, and communication latency of each computing node in the network, computing nodes are divided into three categories: remote nodes, edge nodes, and central nodes.

[0040] S3. Perform task allocation and collaborative computing, which includes an initial task allocation unit and a task scheduling and collaboration unit. The initial task allocation unit is used to allocate computing nodes according to the task level, and the task scheduling and collaboration unit is used to dynamically adjust the task allocation by monitoring the computing load and task urgency of each node.

[0041] S4. Perform data synchronization and consistency maintenance, which includes a local data processing unit, an asynchronous data synchronization unit, and a consistency maintenance unit.

[0042] S5. Perform adaptive scheduling and fault tolerance, which includes an adaptive scheduling unit and a fault tolerance mechanism unit. The adaptive scheduling unit is used to autonomously schedule tasks according to changes in the action environment, changes in task priority, and changes in node load. The fault tolerance mechanism unit is used to automatically redistribute tasks when some nodes fail.

[0043] S6. The collected data is transmitted to the edge device in a predetermined format, and then the cloud computing center analyzes and processes the data and transmits it to the end user.

[0044] The technical effects and advantages of this invention based on a hierarchical distributed computing and communication system and method integrating cloud, edge, and terminal:

[0045] 1. This invention ensures data consistency in task processing through distributed data synchronization and consistency. The fault-tolerance mechanism ensures that tasks can continue to be executed even if some nodes fail. The system supports multi-level node deployment, is suitable for mobile networks of different sizes, and can flexibly expand the number of nodes and task complexity according to the needs of the operation.

[0046] 2. This invention aims to improve the collaboration efficiency of remote and edge computing nodes and achieve efficient real-time action data processing and action decision-making. By dividing action computing tasks into different levels and coordinating processing among remote, edge, and central nodes, the real-time performance, accuracy, and reliability of action computing can be ensured. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the hierarchical distributed computing and communication system based on cloud-edge-device integration according to the present invention.

[0048] Figure 2 This is a diagram illustrating the overall architecture of the hierarchical distributed computing and communication system based on cloud-edge-device integration of the present invention.

[0049] Figure 3 This is a schematic diagram of the task hierarchy distribution in the task hierarchy module of the present invention;

[0050] Figure 4 This is a flowchart illustrating the hierarchical distributed computing and communication method based on cloud-edge-device integration of the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Example 1

[0053] Please see Figures 1 to 3 As shown in this embodiment, the hierarchical distributed computing and communication system based on cloud-edge-device integration includes:

[0054] The task classification module categorizes action computing tasks into high-level, medium-level, and low-level tasks based on their urgency and complexity. It's important to note that urgency is determined by personnel based on time sensitivity, impact, and resource urgency. Time sensitivity includes whether the task has a clear time limit, whether it needs immediate initiation or continuous processing, and whether the timeframe is time-sensitive. Impact includes the severity of direct consequences if not handled promptly and whether the problem will rapidly spread to other areas. Resource urgency includes whether special resources need to be allocated immediately and whether there are backup plans to alleviate the emergency. Task complexity is determined using several indicators: structural complexity, rate of change, and predictability. Structural complexity refers to the number of interconnected subsystems involved in the event; rate of change refers to the rapid change in the nature of the problem and the rate at which its impact expands; predictability refers to the availability of historical cases for reference and the influence of unobservable variables on the outcome. Each of these parameters can be assigned a weight by relevant personnel, and the corresponding urgency and complexity can be calculated based on these weights.

[0055] The node partitioning module categorizes computing nodes in the network into three types based on their geographical location, computing power, and communication latency: remote nodes, edge nodes, and central nodes. It's important to note that each of these factors—geographical location, computing power, and communication latency—is assigned a weight. Geographical distance is generally positively correlated with communication latency, so their weights can be relatively low. The calculation formula is: distance * 0.3 + communication latency * 0.2 + computing power * 0.5. The resulting values ​​are then arranged from smallest to largest. Smaller values ​​are designated as edge nodes, intermediate values ​​as remote nodes, and larger values ​​as central nodes.

[0056] The task allocation and collaborative computing module includes an initial task allocation unit and a task scheduling and collaboration unit. The initial task allocation unit is used to allocate computing nodes according to the task level, and the task scheduling and collaboration unit is used to dynamically adjust the task allocation by monitoring the computing load of each node and the urgency of the task.

[0057] The data synchronization and consistency maintenance module includes a local data processing unit, an asynchronous data synchronization unit, and a consistency maintenance unit.

[0058] The adaptive scheduling and fault tolerance mechanism module includes an adaptive scheduling unit and a fault tolerance mechanism unit. The adaptive scheduling unit is used to autonomously schedule tasks according to changes in the action environment, changes in task priority, and changes in node load. The fault tolerance mechanism unit is used to automatically reallocate tasks when some nodes fail.

[0059] The data transmission communication module is used to transmit the collected data to the edge device in a predetermined format, and then use the cloud computing center to analyze and process the data and transmit it to the end user.

[0060] Furthermore, in the task classification module:

[0061] Action computation tasks classified as high-level tasks are rapidly computed on edge nodes, action computation tasks classified as medium-level tasks are jointly processed by central nodes and edge nodes, and action computation tasks classified as low-level tasks are assigned to remote nodes for processing.

[0062] Specifically, high-level tasks such as firepower allocation and target locking require rapid calculations at edge nodes; medium-level tasks such as threat assessment and intelligence analysis are handled jointly by central and edge nodes; and low-level tasks such as post-operation data analysis and historical data storage are assigned to remote nodes for processing.

[0063] Furthermore, in the node partitioning module:

[0064] The remote node is located in the command center or rear base. It has powerful data storage and analysis capabilities and is mainly used for storing historical data, data mining, and low real-time tasks of strategy research.

[0065] The edge nodes are deployed at the front line of operations or in areas close to operations. They have strong real-time computing capabilities and are mainly responsible for handling high real-time tasks, such as firepower calculation, target tracking, and threat prediction.

[0066] The central node is located between the edge nodes and the remote nodes. As a central coordination system, it distributes tasks among the various parts of the system and is responsible for processing some medium-level tasks.

[0067] Furthermore, in the task allocation and collaborative computing module:

[0068] The initial task allocation unit: based on the task level, high-level tasks are preferentially allocated to edge nodes, medium-level tasks are allocated between central nodes and edge nodes, and low-level tasks are allocated to remote nodes.

[0069] The task scheduling and coordination unit: In actual operation, the system dynamically adjusts task allocation by monitoring the computing load and task urgency of each node. If the computing power of the edge node reaches a bottleneck, the system will transfer some high-level tasks to the central node. If the central node is overloaded, the system will prioritize delegating medium-level tasks to remote nodes.

[0070] Furthermore, in the data synchronization and consistency maintenance module:

[0071] It is used to preprocess and simplify calculations on edge nodes using data provided by local sensors and monitoring equipment, so as to reduce the amount of data transmitted;

[0072] The asynchronous data synchronization unit is used to asynchronously transmit part of the calculation results to the central and remote nodes after the action calculation is completed at the edge node, for further data analysis and decision support.

[0073] The consistency maintenance unit is used to ensure data consistency among nodes through distributed databases and consistency algorithms (such as Paxos, Raft, etc.), and to avoid erroneous decisions caused by data asynchrony.

[0074] Specifically, preprocessing includes: data cleaning, data transformation, and data dimensionality reduction. Data cleaning mainly involves: filling in missing data through deletion, interpolation, or prediction; then using statistical methods or machine learning algorithms to identify and process outliers; and finally deleting duplicate records to ensure data uniqueness. Data transformation includes: scaling data to a uniform range (e.g., 0~1); converting categorical data into numerical forms (e.g., one-hot encoding, label encoding); creating new features or selecting important features to improve model performance. Data dimensionality reduction includes: reducing the number of features through linear transformations while retaining key information; visualizing high-dimensional data while preserving local structure; and performing non-linear dimensionality reduction using neural networks. Simplifying computation includes: using GPUs, TPUs, etc., to accelerate computation; using frameworks such as Hadoop and Spark to process large-scale data; and using techniques such as pruning, quantization, and knowledge distillation to reduce model complexity.

[0075] Furthermore, in the adaptive scheduling and fault tolerance mechanism module:

[0076] The adaptive scheduling unit is used to autonomously schedule tasks according to changes in the action environment, changes in task priority, and changes in node load, so as to ensure that the most important action computing tasks are completed first.

[0077] Task priority changes include objective adjustments, resource changes, external factors, risk assessments, stakeholder feedback, schedule updates, and technological changes. Specifically, task priority changes correspond to the task classification module. Objective adjustments refer to changes in task direction or objectives that may lead to an increase or decrease in the priority of certain tasks. Resource changes refer to increases or decreases in manpower, funding, and equipment that will affect task priority. External factors refer to adjustments in priority caused by changes in action requirements, forms, or adversary actions. Risk assessments refer to the identification of new risks or the reassessment of existing risks that may affect task priority. Stakeholder feedback refers to the opinions of management or team members that may affect task priority. Schedule updates refer to the advancement or postponement of certain tasks. Technological changes refer to the emergence of new technologies that may make certain tasks more urgent or unnecessary. The specific ordering of priorities is usually determined based on the urgency of the task, the severity of the threat, and the availability of resources.

[0078] Furthermore, it should be noted that when conducting a risk assessment of the operational environment, attention should be paid to risk identification, that is, identifying potential risks that may affect the safety of the mission or troops. Specifically, this involves analyzing intelligence data, reviewing historical data and lessons learned from similar missions, and listing possible risk sources through brainstorming, expert consultation, etc.; assessing the probability and potential impact of each risk, evaluating the likelihood and severity of risks based on experience and expert judgment, or using mathematical models or statistical tools to calculate the specific probability and degree of impact of risks; determining which risks need to be prioritized, ranking them according to risk level, focusing on high-risk items, and assessing risk tolerance in conjunction with mission objectives and resource constraints; developing measures to mitigate, transfer, or accept risks, adjusting plans to completely avoid risks (such as changing the route of travel), taking measures to reduce the probability or impact of risks (such as strengthening reconnaissance, increasing protection), and transferring risks to other parties (such as...). (Through sharing responsibilities with allies), inaction is chosen for low-probability or low-impact risks; continuous tracking of risk changes ensures the effectiveness of responses, real-time monitoring of the operational environment and risk indicators, regular assessment of risk status and the effectiveness of responses, adjustment of risk assessments based on new intelligence or environmental changes, and updating of risk assessment reports and adjusted response plans; ensuring all relevant personnel understand the risks and their responses, communicating risk assessment results through briefings, reports, or command systems, ensuring commanders at all levels and team members understand their responsibilities and response strategies, and establishing clear risk communication records; recording the risk assessment process and results to provide reference for future missions, archiving risk assessment reports, response plans, and monitoring results, summarizing experiences and lessons learned, and optimizing the risk assessment process; thus, through a systematic risk assessment process, the military can effectively identify and manage potential threats, reduce uncertainty, increase mission success rates, and reduce losses.

[0079] The node load changes include changes in task allocation, data traffic fluctuations, dynamic resource adjustments, hardware performance differences, network condition changes, and faults or anomalies. Specifically, changes in task allocation mean that the task allocation between the cloud, edge, and device may be dynamically adjusted according to demand, leading to an increase or decrease in the load of some nodes. For example, edge computing nodes may experience an increased load due to processing more local data, while the cloud load may decrease accordingly. Data traffic fluctuations mean that fluctuations in data generation and transmission (such as peak periods or sudden traffic) will cause changes in node load. For example, when IoT devices generate a large amount of data during a specific period, the load on edge nodes will increase significantly. Dynamic resource adjustments mean that resource adjustment strategies (such as load balancing and task migration) may lead to a redistribution of load among nodes. Hardware performance differences mean that the hardware performance (such as computing power and storage capacity) of different nodes leads to uneven load distribution. Network condition changes mean that network latency, bandwidth limitations, or interruptions may prevent tasks from being allocated to the cloud in a timely manner, increasing the load on edge nodes. Faults or anomalies mean that a fault in a node may cause other nodes to take over its tasks, increasing the load.

[0080] The fault tolerance mechanism unit is used to ensure high availability and reliability of the system by automatically redistributing tasks when some nodes fail through multi-replica storage, hot backup and node failure detection mechanism.

[0081] This refers to multiple copies of data stored in multiple locations or on multiple devices. Specifically, it's a data storage strategy designed to improve data reliability and availability. Its purpose is to prevent data loss due to single points of failure and ensure data accessibility even if some storage nodes fail. By distributing read requests across multiple copies, system performance is improved. This can be achieved through synchronous or asynchronous replication. Synchronous replication updates all copies simultaneously when data is written, ensuring consistency but potentially increasing latency. Asynchronous replication updates copies slightly later after data is written, reducing latency but potentially causing temporary inconsistencies.

[0082] Hot backup, also known as online or dynamic backup, refers to backup performed while the database is running normally and accessible to users. Specifically, this backup method does not require stopping the database service; it allows for real-time backup while the system is running. Even if data is being edited or read, the backup will not generate new data, providing continuous data protection. Hot backup requires the database to operate in archivelog mode to ensure data consistency. Users can still access the database throughout the backup process, facilitating workflows in multi-user systems.

[0083] The node failure detection mechanism includes heartbeat detection, timeout retransmission and acknowledgment mechanism, log and monitoring information analysis, fault injection testing, and distributed consistency protocol detection. Specifically, heartbeat detection involves nodes periodically sending heartbeat signals to other nodes or the central server to indicate normal operation. If the receiver does not receive a heartbeat signal within a specified time, it may consider the sending node to be faulty. The timeout retransmission and acknowledgment mechanism involves a node starting a timer after sending data or a request, waiting for an acknowledgment response from the receiver. If no acknowledgment is received within the set time, the data or request is retransmitted. Multiple failed retransmissions may indicate a node failure. Log and monitoring information analysis involves nodes recording detailed logs, including operations and errors. The monitoring system collects logs and node performance metrics, such as CPU and memory usage. By analyzing logs and metric data, abnormal patterns or error messages can be detected to determine if a node is faulty. Fault injection testing involves artificially injecting faults into nodes during system operation, such as simulating network outages or CPU overloads, and observing the system's response and the detection capabilities of other nodes to verify the effectiveness of the fault detection mechanism. Distributed consensus protocol testing is conducted in distributed systems where nodes maintain data consistency through distributed consensus protocols such as Paxos and Raft. If a node fails to reach consensus for an extended period or experiences frequent conflicts during protocol execution, it may indicate a node failure.

[0084] Furthermore, the data transmission communication module includes: a data acquisition and transmission unit, a communication architecture and model building unit, an edge device communication and intent recognition unit, and a data processing and matching unit;

[0085] The data acquisition and transmission unit is used to acquire data through sensors and transmit it wirelessly to edge devices according to predetermined formats (text formats such as CSV, JSON, XML; binary formats such as BIN, Avro, Parquet; database formats such as SQLite, MySQL, PostgreSQL, MongoDB; image formats such as JPEG, PNG, GIF; audio formats such as MP3, WAV, AAC; video formats such as MP4, AVI, MKV; compression formats such as ZIP, RAR, 7z; other formats such as PDF, HTML). The data is then analyzed and processed by a cloud computing center before being transmitted to the end user. Thus, this invention ensures data security and integrity, and avoids excessive latency when transmitting massive amounts of data in real time. The data acquisition and transmission unit, using sensors and edge devices, includes: preprocessing the collected data, classifying it using a unified device, and training a model.

[0086] It should be noted that the communication architecture of this invention needs to be designed according to basic principles such as reliability, scalability, maintainability, and one-to-many communication to realize communication between cloud computing, edge devices, and mobile terminals. For communication between cloud computing and the edge, in order to ensure that data from mobile terminals and commands from the cloud computing center can be transmitted to edge devices in a timely manner, a standard network communication interface can be adopted, based on TCP / IP, using the Client / Socket model to achieve point-to-point data transmission with the Socket system in the client and server programs. For communication between edge devices and mobile terminals, the basic requirement of one-to-many must be met during data collection, fully considering various types of network transmission methods, and pre-setting corresponding interfaces to make the mobile data processing system more complete.

[0087] The model building unit constructs the model using a distributed data transmission framework. Specifically, the model construction process includes: 1) Requirements analysis: determining the specific requirements of the model, such as data transmission volume, latency requirements, fault tolerance, etc., and clarifying the data source and destination; 2) Architecture design: selecting a distributed framework, such as Apache Kafka, Apache Flink, Apache Spark, etc., and designing the system architecture, including data flow, node roles, communication protocols, etc.; 3) Environment setup: preparing servers, network equipment, etc., and deploying the selected distributed framework and related tools; 4) Data partitioning and distribution: dividing the data into multiple parts and distributing them to different nodes, selecting appropriate distribution strategies, such as hash partitioning, range partitioning, etc.; 5) Data transmission: selecting transmission protocols such as TCP / IP and HTTP, and writing code to implement data sending and conversion; 6) Data processing: performing necessary cleaning and conversion during data transmission, utilizing the parallel capabilities of the distributed system to accelerate processing; 7) ... 8) Fault tolerance and recovery: Implement fault tolerance mechanisms, such as data backup and replication mechanisms, and design automatic or manual fault recovery processes; 9) Performance optimization: Ensure load balancing across nodes, optimize network communication, reduce latency, and improve throughput; 10) Testing and verification: Conduct unit testing to verify the functionality of each module, conduct integration testing to verify the collaborative operation of the entire system, and conduct performance testing to verify the system's performance under high load; 11) Deployment and monitoring: Deploy the model to the production environment, monitor the system's operating status, and promptly identify and resolve problems; 12) Maintenance and upgrades: Ensure long-term stable operation of the system, and expand functionality or optimize performance as needed.

[0088] It should be noted that in actual operation, the data is preprocessed according to the preset data format and actual application conditions (the same data processing method as the data synchronization and consistency maintenance module) to form a wireless or wired data transmission method. Currently, the most widely used data acquisition methods are serial port connection spread spectrum chips, module reserved Bluetooth, etc. Different data acquisition methods have different characteristics. Spread spectrum chips have a longer transmission distance (1~20km), lower power consumption, and lower price, but a lower transmission rate (0.018kbit / s~37.5kbit / s); Bluetooth has a shorter transmission distance (80~100m) and higher power consumption, but a faster transmission rate (1Mbit / s) and relatively simpler hardware. In practical applications, the Bluetooth transmission used by sensors and edge devices can be configured using a multi-threaded approach.

[0089] Furthermore, to ensure the effectiveness of data transmission between edge devices and the cloud computing center, certain modifications are needed to the monitoring content of the edge devices. To ensure accurate information transmission, each edge device needs to be configured with a recognizable identifier, such as "year→month→day→serial number." After connecting to the cloud, each edge device can store its unique ID locally and send its access request and device number to the cloud computing center. Once the request number is confirmed to be correct, the cloud computing center can establish a connection and categorize the data into different types (cloud computing center model calibration training data, local device storage data, etc.). Based on this, the server can open a response port and, starting from a sub-thread, perform a series of corresponding operations (edge ​​device communication intent recognition, edge device communication request and number transmission, edge device connection completion with the cloud computing center, cloud computing center data acquisition and feedback, etc.) to achieve bidirectional communication between the server and the client, thereby effectively reducing latency.

[0090] The communication intent identification unit is used to infer the underlying intent (or purpose) by analyzing the communication behavior between edge devices. Specifically, firstly, it needs to collect communication data from the edge devices, including: communication protocols, message content sent and received by the devices, timestamps, and device IDs participating in the communication. Secondly, the collected data is preprocessed for subsequent analysis. The preprocessing process includes: data cleaning, i.e., removing noise and irrelevant data; extracting useful features from the raw data, such as type, frequency, and size; and labeling the data to indicate the intent of each communication if supervised learning is required. Thirdly, machine learning or deep learning techniques are used to identify the behavioral patterns of device communication. Related techniques include: cluster analysis, i.e., using unsupervised learning methods to cluster similar communication behaviors and discover typical patterns; time series analysis, analyzing the time series of communication data to identify periodic or sudden behaviors; and anomaly detection, identifying abnormal communication that does not conform to normal behavior patterns. Fourthly, based on the identified behavioral patterns, the communication intent of the devices is inferred. The methods for identifying intentions include: rule engines, which match communication behaviors with intentions using predefined rules (e.g., if device A sends a specific message to device B, it is inferred to be a "data request"); machine learning models, which train classification models to map communication behaviors to specific intention recognitions (common models include decision trees, random forests, and support vector machines); deep learning models, which can be used for complex communication behaviors such as LSTM and Transformer for intention recognition; fifth, deploying the trained model to edge nodes or terminal devices for real-time inference. During deployment, the model can be lightweighted by compressing complex models into lightweight models suitable for edge devices. Performing model inference on edge nodes reduces reliance on the cloud and lowers latency; finally, integrating the intention recognition module into the existing IoT system and optimizing it. Real-time monitoring of device communication intentions is crucial for timely detection of anomalies. Continuous optimization of the intention recognition model based on user feedback or automatic evaluation ensures the security and privacy of communication data and prevents data leakage.

[0091] The communication process for edge device communication needs and number transmission includes: 1) Local data processing and demand triggering: Edge devices collect and initially process data locally, such as a smart camera analyzing video streams and detecting abnormal events, triggering a communication demand to send alarm information to the cloud or other edge devices; 2) Communication strategy determination: Based on the urgency, type, and system resource status of the data, edge devices determine communication strategies, such as using low-latency 5G networks for data with high real-time requirements, and transmitting non-critical data when the network is idle; 3) Protocol adaptation: Edge devices need to adapt to different communication protocols between the cloud and the edge, converting data into a format that the cloud platform or other edge devices can understand, such as converting local sensor data from a custom format to the MQTT protocol format; 4) Communication with edge nodes: Edge devices first communicate with nearby edge nodes to transmit data to the edge server or edge gateway, possibly through short-range wireless communication technologies such as WIFI and Bluetooth; 5) Edge-to-cloud transmission: After edge nodes aggregate and process data from multiple edge devices, they transmit the data to the cloud through a high-speed network, such as using dedicated lines or the Internet to send data to the cloud data center; 6) Cloud feedback and interaction: The cloud processes the data. After analysis, instructions or feedback information may be sent to edge devices, which are then transmitted back to the edge devices via the network, achieving closed-loop interaction between the cloud, edge, and endpoint. The number transmission process includes: 1) Number allocation and registration: Edge devices are assigned unique identification numbers, such as device IDs or MAC addresses, during production or deployment. These numbers are then registered with the edge management system or cloud platform, associating the numbers with the device's location, functions, and other information; 2) Number mapping and synchronization: The cloud-edge-end system establishes a number mapping table, mapping the edge device numbers to logical identifiers and user identifiers in the cloud, and synchronizing between the cloud, edge, and endpoint to ensure correct identification and synchronization between the two systems. 3) Number carrying in communication: When an edge device initiates communication, it carries its own number and the target device number in the header or a specific field of the data packet for identification and routing by nodes and devices in the network; 4) Number verification and authorization: After receiving the data, the receiver verifies the sender's identity and permissions based on the number, and checks the authorization list on the local machine or in the cloud to confirm whether the device is allowed to communicate and access the corresponding resources; 5) Number update and maintenance: If the number of the edge device needs to be changed due to device replacement, upgrade, or other reasons, the system will update the number information in all parts of the cloud, edge, and terminal to ensure the continuity and accuracy of communication;

[0092] The steps to connect edge devices to the cloud computing center are as follows: First, edge device preparation: 1) Hardware check: Ensure the edge device's hardware is functioning properly, including the processor, memory, storage, network interface, etc., and that there are no hardware faults or damage; 2) Software configuration: Install the necessary operating system, drivers, and communication software, and configure the device's network parameters, such as IP address, subnet mask, network management, DNS server, etc., to enable network access; 3) Security settings: Set the device's security authentication information, such as username, password, digital certificate, etc., for subsequent secure connections with the cloud computing center. Second, cloud computing center configuration: 1) Resource preparation: Allocate computing, storage, network, and other resources to the edge device in the cloud computing center, and create corresponding virtual hosts, storage volumes, and network ports, etc.; 2) Service configuration: Configure services related to communication with the edge device, such as message queues and data storage services, and set communication protocols, data formats, and interface specifications, etc.; 3) Security policy settings: Set security policies in the cloud computing center, such as access control lists (ACLs), to allow specific... Access the edge device via its IP address or MAC address, configure firewall rules, and open the corresponding communication ports. Finally, establish a connection between the edge device and the cloud computing center. 1) Initiate a connection request: The edge device initiates a connection request to the cloud computing center via the network based on the configured address and port information of the cloud computing center. This request includes the device's authentication information. 2) Authentication: After receiving the request, the cloud computing center authenticates the edge device based on the pre-configured authentication information. If authentication is successful, the connection process continues; otherwise, the connection is rejected. 3) Handshake and negotiation: After authentication, the edge device and the cloud computing center perform a handshake and negotiation to determine communication parameters, such as the protocol version used and the data encryption method. 4) Connection establishment: After the handshake and negotiation are completed, the edge device and the cloud computing center establish a connection and can begin data transmission and interaction. 5) Connection monitoring and maintenance: After the connection is established, both parties continuously monitor the connection status. The edge device and the cloud computing center periodically send heartbeat packets to check if the connection is normal. If problems occur, reconnection or fault handling is performed promptly.

[0093] The process by which a cloud computing center acquires data from edge devices and terminal devices is as follows: First, data collection is performed. Edge devices (such as sensors, cameras, and smart terminals) are responsible for collecting raw data (such as temperature, humidity, images, and videos). Preliminary processing (such as filtering, compression, and feature extraction) is conducted at the edge nodes to reduce the amount of data transmitted to the cloud. Next, data transmission occurs. High-efficiency communication protocols (such as MQTT, CoAP, and HTTP / 2) are used to transmit data from the edge devices to the cloud computing center. High-speed network technologies such as 5G and Wi-Fi 6 ensure low latency and high reliability of data transmission. When the network is unstable, edge nodes can cache data and upload it to the cloud after the network recovers. Finally, data storage is performed. The cloud computing center uses distributed storage systems (such as HDFS and object storage) to store massive amounts of data. Depending on the data type and purpose, the data is stored in different databases (such as relational databases, time-series databases, and NoSQL databases).

[0094] It should be noted that after data collection is completed, the data is preprocessed through edge devices, mobile terminals, and the cloud computing center. Specifically, based on this, the cloud computing center controls multiple edge devices, distributing the massive data collection work across different devices, reducing the pressure on the cloud computing center and improving the efficiency of data collection and processing. Simultaneously, edge devices are used as data collection terminals, employing web crawling technology and related algorithms for data collection. After completing all data collection work, the goal is to set up terminals to obtain unified resource positioning within the system. After completing necessary data collection, the aim is to reduce the scale of data processing and analysis. A distance-based method is used to accurately identify data with sufficiently relevant elements (data with inaccurate format and content, data with incorrect logical relationships, data with defects that have been deleted or supplemented, useless data, etc.). The principle of the distance-based method is as follows: for a dataset, once the set distance is exceeded, the object can be identified as anomalous data based on the dataset and distance. After judging and identifying anomalous data, a preprocessing scheme can be edited using Python, i.e., the data type to be analyzed is obtained using the Beautiful Soup database. Necessary post-processing of the data is completed using appropriate tools (such as XPath).

[0095] The data processing and matching unit includes: acquiring data related to system adjustment through data matching and other means, obtaining corresponding identification parameters, and recording the parameters when necessary to accurately calibrate the tracking function and prevent corresponding factors from affecting the operation of the data acquisition channel; after comprehensively and accurately acquiring the initial data, performing secondary filtering on the massive data according to the matching principle, promptly removing abnormal data, judging the integrity and relevance of the data, and determining whether it meets the inspection standards; if it does not meet the standard requirements, the data needs to be filtered again, using data feature functions and related sensors to complete a series of operations such as multi-source data fusion, feature comparison, feature association detection, and feature information system conversion; in actual operation, it is necessary to convert the sensor signals according to the running function to obtain a unified data information signal, and perform necessary noise and blemish filtering after the signal source is clear, so as to obtain more effective data; at the same time, appropriate algorithms should be adopted to strengthen the data processing system, enhance the data monitoring effect, and ensure the effective fusion of extracted signal features and datasets;

[0096] It should be noted that the matching principle refers to filtering out data that meets the requirements based on specific rules or conditions when performing secondary filtering of data; specifically, different matching principles need to be used for different scenarios and different content. For scenarios requiring high precision, exact matching can be used, where data must completely match the preset conditions. For text search and keyword matching, fuzzy matching can be used, where data only needs to partially match the conditions. For filtering data within a specific time period or numerical range, range matching can be used, where data falls within a preset numerical or time range. For complex text matching, regular expression matching can be used, defining complex rules through regular expressions to filter data that conforms to the pattern. For multi-condition filtering, logical matching can be used, combining multiple conditions using logical operators (AND, OR, NOT). For text and image similarity matching, similarity matching can be used, calculating the similarity between data and conditions and filtering data with similarity scores higher than a threshold. For structured data, such as filtering data based on specific field values, field matching can be used, filtering based on the values ​​of the data fields. For complex data analysis scenarios, multi-dimensional matching can be used, combining conditions from multiple dimensions for filtering. For real-time data processing, dynamic matching can be used, dynamically filtering data based on changing conditions.

[0097] In this embodiment, the present invention can ensure data consistency in task processing through distributed data synchronization and consistency. The fault tolerance mechanism unit ensures that the task can continue to be executed even if some nodes fail. The system supports multi-level node deployment, is suitable for action networks of different sizes, and can flexibly expand the number of nodes and task complexity according to action needs.

[0098] Example 2

[0099] Please see Figure 4 As shown in the figure, the hierarchical distributed computing communication method based on cloud-edge-device integration described in this embodiment includes the following steps:

[0100] S1. Based on the urgency and complexity of the task, action calculation tasks are divided into high-level tasks, medium-level tasks, and low-level tasks.

[0101] S2. Based on the geographical location, computing power, and communication latency of each computing node in the network, computing nodes are divided into three categories: remote nodes, edge nodes, and central nodes.

[0102] S3. Perform task allocation and collaborative computing, which includes an initial task allocation unit and a task scheduling and collaboration unit. The initial task allocation unit is used to allocate computing nodes according to the task level, and the task scheduling and collaboration unit is used to dynamically adjust the task allocation by monitoring the computing load and task urgency of each node.

[0103] S4. Perform data synchronization and consistency maintenance, which includes a local data processing unit, an asynchronous data synchronization unit, and a consistency maintenance unit.

[0104] S5. Perform adaptive scheduling and fault tolerance, which includes an adaptive scheduling unit and a fault tolerance mechanism unit. The adaptive scheduling unit is used to autonomously schedule tasks according to changes in the action environment, changes in task priority, and changes in node load. The fault tolerance mechanism unit is used to automatically redistribute tasks when some nodes fail.

[0105] S6. The collected data is transmitted to the edge device in a predetermined format, and then the cloud computing center analyzes and processes the data and transmits it to the end user.

[0106] In this embodiment, the present invention reduces the computational pressure on the central node through hierarchical distributed computing, while edge nodes can process critical action tasks in real time, reducing latency and improving decision-making speed; the system dynamically allocates tasks according to the complexity of the task and the computing power of the node, avoiding single-point overload and improving the overall performance of the system.

[0107] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0108] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only one method, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0109] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

[0110] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A hierarchical distributed computing and communication system based on cloud-edge-device integration, characterized in that: include: The task classification module is used to classify action computing tasks into high-level tasks, medium-level tasks, and low-level tasks based on their urgency and complexity. The urgency level is determined by a weighted calculation of time sensitivity, impact level, and resource demand urgency; the complexity is determined by a weighted calculation of structural complexity, dynamic change rate, and predictability. The node partitioning module categorizes computing nodes into three types based on their geographical location, computing power, and communication latency: remote nodes, edge nodes, and central nodes. Node classification is performed using the formula: distance * 0.3 + communication latency * 0.2 + computing power * 0.

5. Within this module: remote nodes are located in command centers or rear bases, possessing powerful data storage and analysis capabilities, primarily used for low-real-time tasks such as storing historical data, data mining, and strategy research; edge nodes are deployed at the front lines or near operational areas, possessing strong real-time computing capabilities, primarily responsible for handling high-real-time tasks, including firepower calculations, target tracking, and threat prediction; and central nodes are located between edge and remote nodes, serving as a central coordination system, distributing tasks among different parts of the system and handling some medium-level tasks. The task allocation and collaborative computing module includes an initial task allocation unit and a task scheduling and coordination unit. The initial task allocation unit allocates computing nodes according to the task's priority: high-priority tasks are preferentially allocated to edge nodes for fast computation; medium-priority tasks are processed jointly by central and edge nodes; and low-priority tasks are assigned to remote nodes. The task scheduling and coordination unit dynamically adjusts task allocation by monitoring the computational load and task urgency of each node. Specifically, if the computational capacity of an edge node reaches a bottleneck, some high-priority tasks are transferred to the central node; if the central node is overloaded, medium-priority tasks are preferentially delegated to remote nodes. The data synchronization and consistency maintenance module includes a local data processing unit, an asynchronous data synchronization unit, and a consistency maintenance unit; the consistency maintenance unit ensures data consistency among nodes through a distributed database and a consistency algorithm. The adaptive scheduling and fault tolerance mechanism module includes an adaptive scheduling unit and a fault tolerance mechanism unit. The adaptive scheduling unit is used to autonomously schedule tasks based on changes in the operational environment, changes in task priorities, and changes in node load. The changes in task priorities include target adjustments, resource changes, external factors, risk assessments, stakeholder feedback, progress updates, and technological changes. The changes in node load include changes in task allocation, data traffic fluctuations, dynamic resource adjustments, hardware performance differences, network condition changes, and faults. The fault tolerance mechanism unit includes multi-replica storage, hot backup, and node fault detection mechanisms to automatically reallocate tasks when some nodes fail. The data transmission communication module is used to transmit the collected data to the edge device in a predetermined format, and then use the cloud computing center to analyze and process the data and transmit it to the end user. The data transmission communication module includes a data acquisition and transmission unit, a communication architecture and model construction unit, an edge device communication and intent recognition unit, and a data processing and matching unit. The edge device communication and intent recognition unit is used to analyze the communication behavior between edge devices to infer intent.

2. The hierarchical distributed computing and communication system based on cloud-edge-device integration according to claim 1, characterized in that, In the data synchronization and consistency maintenance module: The local data processing unit is used to preprocess and simplify calculations on edge nodes using data provided by local sensors and monitoring equipment; the preprocessing includes data cleaning, data transformation, and data dimensionality reduction. The asynchronous data synchronization unit is used to asynchronously transmit part of the calculation results to the central and remote nodes after the edge nodes have completed the calculation.

3. The hierarchical distributed computing and communication system based on cloud-edge-device integration according to claim 1, characterized in that, The multi-replica storage is used to store multiple copies of data in multiple locations or devices, which is achieved through synchronous or asynchronous replication. The hot backup is divided into online backup or dynamic backup. The hot backup refers to the backup performed when the database is running normally and accessible to users. The node fault detection mechanism includes heartbeat detection, timeout retransmission and acknowledgment mechanism, log and monitoring information analysis, fault injection testing, and distributed consistency protocol detection.

4. The hierarchical distributed computing and communication system based on cloud-edge-device integration according to claim 1, characterized in that, The data acquisition and transmission unit is used to acquire data through sensors and transmit the data wirelessly to the edge device in a predetermined format. The data is then analyzed and processed by the cloud computing center and transmitted to the end user. The communication architecture and model building unit builds the model by adopting a distributed data transmission framework, wherein the communication architecture is used to realize communication between cloud computing, edge devices, and mobile terminals; The edge device communication and intent recognition unit analyzes the communication behavior between edge devices through machine learning or deep learning techniques to infer intent, including cluster analysis, time series analysis, and anomaly detection.

5. The hierarchical distributed computing and communication system based on cloud-edge-device integration according to claim 4, characterized in that, The data processing and matching unit is used to process the collected data and use the cloud computing center to control multiple edge devices to distribute the massive data collection work to different devices. In the secondary filtering of data, data that meets the requirements is selected based on matching rules or conditions such as exact matching, fuzzy matching, range matching, regular expression matching or similarity matching.

6. A hierarchical distributed computing and communication method based on cloud-edge-device integration, characterized in that: The hierarchical distributed computing and communication system based on cloud-edge-device integration as described in any one of claims 1 to 5 is characterized by comprising the following steps: S1. Based on the urgency and complexity of the task, the action calculation task is divided into high-level tasks, medium-level tasks and low-level tasks. The urgency is calculated by setting weights for time sensitivity, impact and resource demand urgency, and the complexity is calculated by setting weights for structural complexity, dynamic change rate and predictability. S2. Based on the geographical location, computing power, and communication latency of each computing node in the network, the computing nodes are divided into three categories: remote nodes, edge nodes, and central nodes. The classification is carried out using the calculation formula: distance × 0.3 + communication latency × 0.2 + computing power × 0.

5. S3. Perform task allocation and collaborative computing, which includes an initial task allocation unit and a task scheduling and collaboration unit. The initial task allocation unit is used to allocate computing nodes according to the task level, and the task scheduling and collaboration unit is used to dynamically adjust the task allocation by monitoring the computing load and task urgency of each node. S4. Perform data synchronization and consistency maintenance, which includes a local data processing unit, an asynchronous data synchronization unit, and a consistency maintenance unit. Specifically, it includes local data processing, asynchronous data synchronization, and consistency maintenance through Paxos or Raft algorithms. S5. Perform adaptive scheduling and fault tolerance, which includes an adaptive scheduling unit and a fault tolerance mechanism unit. The adaptive scheduling unit is used to autonomously schedule tasks according to changes in the action environment, changes in task priority, and changes in node load. The fault tolerance mechanism unit is used to automatically redistribute tasks through multi-replica storage, hot backup, and node fault detection mechanisms when some nodes fail. S6. The collected data is transmitted to the edge device in a predetermined format, and then the cloud computing center analyzes and processes the data and transmits it to the end user. This includes edge device communication intent recognition and secondary data filtering and matching.

Images