Data processing system and method based on cloud edge collaboration

Abstract

The invention relates to the technical field of data processing, in particular to a data processing system and method based on cloud edge collaboration, and the system comprises an edge calculation module, an area monitoring module and a cloud calculation module. Wherein the edge computing module comprises a plurality of edge computing nodes and is used for carrying out data processing on corresponding edge side original data, sending a data processing result to the cloud computing module and determining a computing power representation state of the edge computing nodes; the region monitoring module comprises a plurality of region collaboration nodes and is used for acquiring computing power representation states of the connected edge computing nodes so as to determine a computing power collaboration strategy; and the cloud computing module is used for acquiring the data processing result and the data co-processing request of each edge computing node, and responding to the data co-processing request. According to the method, the problem of tension of cloud computing power is avoided through cooperative computing between the edge sides.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular, to a data processing system and method based on cloud-edge collaboration. Background Art

[0002] Cloud-edge collaborative data processing technology is a new computing paradigm that has emerged with the development of the Internet of Things (IoT), 5G communication, and artificial intelligence technologies. In scenarios such as industrial IoT and smart cities, if all the data generated by a large number of terminal devices is uploaded to the cloud for processing, it will cause problems such as network congestion and response latency. Therefore, edge computing realizes local real-time processing of data by sinking computing power to the network edge side close to the data source, while cloud computing retains its advantages in large-scale storage, complex model training, and global optimization. Cloud-edge collaboration organically combines the two through a hierarchical architecture. The edge layer is responsible for data collection, filtering, and lightweight computing, the cloud layer undertakes in-depth analysis and resource scheduling, and the network layer ensures efficient and reliable data transmission. The key problems to be solved by this technology include dynamic task allocation, data consistency maintenance, security and privacy protection, etc. Its development is promoted by the popularization of 5G networks, the lightweighting of AI algorithms, and the improvement of edge hardware performance, and has become an important infrastructure to support digital transformation.

[0003] The prior art discloses a cloud-edge collaborative system for processing data of IoT devices, including a cloud module and multiple edge node modules, where; the edge node module includes: a data collection sub-module; a data preprocessing sub-module; a local storage sub-module; a data transmission sub-module; a response sub-module; a self-recovery sub-module; an autonomous decision-making sub-module; the cloud module includes: a cloud receiving sub-module; a cloud analysis sub-module; a cloud storage sub-module; a cloud control sub-module; the cloud module and the edge node module are connected through a network to achieve two-way data transmission and the transfer of control instructions. This invention ensures that the system can automatically isolate problem areas, restart services, and verify the recovery effect when a failure occurs, greatly improving the stability and reliability of the system, and reducing manual intervention and downtime. It can be seen that the prior art has the following problems:

[0004] The problem of tight cloud computing power during busy hours is not considered due to the lack of collaborative processing between adjacent edge sides. Summary of the Invention

[0005] Therefore, the present invention provides a data processing system and method based on cloud-edge collaboration to overcome the problem of tight cloud computing power during busy hours caused by the lack of collaborative processing between adjacent edge sides in the prior art.

[0006] To achieve the above object, on the one hand, the present invention provides a data processing system based on cloud-edge collaboration, including:

[0007] The edge computing module includes several edge computing nodes. A single edge computing node includes an edge computing unit for processing corresponding raw edge - side data, an edge transmission unit for sending the data processing result to the cloud computing module, and a computing power monitoring unit for determining the computing power characterization state of the edge computing node;

[0008] The area monitoring module includes several area cooperation nodes. Each single area cooperation node is respectively connected to several edge computing nodes, and is used to obtain the computing power characterization states of the connected edge computing nodes to determine a computing power cooperation strategy;

[0009] The cloud computing module is connected to the edge transmission units of each edge computing node, and is used to obtain the data processing results and data cooperative processing requests of each edge computing node, and respond to the data cooperative processing requests.

[0010] As a preferred technical solution of the data processing system based on cloud - edge cooperation, the edge computing unit, in response to the acquisition of corresponding raw edge - side data, controls the edge computing unit to determine its data source according to the data identifier and / or communication protocol type of the raw data, so as to perform corresponding data analysis and processing.

[0011] As a preferred technical solution of the data processing system based on cloud - edge cooperation, the computing power monitoring unit determines the computing power characterization state of the node according to the multi - dimensional resources of the corresponding edge computing node;

[0012] Among them, the multi - dimensional resources include CPU utilization rate, memory occupancy rate, and GPU utilization rate, and the computing power characterization state includes a computing power tension state and a computing power idle state.

[0013] As a preferred technical solution of the data processing system based on cloud - edge cooperation, the area monitoring module divides the edge coverage area into several sub - monitoring areas with uniform node density according to the geographical locations of the edge computing nodes through a clustering algorithm, and determines whether the division of the single sub - monitoring area is reasonable according to the historical computing power characterization states of the edge computing nodes within a preset time in the sub - monitoring area.

[0014] As a preferred technical solution of the data processing system based on cloud - edge cooperation, the area monitoring module controls the re - division of the sub - monitoring area according to the determination result that the proportion of the number of unreasonably divided sub - monitoring areas is greater than a preset proportion, so that the number of re - divided sub - monitoring areas is greater than the number of sub - monitoring areas before re - division.

[0015] As a preferred technical solution of the data processing system based on cloud - edge cooperation, the area monitoring module determines whether the division of the single sub - monitoring area is reasonable according to the historical computing power characterization states of the edge computing nodes within a preset time in the sub - monitoring area, including,

[0016] The area monitoring module determines that the division of the sub-monitoring area is reasonable according to the determination result that the historical computing power representation state meets the area division condition;

[0017] Wherein, the area division condition is that there are a preset number of edge computing nodes in the sub-monitoring area, and the duration ratio of the computing power tension state of the edge computing nodes is less than or equal to the reference ratio, and the preset time is at least one week.

[0018] As a preferred technical solution of the cloud-edge collaborative data processing system, the area monitoring module sets at least one area collaboration node in any sub-monitoring area, and the area collaboration nodes are respectively connected to the computing power monitoring units of the edge computing nodes in the sub-monitoring area to obtain the computing power representation states of the corresponding edge computing nodes, and determine whether the corresponding edge computing nodes need to formulate a computing power collaboration strategy according to the computing power representation states.

[0019] As a preferred technical solution of the cloud-edge collaborative data processing system, the area monitoring module determines the computing power collaboration strategy according to the node quantity ratio of the edge computing nodes that formulate the computing power collaboration strategy to the edge computing nodes that do not formulate the computing power collaboration strategy, including,

[0020] According to the determination result that the node quantity ratio is greater than the first preset quantity ratio, control the edge computing nodes in the computing power tension state to transmit data processing tasks and data processing results to the edge computing nodes in the computing power idle state, so that the edge computing nodes in the computing power idle state execute the data processing tasks and transmit all data processing results to the cloud computing module.

[0021] As a preferred technical solution of the cloud-edge collaborative data processing system, the area monitoring module determines the computing power collaboration strategy according to the node quantity ratio of the edge computing nodes that formulate the computing power collaboration strategy to the edge computing nodes that do not formulate the computing power collaboration strategy, including,

[0022] According to the determination result that the node quantity ratio is greater than the second preset quantity ratio, control the edge computing nodes in the computing power tension state to transmit data processing tasks and data collaborative processing requests to the cloud computing module;

[0023] Wherein, the second preset quantity ratio is greater than the first preset quantity ratio.

[0024] On the other hand, the present invention also provides a cloud-edge collaborative data processing method, including:

[0025] In response to the acquisition of the original data on the edge side, control the corresponding edge computing unit to determine its data source according to the data identifier and / or communication protocol type of the original data, so as to perform corresponding data analysis and processing;

[0026] The computing power monitoring unit determines the computing power representation status of the corresponding edge computing node according to the multi-dimensional resources of the edge computing node;

[0027] According to the computing power representation status of each corresponding edge computing node in a single sub-monitoring unit, determine whether the corresponding edge computing node needs to formulate a computing power collaboration strategy;

[0028] Determine the computing power collaboration strategy according to the ratio of the number of edge computing nodes that formulate the computing power collaboration strategy to the number of edge computing nodes that do not formulate the computing power collaboration strategy, including,

[0029] According to the determination result that the ratio is greater than the first preset ratio, control the edge computing node in a computing power tension state to transmit data processing tasks and data processing results to the edge computing node in a computing power idle state, so that the edge computing node in a computing power idle state executes the data processing task and transmits all data processing results to the cloud computing module;

[0030] According to the determination result that the ratio is greater than the second preset ratio, control the edge computing node in a computing power tension state to transmit data processing tasks and data collaboration processing requests to the cloud computing module.

[0031] Compared with the prior art, the beneficial effects of the present invention are that the data processing system based on cloud-edge collaboration provided by the present invention realizes efficient data processing and flexible scheduling of computing power resources by integrating the advantages of edge computing and cloud computing. This system improves the real-time performance and accuracy of data processing, and optimizes the allocation and utilization of computing power resources;

[0032] In particular, the area monitoring module reasonably divides the sub-monitoring area through comprehensive consideration of the multi-dimensional resources of the edge computing node and the clustering algorithm, which not only realizes the accurate monitoring and efficient utilization of computing power resources, but also significantly improves the flexibility and adaptability of the system;

[0033] In particular, by flexibly setting area collaboration nodes in the sub-monitoring area through the area monitoring module and dynamically adjusting the computing power collaboration strategy according to the computing power representation status, it not only realizes the efficient computing power collaboration between edge computing nodes, but also ensures the reasonable utilization of cloud computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a connection diagram of the data processing system based on cloud-edge collaboration according to an embodiment of the present invention;

[0035] Figure 2 It is a working flow chart of the area monitoring module dividing the sub-monitoring area according to an embodiment of the present invention;

[0036] Figure 3 It is a working flow chart of the area monitoring module determining the computing power collaboration strategy according to an embodiment of the present invention;

[0037] Figure 4 This is a step diagram of the data processing method based on cloud-edge collaboration according to an embodiment of the present invention. Detailed implementation manners

[0038] In order to make the objectives and advantages of the present invention clearer and more understandable, the present invention will be further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0039] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention and do not limit the protection scope of the present invention.

[0040] It should be noted that in the description of the present invention, the terms indicating the direction or positional relationship such as "upper", "lower", "left", "right", "inner", "outer", etc. are based on the direction or positional relationship shown in the drawings. This is only for convenience of description and does not indicate or imply that the device or component must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention.

[0041] In addition, it should also be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installation", "connection", and "connection" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0042] Please refer to Figure 1 As shown, it is a connection diagram of the data processing system based on cloud-edge collaboration according to an embodiment of the present invention. An embodiment of the present invention provides a data processing system based on cloud-edge collaboration, including:

[0043] An edge computing module, including a plurality of edge computing nodes. A single edge computing node includes an edge computing unit for processing the corresponding raw data on the edge side, an edge transmission unit for sending the data processing result to the cloud computing module, and a computing power monitoring unit for determining the computing power characterization state of the edge computing node; it can be understood that the edge transmission unit of any edge computing node can also perform wireless communication with other edge computing nodes in the same sub-monitoring area according to the instructions of the area monitoring module, so that the nodes in the computing power idle state share the data processing tasks of the nodes in the computing power tense state;

[0044] The area monitoring module includes several area cooperation nodes. Each individual area cooperation node is respectively connected to several edge computing nodes, and is used to obtain the computing power representation status of each connected edge computing node to determine the computing power cooperation strategy. It can be understood that the computing range covered by each edge computing node is the edge coverage area. The area monitoring module divides the edge coverage area into several sub-monitoring areas, and each sub-monitoring area has at least one area cooperation node (preferably, there is one area cooperation node in a sub-monitoring area).

[0045] The cloud computing module is connected to the edge transmission units of each edge computing node, and is used to obtain the data processing results and data cooperation processing requests of each edge computing node, and respond to the data cooperation processing requests. It can be understood that the cloud computing module usually receives and stores the data processing results on the edge side, and performs overall subsequent resource optimization / scheduling according to the data processing results.

[0046] It can be understood that the edge computing node can directly process the original data on the edge side, reducing the latency of data transmission to the cloud, thereby improving the real-time performance of data processing. At the same time, through the computing power cooperation strategy of the area monitoring module, nodes with idle computing power can share the tasks of nodes with tight computing power, further improving the overall data processing ability. The computing power monitoring unit can monitor the computing power representation status of the edge computing node in real time, and the area monitoring module formulates the computing power cooperation strategy based on this information. This mechanism ensures that the computing power resources can be flexibly scheduled according to actual needs, avoiding the waste of computing power resources and the occurrence of bottleneck phenomena.

[0047] Specifically, in response to the acquisition of the corresponding original data on the edge side, the edge computing unit controls the edge computing unit to determine its data source according to the data identifier and / or communication protocol type of the original data, so as to perform corresponding data analysis and processing.

[0048] It can be understood that the original data streams of different industrial devices each have their own data identifiers and communication protocols. According to the data identifiers and communication protocols, it can be determined from which industrial device the original data stream comes. After preprocessing the original data stream, these data can be used according to some data processing / early warning / analysis logics preset on the edge side, and all these are completed on the edge side. Therefore, the computing power on the edge side is required.

[0049] In implementation, when there is a large amount of original data, it will cause the computing power on the edge side to be tight.

[0050] It can be understood that the edge computing unit can intelligently identify and process the raw data from different industrial devices. By effectively utilizing data identification and communication protocols, it not only improves the pertinence and efficiency of data processing, but also can flexibly handle the situation of tight computing power when facing a large amount of raw data, ensuring the continuity and stability of data processing;

[0051] Please refer to Figure 2 shown in the figure, which is the working flowchart of the sub-monitoring area division of the area monitoring module in the embodiment of the present invention. Specifically, the computing power monitoring unit determines the computing power representation state of the corresponding edge computing node according to the multi-dimensional resources of the edge computing node;

[0052] Among them, the multi-dimensional resources include CPU utilization rate, memory occupancy rate and GPU utilization rate, and the computing power representation state includes a computing power tight state and a computing power idle state.

[0053] It can be understood that determining the computing power representation state of the corresponding node through one or more of the CPU utilization rate, memory occupancy rate and GPU utilization rate is the prior art, so it will not be elaborated here; in practice, any method in the prior art can be used to determine the computing power representation state.

[0054] It can be understood that the computing power monitoring unit can comprehensively and accurately reflect the computing power representation state of the edge computing node, including the computing power tight state and the computing power idle state; this accurate monitoring ability provides reliable data support for the area monitoring module to formulate a computing power collaboration strategy, ensuring the reasonable allocation and utilization of computing power resources.

[0055] Specifically, the area monitoring module divides the edge coverage area into several sub-monitoring areas with uniform node density according to the geographical locations of the edge computing nodes through a clustering algorithm, and determines whether the division of the sub-monitoring area is reasonable according to the historical computing power representation states of the edge computing nodes in a single sub-monitoring area within a preset time.

[0056] It can be understood that the K-Means clustering algorithm or the DBSCAN density clustering algorithm can be selected to divide the edge coverage area into several sub-monitoring areas. Both of them are the prior art and will not be elaborated here; in practice, K in the K-Means clustering algorithm is at least 5; the minimum number of points MinPts in the DBSCAN density clustering algorithm is at least 5, and the radius Eps is determined according to the edge coverage area (the radius Eps is proportional to the area of the edge coverage area);

[0057] It can be understood that after dividing several sub-monitoring areas, the historical computing power representation states of the edge computing nodes in each sub-monitoring area are respectively determined to determine whether the division of the sub-monitoring area is reasonable.

[0058] Specifically, the area monitoring module controls the re - division of sub - monitoring areas according to the determination result that the proportion of unreasonably divided sub - monitoring areas is greater than the preset proportion, so that the number of sub - monitoring areas after re - division is greater than the number of sub - monitoring areas before re - division.

[0059] It can be understood that when there are fewer unreasonably divided sub - monitoring areas, standby edge computing nodes can be added to the corresponding sub - monitoring areas to ensure the computing power of the sub - monitoring areas; but if most of the sub - detector areas are unreasonably divided, it means that the current division is unreasonable and needs to be re - divided.

[0060] In implementation, the preset proportion ≥ 30%, preferably set to 40%; it can be understood that the smaller the preset proportion is set, the more reasonable the finally divided sub - monitoring areas will be.

[0061] Specifically, the area monitoring module determines whether the division of a single sub - monitoring area is reasonable according to the historical computing power representation status of each edge computing node in the sub - monitoring area within a preset time, including,

[0062] The area monitoring module determines that the division of the sub - monitoring area is reasonable according to the determination result that the historical computing power representation status meets the area division condition;

[0063] Among them, the area division condition is that there are at least a preset number of edge computing nodes in the sub - monitoring area, and the proportion of the duration of the computing power tension state of the edge computing node (i.e., the preset number of edge computing nodes) (i.e., the ratio of the duration of the computing power tension state within the preset time to the length of the preset time) is less than or equal to the reference proportion, and the preset time is at least one week.

[0064] In implementation, the preset number ≥ 95% of the number of edge computing nodes in the sub - monitoring area. That is, if sub - monitoring area A includes 100 edge computing nodes, then the preset number of this sub - monitoring area is 95;

[0065] In implementation, when the preset time is one week, the preset time period is from the current moment to the previous week; in one implementation, if the current moment is 8:00 am on Sunday of the second week of March 2025, then the preset time period is from 8:00 am on Sunday of the first week of March 2025;

[0066] It can be understood that the number of edge - side computing nodes is increasing continuously, so the scope of the sub - monitoring area should also be updated accordingly, usually once every preset time period.

[0067] In implementation, the reference ratio should be less than or equal to 10%; it can be understood that in the intelligent industrial production scenario, edge computing nodes are mainly responsible for tasks such as real-time monitoring of the operating status of production equipment, fault diagnosis, and generation of control instructions; these tasks have high requirements for real-time performance. Once there is a tight computing power situation, it may lead to equipment control delay, affect product quality, and even cause production accidents; on average, there is 1 to 2 hours of tight computing power per day, but for most of the time (about 22 to 23 hours), the computing power usage is relatively stable. Considering the importance of the business and the strict requirements for real-time performance, in order to ensure the stability and reliability of the production process, the reference ratio is usually set at 10%.

[0068] It can be understood that edge computing nodes that can accurately identify the tight computing power state and idle computing power state within the sub-monitoring area, and through the computing power collaboration strategy, achieve flexible scheduling of computing power resources. This mechanism avoids waste of computing power resources and bottleneck phenomena, and improves the overall utilization rate of computing power resources; through reasonable sub-monitoring area division and accurate computing power monitoring, it can timely detect and respond to the tight computing power situation, ensure the continuity and stability of data processing, and at the same time can be dynamically adjusted according to actual needs, enhancing the adaptability and robustness of the system; in the intelligent industrial production scenario, the system can real-time monitor the operating status of production equipment, perform tasks such as fault diagnosis and generation of control instructions. This real-time monitoring and fast response ability not only improves production efficiency, but also provides strong support for intelligent production and decision-making. By optimizing the utilization of computing power resources, the system can also reduce production costs and improve overall economic benefits.

[0069] Please refer to Figure 3 As shown, it is the working flowchart of the area monitoring module of the embodiment of the present invention for determining the computing power collaboration strategy. Specifically, the area monitoring module sets at least one area collaboration node in any one sub-monitoring area, and the area collaboration node is respectively connected to the computing power monitoring units of each edge computing node in the sub-monitoring area to obtain the computing power representation status of each corresponding edge computing node, and determine whether the corresponding edge computing node needs to formulate a computing power collaboration strategy according to the computing power representation status.

[0070] It can be understood that if the current computing power representation status is a tight computing power state, it is determined that the corresponding edge computing node needs to formulate a computing power collaboration strategy.

[0071] Specifically, the area monitoring module determines the computing power collaboration strategy according to the ratio of the number of edge computing nodes that formulate the computing power collaboration strategy to the number of edge computing nodes that do not formulate the computing power collaboration strategy, including

[0072] According to the determination result that the node quantity ratio is greater than the first preset quantity ratio (in implementation, 40% < the first preset quantity ratio ≤ 65%, preferably, the first preset quantity ratio = 50%. The larger the first preset quantity ratio is, the more stringent the conditions for computing power collaboration are. By choosing a smaller first preset quantity ratio, other computing power idle state nodes within the same sub-monitoring area can be mobilized to collaboratively process the data of the computing power tense state nodes, quickly solving the problems of some computing power tense state nodes within the sub-monitoring area, and avoiding directly transmitting to cloud computing, which causes cloud computing power tension and wastes the computing power of the remaining edge nodes), control the edge computing nodes in the computing power tense state to stop transmitting the data processing results to the cloud computing module (that is, the process of stopping sending to the cloud synchronously sends the data processing results calculated at this node when sending data processing tasks to other edge nodes, and all data processing results are sent to the cloud through the edge nodes assisting in data processing) and transmit the data processing tasks and data processing results to the edge computing nodes in the computing power idle state, so that the edge computing nodes in the computing power idle state execute the data processing tasks and transmit all data processing results to the cloud computing module (including the data processing results of the computing power idle state itself, the data processing results of the computing power idle state assisting the computing power tense state nodes, and the data processing results of the computing power tense state nodes).

[0073] Specifically, the area monitoring module determines the computing power collaboration strategy according to the node quantity ratio of the edge computing nodes formulating the computing power collaboration strategy to the edge computing nodes not formulating the computing power collaboration strategy, including

[0074] According to the determination result that the node quantity ratio is greater than the second preset quantity ratio, control the edge computing nodes in the computing power tense state to transmit the data processing tasks and data collaboration processing requests to the cloud computing module; it can be understood that 70% ≤ the first preset quantity ratio ≤ 85%, preferably, the second preset quantity ratio = 75%. The larger the second preset quantity ratio is, the less data needs to be collaboratively processed by the cloud, that is, the cloud computing module can maintain a relatively strong computing power; in implementation, when the node quantity ratio is greater than 70%, first, the computing power idle state nodes are used to collaboratively process the data processing tasks of the computing power tense nodes (usually, one computing power idle state node collaboratively processes the data processing tasks of one computing power tense node), and the data processing tasks of the remaining computing power tense nodes are sent to the cloud computing module to request the cloud for data collaborative processing;

[0075] Among them, the second preset quantity ratio is greater than the first preset quantity ratio.

[0076] It can be understood that the area monitoring module can accurately identify edge computing nodes in a state of tight computing power and a state of idle computing power, and dynamically adjust the computing power collaboration strategy according to the node quantity ratio. This mechanism ensures the reasonable allocation and efficient utilization of computing power resources within the sub-monitoring area, avoiding waste and bottleneck phenomena of computing power resources. When the node quantity ratio exceeds the second preset quantity ratio, it can intelligently send the data processing tasks of some edge computing nodes in a state of tight computing power to the cloud computing module for collaborative processing. This strategy not only reduces the computing power burden on edge computing nodes but also ensures the reasonable utilization of cloud computing resources, avoiding excessive consumption of cloud resources. Through the computing power collaboration strategy, edge computing nodes in a state of tight computing power can timely transmit data processing tasks and data processing results to nodes in a state of idle computing power, enabling the nodes in a state of idle computing power to quickly take over and complete the tasks. This mechanism significantly improves the efficiency of data processing and ensures the continuity and stability of data processing. According to different computing power requirements and resource conditions, flexibly adjusting the computing power collaboration strategy can quickly adapt to various complex and changeable scenarios, ensuring the stability and reliability of the system. Through efficient computing power collaboration and cloud resource utilization, the operating cost can also be reduced. On the one hand, the computing power collaboration strategy reduces waste of computing power resources and improves resource utilization. On the other hand, the reasonable utilization of cloud resources avoids unnecessary investment and maintenance costs.

[0077] Please refer to Figure 4 as shown, which is a step diagram of the data processing method based on cloud-edge collaboration in an embodiment of the present invention. An embodiment of the present invention also provides a data processing method based on cloud-edge collaboration for a data processing system based on cloud-edge collaboration, including

[0078] Step S1, in response to the acquisition of raw data on the edge side, control the corresponding edge computing unit to determine its data source according to the data identifier and / or communication protocol type of the raw data, so as to perform corresponding data analysis and processing;

[0079] Step S2, the computing power monitoring unit determines the computing power representation state of the corresponding edge computing node according to multi-dimensional resources of the corresponding edge computing node;

[0080] Step S3, determine whether the corresponding edge computing node needs to formulate a computing power collaboration strategy according to the computing power representation states of the edge computing nodes in each corresponding sub-monitoring unit;

[0081] Step S4, determine the computing power collaboration strategy according to the node quantity ratio of the edge computing nodes that formulate the computing power collaboration strategy to the edge computing nodes that do not formulate the computing power collaboration strategy, including,

[0082] According to the determination result that the node quantity ratio is greater than the first preset quantity ratio, control the edge computing nodes in the computing power tension state to transmit data processing tasks and data processing results to the edge computing nodes in the computing power idle state, so that the edge computing nodes in the computing power idle state execute the data processing tasks and transmit all the data processing results to the cloud computing module;

[0083] According to the determination result that the node quantity ratio is greater than the second preset quantity ratio, control the edge computing nodes in the computing power tension state to transmit data processing tasks and data collaborative processing requests to the cloud computing module.

[0084] So far, the technical solution of the present invention has been described in combination with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or replacements to the relevant technical features, and the technical solutions after these changes or replacements will all fall within the protection scope of the present invention.

[0085] The above are only the preferred embodiments of the present invention and are not used to limit the present invention; for those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A data processing system based on cloud-edge collaboration, characterized in that, Including: An edge computing module, including a number of edge computing nodes. A single edge computing node includes an edge computing unit for processing corresponding edge-side raw data, an edge transmission unit for sending the data processing result to the cloud computing module, and a computing power monitoring unit for determining the computing power representation state of the edge computing node; A regional monitoring module, including a number of regional cooperation nodes. Each single regional cooperation node is respectively connected to a number of edge computing nodes, and is used to obtain the computing power representation states of the connected edge computing nodes to determine a computing power cooperation strategy; A cloud computing module, which is connected to the edge transmission units of each edge computing node, and is used to obtain the data processing results and data cooperation processing requests of each edge computing node, and respond to the data cooperation processing requests.

2. The data processing system based on cloud-edge collaboration according to claim 1, wherein, In response to the collection of corresponding edge-side raw data, the edge computing unit controls the edge computing unit to determine its data source according to the data identifier and / or communication protocol type of the raw data, so as to perform corresponding data analysis and processing.

3. The data processing system based on cloud-edge collaboration according to claim 1, wherein The computing power monitoring unit determines the computing power representation state of the node according to the multi-dimensional resources of the corresponding edge computing node; Among them, the multi-dimensional resources include CPU utilization rate, memory occupancy rate and GPU utilization rate, and the computing power representation state includes a computing power tension state and a computing power idle state.

4. The data processing system based on cloud-edge collaboration according to claim 1, wherein, The regional monitoring module divides the edge coverage area into several sub-monitoring areas with uniform node density according to the geographical locations of the edge computing nodes through a clustering algorithm, and determines whether the division of the sub-monitoring area is reasonable according to the historical computing power representation states of the edge computing nodes in the single sub-monitoring area within a preset time.

5. The data processing system based on cloud-edge collaboration according to claim 4, characterized in that Based on the determination result that the proportion of the number of sub-monitoring areas with unreasonable division is greater than the preset proportion, the regional monitoring module controls the re-division of the sub-monitoring areas so that the number of re-divided sub-monitoring areas is greater than the number of sub-monitoring areas before re-division.

6. The data processing system based on cloud-edge collaboration according to claim 4, wherein The regional monitoring module determines whether the division of the sub-monitoring area is reasonable according to the historical computing power representation states of the edge computing nodes in the single sub-monitoring area within a preset time, including The regional monitoring module determines that the division of the sub-monitoring area is reasonable according to the determination result that the historical computing power representation state meets the regional division condition; Among them, the regional division condition is that there are a preset number of edge computing nodes in the sub-monitoring area, and the duration ratio of the computing power tension state of the edge computing node is less than or equal to the reference ratio, and the preset time is at least one week.

7. The data processing system based on cloud-edge collaboration according to claim 1, wherein The regional monitoring module sets at least one regional cooperation node in any one sub-monitoring area. The regional cooperation node is respectively connected to the computing power monitoring units of the edge computing nodes in the sub-monitoring area, and is used to obtain the computing power representation states of the corresponding edge computing nodes, and determine whether the corresponding edge computing nodes need to formulate a computing power cooperation strategy according to the computing power representation state.

8. The data processing system based on cloud-edge collaboration according to claim 1, wherein The regional monitoring module determines the computing power cooperation strategy according to the ratio of the number of edge computing nodes that formulate the computing power cooperation strategy to the number of edge computing nodes that do not formulate the computing power cooperation strategy, including According to the determination result that the node quantity ratio is greater than the first preset quantity ratio, control the edge computing nodes in the computing power tension state to transmit data processing tasks and data processing results to the edge computing nodes in the computing power idle state, so that the edge computing nodes in the computing power idle state execute the data processing tasks and transmit all the data processing results to the cloud computing module.

9. The data processing system based on cloud-edge collaboration according to claim 1, wherein, The area monitoring module determines the computing power cooperation strategy according to the node quantity ratio between the edge computing nodes that formulate the computing power cooperation strategy and the edge computing nodes that do not formulate the computing power cooperation strategy, including According to the determination result that the node quantity ratio is greater than the second preset quantity ratio, control the edge computing nodes in the computing power tension state to transmit data processing tasks and data cooperation processing requests to the cloud computing module; wherein, the second preset quantity ratio is greater than the first preset quantity ratio.

10. A data processing method based on cloud-edge collaboration using the data processing system based on cloud-edge collaboration according to any one of claims 1-9, characterized in that, including In response to the acquisition of the original data on the edge side, control the corresponding edge computing unit to determine its data source according to the data identifier and / or communication protocol type of the original data, so as to perform corresponding data analysis and processing; The computing power monitoring unit determines the computing power representation state of the corresponding edge computing node according to the multi-dimensional resources of the corresponding edge computing node; According to the computing power representation states of the corresponding edge computing nodes in a single sub-monitoring unit, determine whether the corresponding edge computing nodes need to formulate a computing power cooperation strategy; Determine the computing power cooperation strategy according to the node quantity ratio between the edge computing nodes that formulate the computing power cooperation strategy and the edge computing nodes that do not formulate the computing power cooperation strategy, including According to the determination result that the node quantity ratio is greater than the first preset quantity ratio, control the edge computing nodes in the computing power tension state to transmit data processing tasks and data processing results to the edge computing nodes in the computing power idle state, so that the edge computing nodes in the computing power idle state execute the data processing tasks and transmit all the data processing results to the cloud computing module; According to the determination result that the node quantity ratio is greater than the second preset quantity ratio, control the edge computing nodes in the computing power tension state to transmit data processing tasks and data cooperation processing requests to the cloud computing module.

Images