Distributed application platform and system based on computing power network

Abstract

The invention provides a distributed application platform and system based on a computing power network, and belongs to the technical field of resource allocation, and the platform comprises an obtaining module which obtains a data task processing log of each sub-computer of a distributed network based on a first sub-resource, and receives all data processing tasks uploaded by a user side; the control module is used for performing state control on all the sub-computers in the distributed application platform; the construction module is used for determining a second resource of the distributed application platform and constructing the computing power network based on the second resource of the distributed application platform; and the distribution module is used for constructing a communication channel between the user side and the distributed application platform and distributing each data processing task based on the communication channel. The computing power consumption can be reduced, the resource utilization rate and the task allocation execution efficiency can be improved, more efficient distributed system operation and resource management are achieved, the computing power scheduling flexibility of a distributed platform and the overall performance of the system are improved, decision support is provided, and the customer satisfaction and the market competitiveness are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of resource allocation, and in particular to a distributed application platform and system based on a computing power network. Background Art

[0002] With the large-scale implementation of emerging information technology business applications such as cloud computing and big data, new businesses have an increasing demand for computing power. Flexibility, easy scalability and ease of use have become the basic capabilities that future computing networks must possess. my country's computing power infrastructure continues to develop and improve, but the level of computing power is still difficult to meet the huge computing power demand brought about by the surge in data volume. In addition, the development of large AI models requires strong computing power support, and AI technology needs computing power support to provide practical and usable value for enterprise production. Therefore, neither data centers nor artificial intelligence computing centers can meet the demand for computing power in the whole society. It will be a general trend to achieve network interconnection, resource sharing, and collaborative scheduling through computing power networks. Among them, distributed computing technology has become one of the research hotspots in the current field because it can effectively obtain high-performance computing capabilities, is flexible and easy to expand, and has huge development potential. It distributes large-scale data and computing tasks to multiple computing nodes for parallel processing. It has the ability to distribute computing power networks, and at the same time needs to be presented on application platforms and systems to achieve effective utilization of computing power networks.

[0003] Therefore, the present invention provides a distributed application platform and system based on computing power network. Summary of the invention

[0004] The present invention provides a distributed application platform and system based on a computing power network. According to the data task processing log of each sub-computer in the distributed network based on the first sub-resource, the status of all sub-computers in the distributed application platform is controlled, the second resource of the distributed application platform is determined and the computing power network is constructed, and a communication channel between the user end and the distributed application platform is constructed. Each data processing task is allocated according to the communication channel, which can reduce computing power consumption, improve resource utilization and task allocation execution efficiency, ensure accurate control and task distribution of sub-computers, realize more efficient distributed system operation and resource management, realize process optimization, improve the computing power scheduling flexibility of the distributed platform and the overall system performance, provide decision support, and improve customer satisfaction and market competitiveness.

[0005] On the one hand, the present invention provides a distributed application platform based on a computing power network, comprising: Acquisition module: acquires the data task processing log of each sub-computer in the distributed network based on the first sub-resource, and receives all data processing tasks uploaded by the user end; Control module: performs status control on all sub-computers in the distributed application platform according to the data task processing log of each sub-computer in the distributed network based on the first sub-resource; Construction module: determining the second resource of the distributed application platform based on the data task processing logs of all first sub-resources, and constructing a computing power network based on the second resource of the distributed application platform; Allocation module: builds a communication channel between the user end and the distributed application platform, and allocates each data processing task based on the communication channel.

[0006] According to a distributed application platform based on a computing power network provided by the present invention, the control module includes: A first computer set unit: determining a first computer set of the distributed network based on a first sub-resource of each sub-computer of the distributed network, wherein the first computer set includes a plurality of sub-computer sets, and the first sub-resource includes a plurality of sub-resources of the corresponding sub-computers; A state control strategy unit: determining a state control strategy for each sub-computer set based on first sub-resources of all sub-computers included in each sub-computer set in the first computer set; State control unit: performs state control on each sub-computer based on the data task processing log of each sub-computer in the distributed network and the state control strategy of the corresponding sub-computer set.

[0007] According to a distributed application platform based on a computing power network provided by the present invention, the building module includes: First resource total amount unit: obtains the first resource total amount of each sub-resource in the distributed network; Usage unit: each sub-computer in the distributed network determines the usage of each sub-resource based on the data task processing log of the first sub-resource by each sub-computer; Resource requirement unit: analyzes each data processing task uploaded by the user end and determines the resource requirement of each data processing task, wherein the resource requirement includes multiple sub-resource requirements; A first computing unit: determining a second total resource amount of each sub-resource in the distributed application platform based on a first total resource amount of all sub-resources, a usage amount of each sub-resource by each sub-computer, and a resource requirement of each data processing task; in, Represents the total amount of the first resource of the i-th sub-resource in the distributed network, represents the usage of the i-th sub-resource by the j-th sub-computer in the distributed network, N1 represents the number of sub-computers in the distributed network, represents the total usage of the ith sub-resource in the distributed network, represents the remaining amount of the ith sub-resource in the distributed network, represents the sub-resource demand of the k-th data processing task for the i-th sub-resource, N2 represents the number of data processing tasks received by the user end, represents the total demand for the ith sub-resource of all data processing tasks, represents the amount of sub-resource introduced in the distributed network. represents the total amount of the second resource of the i-th sub-resource in the distributed network; The second resource unit determines the second resource of the distributed application platform based on the total amount of the second resource of each sub-resource in the distributed application platform.

[0008] According to a distributed application platform based on a computing power network provided by the present invention, the building module further includes: A first classification unit: classifies the second resource of the distributed application platform to determine a first category, wherein the first category includes a plurality of first subcategories; Node resource unit: obtains node resources of each node in the distributed application platform, wherein the node resources include multiple sub-node resources; Analysis unit: analyzing the resource ratio of node resources of the node, determining the corresponding node category, extracting all nodes with the same node category to determine the first child node, and determining the first node based on all the first child nodes; A second calculation unit: based on all the child nodes under all the first child nodes in the first node, calculates the child node dependency value of each first child node in the first node, and calculates the inter-node dependency value between every two first child nodes in the first node; A third calculation unit: calculating a comprehensive dependency value of the first node based on the sub-node dependency values ​​of all first sub-nodes in the first node and the inter-node dependency value between every two first sub-nodes in the first node; Sub-computing power network unit: constructing a sub-computing power network based on the sub-node dependency value of each first sub-node in the first node; Computing power network unit: constructs a computing power network based on all sub-computing power networks, the node dependency value between every two first sub-nodes in the first node, and the comprehensive dependency value of the first node.

[0009] According to a distributed application platform based on a computing power network provided by the present invention, the second computing unit includes: in, represents the child node dependency value of the ath first child node in the first node, aN3 represents the number of child nodes under the ath first child node in the first node, They respectively represent the resource dependency value, communication dependency value, task dependency value, and data flow dependency value of the b-th child node and the c-th child node under the a-th first child node in the first node, represents the node dependency value between the ath first child node and the dth first child node in the first node, dN3 represents the number of child nodes under the dth first child node in the first node, Indicates the resource dependency value, communication dependency value, task dependency value, and data flow dependency value between the b-th child node under the a-th first child node and the e-th child node under the d-th first child node in the first node; The third computing unit comprises: in, represents the comprehensive dependency value of the first node, N4 represents the number of first child nodes in the first node, Represents the weight of the ath first child node in the first node.

[0010] According to a distributed application platform based on a computing power network provided by the present invention, the allocation module includes: Real-time computing resource status unit: collects the real-time computing resource status of each sub-computer of the distributed application platform; First sorting result unit: sorting all sub-computers from small to large based on the corresponding real-time computing resource status, and determining a first sorting result; Second sorting result unit: sorting all data processing tasks from large to small based on corresponding resource requirements, and determining a second sorting result; A first computing power judgment unit: selects a data processing task corresponding to a first resource requirement in the second sorting result, and performs computing power judgment on the first real-time computing power resource state in the first sorting result; If the computing power judgment result is that the first real-time computing power resource state in the first sorting result can meet the first resource requirement in the second sorting result, the data processing task corresponding to the first resource requirement in the second sorting result is allocated to the sub-computer corresponding to the first real-time computing power resource state in the first sorting result; If the computing power judgment result is that the first real-time computing power resource state in the first sorting result cannot meet the first resource requirement in the second sorting result, then the computing power judgment is performed on the first resource requirement in the second sorting result and the second real-time computing power resource state in the first sorting result, and so on; If the last real-time computing resource state in the first sorting result also cannot meet the first resource requirement in the second sorting result, the corresponding data processing task is allocated to multiple sub-computers based on the first resource requirement in the second sorting result; Adjustment unit: marking the allocated data processing tasks, re-determining the real-time computing resource status of each sub-computer after allocation, and adjusting the order of each real-time computing resource status after reallocation in the first sorting result; The second computing power judgment unit is used to successively judge the computing power of the data processing tasks corresponding to the resource requirements in the second sorting result and the first sorting result after the order is adjusted.

[0011] On the other hand, the present invention also provides a distributed application system based on a computing power network, comprising: The above-mentioned application platforms; as well as the management center and dispatch center.

[0012] According to a distributed application system based on a computing power network provided by the present invention, the management center includes multiple management nodes; Management node unit: The management node is used to perform at least one of resource management, application management, performance management, event management, and operation and maintenance management on all resources and all application objects of the computing power network.

[0013] According to a distributed application system based on a computing power network provided by the present invention, the management node unit includes: Resource management subunit: The management node manages all resources of the computing network based on the cloud management interface; Application management subunit: The management node manages the relevant configuration information of all application objects in the computing network, displays various configuration information of application objects based on lists, and monitors the performance, events and status of application objects to achieve application management; Performance management subunit: The management node collects performance indicator data of all resources and all application objects of the computing network based on the performance collection interface, and triggers alarm notifications for performance indicators that exceed the limit according to the threshold configuration to achieve performance management; Event management subunit: The management node collects the fault data and log data of all resources and application objects of the computing network in a unified manner, and performs standardized processing on the abnormal events generated according to the rules to achieve event management; Operation and maintenance management subunit: The managed node inspects all resources and application objects of the computing network according to the preset inspection plan and generates reports. It sends abnormal status or fault alarms during the inspection process to event monitoring for processing to achieve operation and maintenance management.

[0014] According to a distributed application system based on a computing power network provided by the present invention, the dispatch center includes a dispatch hub; Dispatching center unit: The dispatching center is used to process data on the resources or application objects controlled by the management nodes according to the business requirements of the application collaboration sub-platform to obtain application services; Scheduling center unit: The scheduling center is used to connect to the interface of the managed node, collect resources, and provide unified access to various data sources; it is used to connect to the application and task status interface of the managed node, provide management and monitoring of access tasks, and analyze and optimize the perceived data; it is also used to manage the artificial intelligence models and algorithms in the orchestration and scheduling, as well as the optimization strategies generated by auxiliary strategies.

[0015] Compared with the prior art, the present invention has the following beneficial effects: According to the data task processing log of each sub-computer in the distributed network based on the first sub-resource, the status of all sub-computers in the distributed application platform is controlled, the second resource of the distributed application platform is determined and the computing power network is constructed, and a communication channel between the user end and the distributed application platform is constructed. Each data processing task is allocated according to the communication channel, which can reduce computing power consumption, improve resource utilization and task allocation efficiency, ensure accurate control and task distribution of sub-computers, achieve more efficient distributed system operation and resource management, realize process optimization, improve decision support, and improve customer satisfaction and market competitiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a structural diagram of a distributed application platform based on a computing power network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Embodiment 1: The embodiment of the present invention provides a distributed application platform based on a computing power network, such as Figure 1 As shown, including: Acquisition module: acquires the data task processing log of each sub-computer in the distributed network based on the first sub-resource, and receives all data processing tasks uploaded by the user end; Control module: performs status control on all sub-computers in the distributed application platform according to the data task processing log of each sub-computer in the distributed network based on the first sub-resource; Construction module: determining the second resource of the distributed application platform based on the data task processing logs of all first sub-resources, and constructing a computing power network based on the second resource of the distributed application platform; Allocation module: builds a communication channel between the user end and the distributed application platform, and allocates each data processing task based on the communication channel.

[0020] In this embodiment, the acquisition module acquires data task processing logs from each sub-computer in the distributed network and receives tasks uploaded by users. The acquisition module collects real-time data in the distributed network for monitoring the sub-computers.

[0021] In this embodiment, the data task processing log is a task execution record of the corresponding sub-computer, recording detailed information of the task execution, such as the task start time, completion time, resources used, etc.

[0022] The beneficial effects of the above technical solution are as follows: according to the data task processing log of each sub-computer in the distributed network based on the first sub-resource, the status of all sub-computers in the distributed application platform is controlled, the second resource of the distributed application platform is determined and a computing power network is constructed, a communication channel between the user end and the distributed application platform is constructed, and each data processing task is allocated according to the communication channel, which can reduce computing power consumption, improve resource utilization and task allocation efficiency, ensure accurate control and task distribution of sub-computers, achieve more efficient distributed system operation and resource management, realize process optimization, improve decision support, and improve customer satisfaction and market competitiveness.

[0023] Embodiment 2: An embodiment of the present invention provides a distributed application platform based on a computing power network, wherein the control module comprises: A first computer set unit: determining a first computer set of the distributed network based on a first sub-resource of each sub-computer of the distributed network, wherein the first computer set includes a plurality of sub-computer sets, and the first sub-resource includes a plurality of sub-resources of the corresponding sub-computers; A state control strategy unit: determining a state control strategy for each sub-computer set based on first sub-resources of all sub-computers included in each sub-computer set in the first computer set; State control unit: performs state control on each sub-computer based on the data task processing log of each sub-computer in the distributed network and the state control strategy of the corresponding sub-computer set.

[0024] In this embodiment, the sub-computer represents a single computing node in a distributed network, which is an independent physical or virtual computing device. Each sub-computer has its corresponding first sub-resource, and the sub-resource may be a CPU, memory, storage, etc.

[0025] In this embodiment, the first sub-resource refers to the resources of the corresponding sub-computer in the distributed network, including basic computing capabilities such as processor, memory, storage, network bandwidth, etc. These resources are used to execute computing tasks of the platform.

[0026] In this embodiment, the state control policy unit is a module responsible for formulating policies for each sub-computer collection. These policies determine how the sub-computer collection should allocate resources, adjust loads, or enter different working states (such as high-performance mode, low power consumption mode, etc.) during task processing.

[0027] In this embodiment, the state control unit is an execution-level module, which performs real-time resource allocation and state adjustment on each sub-computer according to the state control strategy.

[0028] In this embodiment, the state control policy is a set of rules for guiding the behavior and resource usage of all sub-computers in the sub-computer set. The policy may include load balancing rules, performance optimization rules, or energy-saving policies, etc., to ensure that each sub-computer runs in the most appropriate state.

[0029] In this embodiment, based on the data task processing log of each sub-computer in the distributed network and the state control strategy of the corresponding sub-computer set, the state of each sub-computer is controlled, including: adjusting the working state of the first sub-resource of each sub-computer in the corresponding gateway connected to the distributed network; resource load balancing control, reallocating tasks to sub-computers with lighter loads; when some sub-computers fail or are in abnormal states, the system can automatically execute fault recovery measures, such as automatic restart, switching to a backup node, or enabling redundant resources; dynamically adjusting the computing resources, memory, or storage allocation available to the sub-computers according to their current task volume and resource usage to improve overall efficiency, etc.

[0030] The beneficial effects of the above technical solution are: according to the data task processing log of each sub-computer in the distributed network based on the first sub-resource, the status of all sub-computers in the distributed application platform is controlled, the resource usage and task logs can be analyzed in real time, the overall resource scheduling and task allocation of the system are optimized, and the computing efficiency and resource utilization are improved.

[0031] Embodiment 3: An embodiment of the present invention provides a distributed application platform based on a computing power network, wherein the building module includes: First resource total amount unit: obtains the first resource total amount of each sub-resource in the distributed network; Usage unit: each sub-computer in the distributed network determines the usage of each sub-resource based on the data task processing log of the first sub-resource by each sub-computer; Resource requirement unit: analyzes each data processing task uploaded by the user end and determines the resource requirement of each data processing task, wherein the resource requirement includes multiple sub-resource requirements; A first computing unit: determining a second total resource amount of each sub-resource in the distributed application platform based on a first total resource amount of all sub-resources, a usage amount of each sub-resource by each sub-computer, and a resource requirement of each data processing task; in, Represents the total amount of the first resource of the i-th sub-resource in the distributed network, represents the usage of the i-th sub-resource by the j-th sub-computer in the distributed network, N1 represents the number of sub-computers in the distributed network, represents the total usage of the ith sub-resource in the distributed network, represents the remaining amount of the ith sub-resource in the distributed network, represents the sub-resource demand of the k-th data processing task for the i-th sub-resource, N2 represents the number of data processing tasks received by the user end, represents the total demand for the ith sub-resource of all data processing tasks, represents the amount of sub-resource introduced in the distributed network. represents the total amount of the second resource of the i-th sub-resource in the distributed network; The second resource unit determines the second resource of the distributed application platform based on the total amount of the second resource of each sub-resource in the distributed application platform.

[0032] In this embodiment, the first resource total amount unit is responsible for obtaining the total amount of resources of each sub-computer in the entire distributed network. Specifically, it collects the resources of each sub-computer (such as processor, memory, bandwidth, etc.) and summarizes them into a "first resource total amount" available to a platform, that is, the sum of each resource in the distributed network.

[0033] In this embodiment, the usage unit determines the actual usage of resources by each sub-computer based on the task processing logs of each sub-computer. It analyzes the resources currently used by each sub-computer (such as CPU occupancy, memory usage, etc.) and records them.

[0034] In this embodiment, when a user submits a task, the resource requirement unit analyzes the specific requirements of each task. Each task requires different computing resources (such as different numbers of CPU cores, memory, storage space, etc.). This unit determines these requirements and determines the sub-resource requirements of each sub-resource.

[0035] In this embodiment, the first calculation unit calculates the "second resource total amount" of each resource in the distributed application platform by combining the first resource total amount of each sub-resource, the usage of each sub-computer, and the resource demand of each task.

[0036] In this embodiment, the second total amount of resources refers to the total amount of resources that can be reasonably allocated and scheduled by the platform after taking into account the usage of all sub-computers.

[0037] In this embodiment, the second resource unit determines the second resource of the entire distributed application platform based on the total amount of the second resource of each sub-resource.

[0038] In this embodiment, the second resource is the actual amount of resources that the platform can use to process user tasks in the current state, and after optimization and integration, it can be flexibly allocated to different tasks.

[0039] The beneficial effects of the above technical solution are: the second resource of the distributed application platform is determined based on the data task processing log of all first sub-resources, and the task processing log and resource demand analysis can be used to accurately match resource allocation, thereby improving resource management efficiency and task processing performance, and ensuring maximum utilization of distributed platform resources.

[0040] Embodiment 4: The embodiment of the present invention provides a distributed application platform based on a computing power network, wherein the building module further includes: A first classification unit: classifies the second resource of the distributed application platform to determine a first category, wherein the first category includes a plurality of first subcategories; Node resource unit: obtains node resources of each node in the distributed application platform, wherein the node resources include multiple sub-node resources; Analysis unit: analyzing the resource ratio of node resources of the node, determining the corresponding node category, extracting all nodes with the same node category to determine the first child node, and determining the first node based on all the first child nodes; A second calculation unit: based on all the child nodes under all the first child nodes in the first node, calculates the child node dependency value of each first child node in the first node, and calculates the inter-node dependency value between every two first child nodes in the first node; A third calculation unit: calculating a comprehensive dependency value of the first node based on the sub-node dependency values ​​of all first sub-nodes in the first node and the inter-node dependency value between every two first sub-nodes in the first node; Sub-computing power network unit: constructing a sub-computing power network based on the sub-node dependency value of each first sub-node in the first node; Computing power network unit: constructs a computing power network based on all sub-computing power networks, the node dependency value between every two first sub-nodes in the first node, and the comprehensive dependency value of the first node.

[0041] In this embodiment, the first category unit classifies the resources of the distributed application platform to determine the first category. The first category is a larger resource category, which can be subdivided into multiple first subcategories. The subcategories can be computing categories, storage categories, network categories, etc.

[0042] In this embodiment, the node categories of the nodes may be computing nodes, storage nodes, network nodes, etc.

[0043] In this embodiment, the number of first sub-nodes in the first node is the same as the number of first sub-categories in the first category.

[0044] In this embodiment, the node resources of each node in the distributed application platform are extracted, including CPU, memory, storage, bandwidth, etc. The main function of each node is judged according to the ratio of multiple sub-resources under each node, and the main function is determined as the node category. For example, if the computing resources (such as CPU, GPU) account for a high proportion, it is a computing node; if the storage resources (such as hard disk, SSD) account for a high proportion, it is a storage node.

[0045] In this embodiment, the node category is determined: based on the resource allocation analysis, a corresponding category (eg, computing node, storage node, network node, etc.) is assigned to each node.

[0046] In this embodiment, all nodes of the same node category are aggregated, for example, all nodes identified as computing nodes are aggregated into a group called first sub-nodes, and the first sub-nodes of each category are determined to ensure that the first sub-nodes in each category have similar functions and resource allocations.

[0047] In this embodiment, the sub-computing power network unit constructs a sub-computing power network according to the sub-node dependency value of each first sub-node in the first node. The sub-computing power network includes all sub-nodes in the same first sub-node, and each first sub-node corresponds to a sub-computing power network.

[0048] In this embodiment, the computing power network of the entire platform is constructed based on all sub-computing power networks, the node dependency value between every two first sub-nodes in the first node, and the comprehensive dependency value of the first node. The computing power network is a comprehensive computing network that covers all sub-nodes and their interdependencies, and is used to optimize computing task scheduling, resource allocation, and system performance in distributed systems.

[0049] The beneficial effects of the above technical solution are as follows: Building a computing power network based on the second resource of the distributed application platform can optimize the structure and performance of the computing power network, further improve resource coordination and task processing efficiency, and make the distributed platform more scalable and flexible.

[0050] Embodiment 5: An embodiment of the present invention provides a distributed application platform based on a computing power network, wherein the second computing unit includes: in, represents the child node dependency value of the ath first child node in the first node, aN3 represents the number of child nodes under the ath first child node in the first node, They respectively represent the resource dependency value, communication dependency value, task dependency value, and data flow dependency value of the b-th child node and the c-th child node under the a-th first child node in the first node, represents the node dependency value between the ath first child node and the dth first child node in the first node, dN3 represents the number of child nodes under the dth first child node in the first node, Indicates the resource dependency value, communication dependency value, task dependency value, and data flow dependency value between the b-th child node under the a-th first child node and the e-th child node under the d-th first child node in the first node; The third computing unit comprises: in, represents the comprehensive dependency value of the first node, N4 represents the number of first child nodes in the first node, Represents the weight of the ath first child node in the first node.

[0051] In this embodiment, the subnode dependency value is a value calculated based on the resource dependency value, communication dependency value, task dependency value, and data flow dependency value of every two subnodes included in the corresponding first subnode.

[0052] In this embodiment, the inter-node dependency value is a value calculated based on the resource dependency value, communication dependency value, task dependency value and data flow dependency value between every two sub-nodes under every two first sub-nodes contained in the first node.

[0053] In this embodiment, the comprehensive dependency value is a value calculated based on the sub-node dependency values ​​of all first sub-nodes and the inter-node dependency value between every two first sub-nodes.

[0054] In this embodiment, the resource dependency values ​​of the bth child node and the cth child node under the ath first child node in the first node are , represents the degree of dependence between the b-th child node and the c-th child node in terms of shared resources. Specifically, the two child nodes may share the same processor, memory pool or storage space. This value reflects how they compete or collaborate in using resources in the same resource pool.

[0055] In this embodiment, the communication dependency values ​​of the b-th child node and the c-th child node under the a-th first child node in the first node are , represents the data communication demand and frequency between the b-th child node and the c-th child node. For example, two child nodes may need to frequently transmit data or exchange messages to complete a task. This dependency value measures the intensity of communication between them.

[0056] In this embodiment, the b-th child node and the c-th child node under the a-th first child node in the first node , represents the dependency between the b-th child node and the c-th child node in task processing. For example, one child node may be responsible for preprocessing data, while the other child node depends on the preprocessing result to continue its task. This dependency value reflects their collaborative relationship in task execution.

[0057] In this embodiment, the b-th child node and the c-th child node under the a-th first child node in the first node , represents the data flow and dependency between the b-th child node and the c-th child node. For example, the intermediate result of a task may flow between these two nodes, or they need to obtain data from a shared data source. This value reflects the intensity and importance of the data interaction between the two.

[0058] In this embodiment, the resource dependency value between the bth child node under the ath first child node and the eth child node under the dth first child node in the first node is , represents the degree of cross-subnode resource sharing or dependence between the b-th child node and the e-th child node. Since they belong to different first child nodes, the resource dependence may be more complex. For example, they need to exchange data through shared storage or network resources. This value reflects the collaboration needs between different resource pools.

[0059] In this embodiment, the communication dependency value between the bth child node under the ath first child node and the eth child node under the dth first child node in the first node is , represents the cross-node communication requirement between the b-th child node and the e-th child node. Since they may not be in the same physical area or the same network subsystem, this value measures their cross-subnode communication requirements when completing tasks, such as frequent exchange of data packets or dependency of remote calls.

[0060] In this embodiment, the task dependency value between the bth child node under the ath first child node and the eth child node under the dth first child node in the first node is , represents the cross-node dependency between the b-th child node and the e-th child node in task execution. Different first child nodes may undertake different types of tasks, and one node may need to wait for the calculation result of another node. This value reflects the workflow and collaboration relationship between them in cross-node tasks.

[0061] In this embodiment, the data flow dependency value between the bth child node under the ath first child node and the eth child node under the dth first child node in the first node is , represents the cross-node data transmission demand between the b-th child node and the e-th child node. For example, during task processing, it may be necessary to transmit data from the b-th child node to the e-th child node through the network. The data flow dependency value reflects the amount of data, flow frequency, and importance in this process.

[0062] The beneficial effects of the above technical solution are as follows: by calculating the sub-node dependency value of each first sub-node in the first node, calculating the inter-node dependency value between every two first sub-nodes in the first node, and calculating the comprehensive dependency value of the first node, the dependency relationship between nodes can be accurately calculated, thereby improving resource coordination and task processing efficiency.

[0063] Embodiment 6: An embodiment of the present invention provides a distributed application platform based on a computing power network, wherein the allocation module includes: Real-time computing resource status unit: collects the real-time computing resource status of each sub-computer of the distributed application platform; First sorting result unit: sorting all sub-computers from small to large based on the corresponding real-time computing resource status, and determining a first sorting result; Second sorting result unit: sorting all data processing tasks from large to small based on corresponding resource requirements, and determining a second sorting result; A first computing power judgment unit: selects a data processing task corresponding to a first resource requirement in the second sorting result, and performs computing power judgment on the first real-time computing power resource state in the first sorting result; If the computing power judgment result is that the first real-time computing power resource state in the first sorting result can meet the first resource requirement in the second sorting result, the data processing task corresponding to the first resource requirement in the second sorting result is allocated to the sub-computer corresponding to the first real-time computing power resource state in the first sorting result; If the computing power judgment result is that the first real-time computing power resource state in the first sorting result cannot meet the first resource requirement in the second sorting result, then the computing power judgment is performed on the first resource requirement in the second sorting result and the second real-time computing power resource state in the first sorting result, and so on; If the last real-time computing resource state in the first sorting result also cannot meet the first resource requirement in the second sorting result, the corresponding data processing task is allocated to multiple sub-computers based on the first resource requirement in the second sorting result; Adjustment unit: marking the allocated data processing tasks, re-determining the real-time computing resource status of each sub-computer after allocation, and adjusting the order of each real-time computing resource status after reallocation in the first sorting result; The second computing power judgment unit is used to successively judge the computing power of the data processing tasks corresponding to the resource requirements in the second sorting result and the first sorting result after the order is adjusted.

[0064] In this embodiment, the real-time computing power resource status unit is responsible for collecting the computing power and resource usage of each sub-computer in real time. The resource status includes processor load, memory usage, network bandwidth, disk I / O, etc. Through real-time monitoring, the system can know whether the resources of each sub-computer are idle or overloaded, so as to allocate tasks according to the status of the node.

[0065] In this embodiment, after collecting the computing power resource status of each node in real time, the first sorting result unit will sort the nodes from small to large according to the computing power resources, and the nodes with fewer resources will be ranked in front. The purpose of this is to give priority to the nodes with fewer resources, maximize the utilization of the overall resources, and avoid the situation where some nodes are overloaded while the resources of other nodes are idle.

[0066] In this embodiment, the second sorting result unit sorts all tasks waiting to be processed according to their resource requirements. Tasks with greater resource requirements will be ranked first. The resource requirements of a task are usually determined by the complexity of the task and the amount of data. By giving priority to tasks with greater requirements, it can be ensured that complex tasks can be processed when resources are sufficient, and tasks with smaller resources can be flexibly arranged to be processed when resources are insufficient.

[0067] In this embodiment, the first computing power judgment unit is responsible for starting from the task with the largest resource requirement and comparing it with the resources of each sub-computer in turn. Each task has corresponding resource requirements. The unit determines whether a sub-computer has sufficient computing power to process the task. If the resources of a node are sufficient to meet the task, the task will be assigned to the node; otherwise, the comparison continues.

[0068] In this embodiment, if the resource status of a node can meet the requirements of the task, the task is assigned to the node. If no single resource of any node can meet the requirements of the task, the task will be assigned to multiple sub-computers for joint processing. This situation is suitable for particularly complex or large tasks that require collaborative computing in a distributed manner.

[0069] In this embodiment, after the task is assigned, the resource status of the node will change, and the adjustment unit will recalculate the real-time resource status of the node and update its position in the first sorting result. This can ensure that during the task processing, there is always a dynamically updated resource list that reflects the current status of the node, which is convenient for subsequent task allocation.

[0070] In this embodiment, the second computing power judgment unit continues to allocate the remaining tasks based on the node resource status after adjustment and sorting. The unit judges each task and node resource in turn to ensure that the resources are fully utilized. This process continues until all tasks are reasonably allocated to suitable nodes.

[0071] The beneficial effects of the above technical solution are: building a communication channel between the user end and the distributed application platform, and allocating each data processing task according to the communication channel. This can dynamically adjust the sorting and allocation strategy of computing resources, optimize resource utilization, ensure efficient processing of large tasks, and significantly improve the computing scheduling flexibility and processing efficiency of the distributed platform.

[0072] Embodiment 7: An embodiment of the present invention provides a distributed application system based on a computing power network, including: An application platform as described in any of Examples 1-6; and a management center and a dispatching center.

[0073] In this embodiment, the management center is the core component responsible for managing resources, applications, and performance in the entire computing network. The management center is composed of multiple management nodes, and each management node is responsible for managing a certain part of the resources or application objects.

[0074] In this embodiment, the dispatch center is the core unit responsible for task scheduling and resource allocation. With the support of the management center, it performs dynamic allocation of tasks based on the real-time computing resource status and the resource requirements of the tasks.

[0075] The beneficial effects of the above technical solution are as follows: the distributed application system includes an application platform, a management center and a dispatching center, which can ensure the maximum resource utilization and high efficiency of task execution in a complex environment, and improve the overall operating performance and flexibility of the system.

[0076] Embodiment 8: The embodiment of the present invention provides a distributed application system based on a computing power network, wherein a management center includes a plurality of management nodes; Management node unit: The management node is used to perform at least one of resource management, application management, performance management, event management, and operation and maintenance management on all resources and all application objects of the computing power network.

[0077] In this embodiment, the management node is responsible for managing all computing resources in the network. Resource management involves registration of computing nodes, status monitoring, resource allocation, and resource coordination between computing nodes.

[0078] In this embodiment, application management refers to the management of applications running on the computing network, which includes application deployment, updating, monitoring, and runtime status management.

[0079] In this embodiment, performance management is responsible for monitoring the operating status of the entire computing network and applications, evaluating the performance of each node and application, and ensuring that system resources are fully utilized.

[0080] In this embodiment, the event management handles anomalies and events that occur in the system, such as hardware failures, network interruptions, or resource bottlenecks. The event management system can automatically or semi-automatically perform fault repair operations.

[0081] In this embodiment, the operation and maintenance management is responsible for the daily operation and maintenance of the entire computing network, including system health checks, data backup, security protection, patch management, etc.

[0082] The beneficial effects of the above technical solution are as follows: the management center includes multiple management nodes, which can enable management tasks to be executed more flexibly and accurately, thereby improving the efficiency and robustness of resource scheduling, system monitoring and operation and maintenance management.

[0083] Embodiment 9: An embodiment of the present invention provides a distributed application system based on a computing power network, wherein the management node unit includes: Resource management subunit: The management node manages all resources of the computing network based on the cloud management interface; Application management subunit: The management node manages the relevant configuration information of all application objects in the computing network, displays various configuration information of application objects based on lists, and monitors the performance, events and status of application objects to achieve application management; Performance management subunit: The management node collects performance indicator data of all resources and all application objects of the computing network based on the performance collection interface, and triggers alarm notifications for performance indicators that exceed the limit according to the threshold configuration to achieve performance management; Event management subunit: The management node collects the fault data and log data of all resources and application objects of the computing network in a unified manner, and performs standardized processing on the abnormal events generated according to the rules to achieve event management; Operation and maintenance management subunit: The managed node inspects all resources and application objects of the computing network according to the preset inspection plan and generates reports. It sends abnormal status or fault alarms during the inspection process to event monitoring for processing to achieve operation and maintenance management.

[0084] In this embodiment, the resource management sub-unit is responsible for unified management and monitoring of all hardware resources (such as computing power, storage, network, etc.) in the computing power network.

[0085] In this embodiment, the application management subunit is responsible for managing all application programs in the network, including configuration monitoring, performance evaluation, event logging, etc.

[0086] In this embodiment, the performance management subunit collects performance indicators of the system and applications, and triggers alarms according to set thresholds to ensure the high efficiency of system operation.

[0087] In this embodiment, the event management subunit uniformly collects and standardizes the processing of fault events and log information to ensure that system abnormal events are responded to in a timely manner.

[0088] In this embodiment, the operation and maintenance management subunit inspects and reports the system according to a preset plan to ensure the health of the system and promptly notify the relevant management modules to handle problems when they are found.

[0089] The beneficial effects of the above technical solution are: determining the resource management sub-unit, application management sub-unit, performance management sub-unit, event management sub-unit and operation and maintenance management sub-unit, realizing efficient and automated monitoring and alarm processing, and standardizing event management and inspection plans to improve operation and maintenance efficiency and ensure the stability of the system and sustainable optimization capabilities.

[0090] Embodiment 10: The embodiment of the present invention provides a distributed application system based on a computing power network, wherein the dispatching center includes a dispatching hub; Dispatching center unit: The dispatching center is used to process data on the resources or application objects controlled by the management nodes according to the business requirements of the application collaboration sub-platform to obtain application services; Scheduling center unit: The scheduling center is used to connect to the interface of the managed node, collect resources, and provide unified access to various data sources; it is used to connect to the application and task status interface of the managed node, provide management and monitoring of access tasks, and analyze and optimize the perceived data; it is also used to manage the artificial intelligence models and algorithms in the orchestration and scheduling, as well as the optimization strategies generated by auxiliary strategies.

[0091] In this embodiment, the dispatch center unit is mainly responsible for resource scheduling and data processing according to business needs to ensure on-demand delivery of application services.

[0092] In this embodiment, the scheduling hub unit serves as the core of the scheduling center and is responsible for resource acquisition, task monitoring, data analysis and optimization. It improves scheduling efficiency and generates optimal strategies through artificial intelligence models and algorithms.

[0093] The beneficial effects of the above technical solution are as follows: by determining the dispatching center unit and the dispatching hub unit, the flexibility, intelligence and resource utilization efficiency of the dispatching can be improved, and the business demand responsiveness can be enhanced.

[0094] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0095] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed application platform based on computing power network, characterized in that: include: Acquisition module: acquires the data task processing log of each sub-computer in the distributed network based on the first sub-resource, and receives all data processing tasks uploaded by the user end; Control module: performs status control on all sub-computers in the distributed application platform according to the data task processing log of each sub-computer in the distributed network based on the first sub-resource; Construction module: determining the second resource of the distributed application platform based on the data task processing logs of all first sub-resources, and constructing a computing power network based on the second resource of the distributed application platform; Allocation module: builds a communication channel between the user end and the distributed application platform, and allocates each data processing task based on the communication channel.

2. According to claim 1, a distributed application platform based on computing power network is characterized in that: The control module comprises: A first computer set unit: determining a first computer set of the distributed network based on a first sub-resource of each sub-computer of the distributed network, wherein the first computer set includes a plurality of sub-computer sets, and the first sub-resource includes a plurality of sub-resources of the corresponding sub-computers; A state control strategy unit: determining a state control strategy for each sub-computer set based on first sub-resources of all sub-computers included in each sub-computer set in the first computer set; State control unit: performs state control on each sub-computer based on the data task processing log of each sub-computer in the distributed network and the state control strategy of the corresponding sub-computer set.

3. A distributed application platform based on computing power network according to claim 2, characterized in that: The building blocks include: First resource total amount unit: obtains the first resource total amount of each sub-resource in the distributed network; Usage unit: each sub-computer in the distributed network determines the usage of each sub-resource based on the data task processing log of the first sub-resource by each sub-computer; Resource requirement unit: analyzes each data processing task uploaded by the user end and determines the resource requirement of each data processing task, wherein the resource requirement includes multiple sub-resource requirements; A first computing unit: determining a second total resource amount of each sub-resource in the distributed application platform based on a first total resource amount of all sub-resources, a usage amount of each sub-resource by each sub-computer, and a resource requirement of each data processing task; in, Represents the total amount of the first resource of the i-th sub-resource in the distributed network, represents the usage of the i-th sub-resource by the j-th sub-computer in the distributed network, N1 represents the number of sub-computers in the distributed network, represents the total usage of the ith sub-resource in the distributed network, represents the remaining amount of the ith sub-resource in the distributed network, represents the sub-resource demand of the k-th data processing task for the i-th sub-resource, N2 represents the number of data processing tasks received by the user end, represents the total demand for the ith sub-resource of all data processing tasks, represents the amount of sub-resource introduced in the distributed network. represents the total amount of the second resource of the i-th sub-resource in the distributed network; The second resource unit determines the second resource of the distributed application platform based on the total amount of the second resource of each sub-resource in the distributed application platform.

4. The distributed application platform based on computing power network according to claim 1, characterized in that: The building block further includes: A first classification unit: classifies the second resource of the distributed application platform to determine a first category, wherein the first category includes a plurality of first subcategories; Node resource unit: obtains node resources of each node in the distributed application platform, wherein the node resources include multiple sub-node resources; Analysis unit: analyzing the resource ratio of node resources of the node, determining the corresponding node category, extracting all nodes with the same node category to determine the first child node, and determining the first node based on all the first child nodes; A second calculation unit: based on all the child nodes under all the first child nodes in the first node, calculates the child node dependency value of each first child node in the first node, and calculates the inter-node dependency value between every two first child nodes in the first node; A third calculation unit: calculating a comprehensive dependency value of the first node based on the sub-node dependency values ​​of all first sub-nodes in the first node and the inter-node dependency value between every two first sub-nodes in the first node; Sub-computing power network unit: constructing a sub-computing power network based on the sub-node dependency value of each first sub-node in the first node; Computing power network unit: constructs a computing power network based on all sub-computing power networks, the node dependency value between every two first sub-nodes in the first node, and the comprehensive dependency value of the first node.

5. A distributed application platform based on computing power network according to claim 4, characterized in that: The second computing unit comprises: in, represents the child node dependency value of the ath first child node in the first node, aN3 represents the number of child nodes under the ath first child node in the first node, They respectively represent the resource dependency value, communication dependency value, task dependency value, and data flow dependency value of the b-th child node and the c-th child node under the a-th first child node in the first node, represents the node dependency value between the ath first child node and the dth first child node in the first node, dN3 represents the number of child nodes under the dth first child node in the first node, Indicates the resource dependency value, communication dependency value, task dependency value, and data flow dependency value between the b-th child node under the a-th first child node and the e-th child node under the d-th first child node in the first node; The third computing unit comprises: in, represents the comprehensive dependency value of the first node, N4 represents the number of first child nodes in the first node, Represents the weight of the ath first child node in the first node.

6. A distributed application platform based on computing power network according to claim 3, characterized in that: The allocation module comprises: Real-time computing resource status unit: collects the real-time computing resource status of each sub-computer of the distributed application platform; First sorting result unit: sorting all sub-computers from small to large based on the corresponding real-time computing resource status, and determining a first sorting result; Second sorting result unit: sorting all data processing tasks from large to small based on corresponding resource requirements, and determining a second sorting result; A first computing power judgment unit: selects a data processing task corresponding to a first resource requirement in the second sorting result, and performs computing power judgment on the first real-time computing power resource state in the first sorting result; If the computing power judgment result is that the first real-time computing power resource state in the first sorting result can meet the first resource requirement in the second sorting result, the data processing task corresponding to the first resource requirement in the second sorting result is allocated to the sub-computer corresponding to the first real-time computing power resource state in the first sorting result; If the computing power judgment result is that the first real-time computing power resource state in the first sorting result cannot meet the first resource requirement in the second sorting result, then the computing power judgment is performed on the first resource requirement in the second sorting result and the second real-time computing power resource state in the first sorting result, and so on; If the last real-time computing resource state in the first sorting result also cannot meet the first resource requirement in the second sorting result, the corresponding data processing task is allocated to multiple sub-computers based on the first resource requirement in the second sorting result; Adjustment unit: marking the allocated data processing tasks, re-determining the real-time computing resource status of each sub-computer after allocation, and adjusting the order of each real-time computing resource status after reallocation in the first sorting result; The second computing power judgment unit is used to successively judge the computing power of the data processing tasks corresponding to the resource requirements in the second sorting result and the first sorting result after the order is adjusted.

7. A distributed application system based on a computing power network, characterized in that: include: The application platform as claimed in any one of claims 1 to 6; As well as the management center and dispatching center.

8. A distributed application system based on computing power network according to claim 7, characterized in that: The management center includes multiple management nodes; Management node unit: The management node is used to perform at least one of resource management, application management, performance management, event management, and operation and maintenance management on all resources and all application objects of the computing power network.

9. A distributed application system based on computing power network according to claim 8, characterized in that: The management node unit includes: Resource management subunit: The management node manages all resources of the computing network based on the cloud management interface; Application management subunit: The management node manages the relevant configuration information of all application objects in the computing network, displays various configuration information of application objects based on lists, and monitors the performance, events and status of application objects to achieve application management; Performance management subunit: The management node collects performance indicator data of all resources and all application objects of the computing network based on the performance collection interface, and triggers alarm notifications for performance indicators that exceed the limit according to the threshold configuration to achieve performance management; Event management subunit: The management node collects the fault data and log data of all resources and application objects of the computing network in a unified manner, and performs standardized processing on the abnormal events generated according to the rules to achieve event management; Operation and maintenance management subunit: The managed node inspects all resources and application objects of the computing network according to the preset inspection plan and generates reports. It sends abnormal status or fault alarms during the inspection process to event monitoring for processing to achieve operation and maintenance management.

10. A distributed application system based on computing power network according to claim 7, characterized in that: The dispatch center includes a dispatch hub; Dispatching center unit: The dispatching center is used to process data on the resources or application objects controlled by the management nodes according to the business requirements of the application collaboration sub-platform to obtain application services; Scheduling center unit: The scheduling center is used to connect to the interface of the managed node, collect resources, and provide unified access to various data sources; it is used to connect to the application and task status interface of the managed node, provide management and monitoring of access tasks, and analyze and optimize the perceived data; It is also used to manage artificial intelligence models and algorithms in orchestration and scheduling, as well as optimization strategies for auxiliary strategy generation.

Images