Density peak clustering multi-granularity task decomposition algorithm based on task representation model

Abstract

This invention relates to a density-peak clustering multi-granularity task decomposition algorithm based on a task representation model, applicable to the industrial internet field. The algorithm includes: obtaining the computational resource requirement vector for each atomic task based on its information; normalizing the computational resource requirement vector for each atomic task; calculating the computational resource weight coefficient for each atomic task; calculating the correlation parameters between atomic tasks; calculating the distance metric between atomic tasks; initializing the truncation distance; calculating the local density of each atomic task; calculating the minimum center offset distance; obtaining the cluster centers; and obtaining the clustering results. This invention describes the resource requirements of tasks by constructing a task representation model and uses task weights to cluster tasks, thereby achieving reasonable resource scheduling and allocation. This invention can allocate tasks to the most suitable computing nodes according to their characteristics, optimizing system performance, reducing task latency, and improving resource utilization.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial internet, and particularly relates to a density peak clustering multi-granularity task decomposition algorithm based on a task representation model. BACKGROUND

[0002] Industrial Internet is a technology that combines physical systems with information technology to achieve the digital interconnection of devices, factories, and production lines, and optimizes industrial production processes through data collection, analysis, and feedback. It not only connects industrial equipment with the Internet, but also integrates advanced computing technology, Internet of Things, cloud computing, edge computing, and big data analysis technology, further improving production efficiency, reducing operating costs, and promoting multi-level comprehensive innovation of industrial, business, and technical models.

[0003] Industrial Internet has been widely applied in many fields, including automobile manufacturing, smart grid, agriculture, construction, and healthcare. These fields have achieved real-time monitoring, predictive maintenance, and intelligent optimization through data-driven decision-making, significantly improving production and service quality. With the rise of smart manufacturing and smart factories, industrial Internet has become a core driving force for the digital transformation of manufacturing.

[0004] However, as the network scale expands and the complexity of tasks increases, traditional centralized cloud computing mode is difficult to meet the demand of industrial Internet of Things for massive data transmission and real-time processing. The characteristics of industrial Internet of Things are large data volume, high real-time requirement, and wide node distribution, so unified computing in the cloud center will bring huge network load and affect the response speed and processing efficiency of the system. To overcome these challenges, edge computing is introduced as an important complementary technology to perform data processing on edge devices close to data sources, reducing latency and improving real-time performance.

[0005] The current industrial Internet architecture usually relies on a three-layer end-edge-cloud architecture: terminal devices (such as sensors and machine devices) are responsible for data collection and preliminary processing; edge nodes (such as edge servers and edge clusters) are responsible for more complex real-time processing and computing; and cloud servers handle complex large-scale tasks and data analysis. In this architecture, the reasonable allocation of tasks is crucial, especially since different levels of nodes have different resource characteristics and computing capabilities. For example, terminal devices have weak computing power and are suitable for processing low-computing real-time tasks; edge nodes have moderate computing power and are suitable for processing tasks with moderate computing power and high timeliness requirements; and cloud servers have strong computing and storage capabilities and can handle complex batch tasks.

[0006] Due to the existence of these heterogeneous resources, how to reasonably schedule and decompose tasks according to their characteristics (such as computational complexity, real-time requirements, data dependencies, etc.) has become a key challenge in the field of industrial internet. The resource requirements of tasks vary significantly, manifesting as different requirements for CPU, GPU, memory, storage, and other computing resources. For example, some tasks may require a large amount of CPU resources, while others may rely on GPU for fast computation and processing. Therefore, in order to maximize system performance and resource utilization, tasks must be analyzed and decomposed in a fine-grained manner to ensure that different tasks are allocated reasonably on appropriate nodes.

[0007] Existing algorithms often lack a comprehensive mechanism for characterizing and measuring tasks when clustering them, failing to fully consider the multi-dimensional resource requirements of tasks and their complex interrelationships. This results in task clustering results that may not accurately reflect the actual characteristics of the tasks, thereby affecting the accuracy and efficiency of resource scheduling, especially in complex industrial internet scenarios.

[0008] In the industrial internet environment, tasks often have complex correlations and resource heterogeneity. However, existing algorithms fail to effectively consider the interdependencies between tasks when decomposing them, resulting in low efficiency of task scheduling. For example, when multiple tasks have data dependencies or share resources, an unreasonable execution order can severely impact the execution efficiency and response time of the entire system. In addition, traditional task decomposition algorithms cannot fully consider the heterogeneity of computing node resources and the special needs of tasks, leading to unreasonable resource scheduling and further restricting overall production efficiency.

[0009] Although clustering algorithms are important tools for optimizing resource allocation, some existing clustering algorithms perform poorly when faced with the high-dimensional data and complex task relationships unique to industrial internet. For example, the K-means algorithm requires the number of clusters to be specified in advance and is sensitive to noise and outliers, which can easily lead to biased clustering results. The DBSCAN algorithm may encounter difficulties when dealing with data with large variations in density, and these limitations make it difficult for them to accurately reflect the complex resource requirements of tasks in task decomposition, thereby affecting the degree of intelligent scheduling. Therefore, existing task clustering methods and scheduling algorithms cannot meet the efficient scheduling needs of actual systems when faced with the high heterogeneity and complex task requirements of industrial internet scenarios, and need to be improved. SUMMARY

[0010] In view of the characteristics of various tasks and different requirements in the current industrial internet, and in order to effectively solve the performance bottleneck problem in task scheduling, the application provides a density peak clustering multi-granularity task decomposition algorithm based on a task representation model. The application proposes a task representation model for tasks in the industrial internet to comprehensively characterize and measure the tasks, and proposes a density peak clustering multi-granularity task decomposition algorithm based on the task representation model according to the task representation model, and completes task clustering decomposition. The industrial internet task representation model mainly characterizes the entire properties of the task based on the properties and related relationships of the atomic tasks, such as resource demand (CPU, GPU, memory, external storage), data exchange volume, bandwidth demand, data dependency, etc. The density peak clustering multi-granularity task decomposition algorithm based on the task representation model will reasonably decompose the task into multi-granularity subtasks according to the measurement of the representation model, provide a reasonable selection for the appropriate subtask allocation to the appropriate node in the mobile edge network for the task deployment and scheduling of the industrial internet system, and improve the overall performance of the system.

[0011] The technical solution adopted by the application to solve the technical problems is as follows:

[0012] The application provides a density peak clustering multi-granularity task decomposition algorithm based on a task representation model, mainly including the following steps:

[0013] Step S1: constructing an industrial internet computing task representation model;

[0014] Step S1.1: obtaining the computing resource demand vector of each atomic task according to the information of each atomic task;

[0015] Step S1.2: normalizing the computing resource demand vector of each atomic task;

[0016] Step S1.3: calculating the atomic task self-computing resource weight coefficient;

[0017] Step S1.4: calculating the correlation parameters between atomic tasks;

[0018] Step S2: density peak clustering multi-granularity task decomposition based on the constructed industrial internet computing task representation model;

[0019] Step S2.1: calculating the distance measurement between atomic tasks;

[0020] Step S2.2: initializing the truncation distance;

[0021] Step S2.3: calculating the local density of each atomic task;

[0022] Step S2.4: calculating the minimum center offset distance;

[0023] Step S2.5: Obtain cluster centers;

[0024] Step S2.6: Obtain clustering results.

[0025] Furthermore, in step S1.1, for each atomic task AT i The specific formula for calculating the resource demand vector is as follows:

[0026]

[0027] Among them, CPU i Represents the CPU demand vector, GPU i Mem represents the GPU demand vector. i Represents the memory requirement vector, Stor i This represents the external storage requirement vector.

[0028] Furthermore, in step S1.2, the Min-max method is used for normalization. By centering the data x according to the minimum value and then scaling it according to the range, the data is made dimensionless and converged to a uniform size, forming normalized weights.

[0029]

[0030] Among them, W i,x Indicates atomic task AT i Normalized weights of data x; x i Indicates atomic task AT i The resource requirement of a certain task; max(x) represents the maximum value of data x; min(x) represents the minimum value of data x; the denominator in equation (1), max(x)-min(x), represents the average requirement of all tasks on CPU, GPU, memory and external storage.

[0031] Furthermore, in step S1.3, the atomic task itself calculates the resource weight W. i The specific calculation formula is as follows:

[0032] W i =λ cpu W i,cpu +λ gpu W i,gpu +λ mem W i,mem +λ Stor W i,stor (2)

[0033] Among them, W i,cpu Indicates atomic task AT i CPU demand weight, W i,gpu Indicates atomic task ATi GPU demand weight, W i,mem Indicates atomic task AT i Memory requirement weight, W i,stor Indicates atomic task AT i External storage demand weight, λ cpu This represents the CPU demand adjustment parameter, λ. gpu This represents the GPU demand adjustment parameter, λ. mem λ represents the parameter for adjusting memory requirements. stor This indicates the parameter for adjusting external storage requirements.

[0034] Furthermore, in step S1.4, based on the dependencies and connections between atomic tasks, for any two atomic tasks AT... i ,AT j The correlation coefficient (ATR) between them was calculated. ij The specific calculation formula is as follows:

[0035]

[0036] in, Indicates atomic task AT i and Atomic Task AT j Special needs between them, λ SD SD represents the adjustment parameter for the normalized interactive data volume. ij Indicates atomic task AT i and Atomic Task AT j The normalized interaction data volume between them, λ DD DD represents the adjustment parameter for normalized data dependency. ij Indicates atomic task AT i and Atomic Task AT j Normalized data dependency between them, λ BD BD represents the adjustment parameter for normalized bandwidth demand. ij Indicates atomic task AT i and Atomic Task AT j The normalized bandwidth requirement between them.

[0037] Furthermore, in step S2.1, the specific formula for calculating the distance metric between the atomic tasks is as follows:

[0038]

[0039] Where, d ij Represents any two atomic tasks AT i ,AT j Distance metric between them, W i Indicates atomic task AT iThe resource weights are calculated by itself, W j Indicates atomic task AT j Self-compute resource weights, ATR ij Represents any two atomic tasks AT i ,AT j The correlation coefficient between them.

[0040] Furthermore, in step S2.2, an optimal cutoff distance is found by minimizing the Gini coefficient; the specific formula for calculating the Gini coefficient G is as follows:

[0041]

[0042] Where, ρ i Indicates atomic task AT i Local density; δ i Indicates atomic task AT i The center offset distance; N represents the total number of data points, i.e., the total number of atomic tasks; S represents the sum of the products of local density and center offset distance, used to normalize the contribution value of atomic tasks and facilitate the calculation of the Gini coefficient. Its specific calculation formula is as follows:

[0043]

[0044] Furthermore, in step S2.3, the specific formula for calculating the local density of each atomic task is as follows:

[0045]

[0046] Where, d ij Represents any two atomic tasks AT i ,AT j Distance metric between them, d c The cutoff distance indicates that atomic tasks within a certain range contribute significantly to the density, while atomic tasks outside this range have a smaller impact on the density.

[0047] Furthermore, in step S2.4, the specific formula for calculating the minimum center offset distance is as follows:

[0048]

[0049] Where, d ij Represents any two atomic tasks AT i ,AT j Distance metric between them, ρ i Indicates atomic task AT i The local density, ρ j Indicates atomic task AT j Local density.

[0050] Furthermore, in steps S2.5 and S2.6, cluster centers are selected based on the obtained optimal local density and minimum center offset distance; all atomic tasks are assigned based on the selected cluster centers; each atomic task is added to the nearest central cluster based on its distance from each cluster center, forming several clusters, thus obtaining a multi-granularity task decomposition scheme.

[0051] The beneficial effects of this invention are:

[0052] This invention comprehensively characterizes and measures various computing tasks in the Industrial Internet, considering the multi-dimensional resource requirements of tasks and their interrelationships, and constructs an Industrial Internet computing task representation model. Based on the constructed Industrial Internet computing task representation model, density peak clustering multi-granularity task decomposition is performed: based on the task representation model, a series of related atomic tasks are combined into composite subtasks through an improved algorithm, forming multi-granularity task clusters to optimize scheduling and resource allocation. Compared with existing technologies, this invention has the following advantages:

[0053] This invention, based on the Density Peaks Clustering (DPC) algorithm and a computational task representation model, proposes a multi-granularity task decomposition algorithm based on the task representation model to address the resource allocation problem of computational tasks in the Industrial Internet. The industrial internet computational task representation model constructed in this invention comprehensively considers the computational resource requirements of different tasks, the correlations between tasks, and the task allocation characteristics in the industrial internet environment, aiming to achieve efficient allocation of heterogeneous computing resources. Through multi-granularity clustering, this invention can decompose tasks into different sub-task groups according to their resource requirements and correlations, thereby optimizing resource allocation and improving task execution efficiency. Simulation experiments verify the effectiveness of this invention; the DPC multi-granularity task decomposition algorithm based on the task representation model provided by this invention exhibits superior performance compared to other similar algorithms. This invention is particularly suitable for scenarios with diverse tasks and heterogeneous resources in the Industrial Internet of Things (IIoT), effectively solving the performance bottleneck problem in task scheduling. Attached Figure Description

[0054] Figure 1 The flowchart of a density peak clustering multi-granularity task decomposition algorithm based on a task representation model provided by the present invention is shown.

[0055] Figure 2 This is a diagram showing the relationship between atomic tasks.

[0056] Figure 3 This is the algorithm decision graph.

[0057] Figure 4 The image shows the algorithm results with a granularity of 2.

[0058] Figure 5 The image shows the algorithm results with a granularity of 3.

[0059] Figure 6 The image shows the algorithm results with a granularity of 4.

[0060] Figure 7 The data represents the algorithm evaluation metrics (granularity is 2).

[0061] Figure 8 The data represents the algorithm evaluation metrics (granularity is 3).

[0062] Figure 9 The data represents the algorithm evaluation metrics (granularity is 4). Detailed Implementation

[0063] The present invention will be further described in detail below with reference to the accompanying drawings.

[0064] like Figure 1 As shown, the present invention provides a density peak clustering multi-granularity task decomposition algorithm based on a task representation model, and its specific implementation process is as follows:

[0065] Step 1: Construction of the Industrial Internet Computing Task Representation Model;

[0066] Tasks in the Industrial Internet are typically diverse and heterogeneous, with significant differences in the requirements for computing resources, storage resources, and communication bandwidth. The distributed architecture of the Industrial Internet usually consists of three layers: edge, cloud, and endpoint, with huge differences in computing power between each layer. The cloud side has powerful computing and storage capabilities, suitable for large-scale data analysis and complex computing tasks, but with high latency; the edge side is usually deployed as edge servers or clusters within the factory, with moderate computing and storage capabilities, suitable for handling time-sensitive tasks; the endpoint side is usually various terminal sensors, with limited computing power, and can only perform data acquisition or simple processing.

[0067] For various tasks in the Industrial Internet, the representation model is the foundation of the entire clustering process. It needs to be comprehensively characterized and measured, taking into account the multi-dimensional resource requirements of tasks and the interrelationships between them, so as to optimize the results of task clustering.

[0068] An atomic task, in the context of the Industrial Internet, refers to an indivisible basic task unit in an application. It represents the smallest executable operation or computational unit in a system, typically with clearly defined computational requirements and resource consumption. Atomic tasks can be viewed as fundamental components of complex computational tasks; by combining these atomic tasks, more complex tasks and applications can be built.

[0069] 1) Based on the information of each atomic task, obtain the computational resource requirement vector for each atomic task;

[0070] In the Industrial Internet, the computational resource requirements of different atomic tasks vary significantly. To effectively characterize these requirements and support subsequent clustering and efficient task allocation and scheduling, it is necessary to provide a detailed description of the tasks' demands on various computational resources. Specifically, computational resource requirements can be analyzed and measured from four dimensions: CPU requirements, GPU requirements, memory requirements, and external storage requirements.

[0071] For each atomic task AT i Its computational resource requirements can be represented as a vector:

[0072]

[0073] Among them, CPU i Represents the CPU demand vector, GPU i Mem represents the GPU demand vector. i Represents the memory requirement vector, Stor i This represents the external storage requirement vector.

[0074] In this vector, each component corresponds to the task's requirements for CPU, GPU, memory, and external storage.

[0075] 2) Normalize the computational resource requirement vector for each atomic task;

[0076] Since different tasks have significantly different requirements for computing resources, it is necessary to normalize these computing resources. Normalization can effectively map the computing resource requirements of different types to the same scale range, avoiding misjudgments caused by differences in the original data volume.

[0077] In this invention, the Min-max method is specifically used for normalization. This involves centering the data x according to its minimum value and then scaling it by its range (maximum value - minimum value), thereby achieving dimensionless data transformation and convergence to a uniform scale, forming normalized weights.

[0078]

[0079] Among them, W i,x Indicates atomic task AT i Normalized weights of data x; x i Indicates atomic task AT iThe resource requirement of a certain task; max(x) represents the maximum value of data x; min(x) represents the minimum value of data x; the denominator in equation (1), max(x)-min(x), represents the average requirement of all tasks on CPU, GPU, memory and external storage. By performing row normalization, the differences in the order of magnitude and dimensions of the computational resource requirements of different tasks can be eliminated.

[0080] 3) Calculate the computational resource weight coefficients for the atomic task itself;

[0081] This invention also introduces adjustment parameters to regulate the influence of various computing resources in the total weight, making the model more adaptable. These parameters can be dynamically adjusted according to the needs of actual application scenarios, ensuring reasonable allocation of computing resources in different environments. For example, in some scenarios, GPU resources may be more scarce than CPU resources, so the GPU demand adjustment parameter λ can be appropriately increased. gpu The value of is prioritized for GPU-intensive tasks. Details are shown in Table 1.

[0082] Table 1. Detailed Information on the Computational Resources of the Atomic Task Itself

[0083]

[0084] In summary, the computational resource weight W for each atomic task can be obtained. i This ensures that resources can be examined and measured in multiple dimensions, and that clustering can simultaneously consider factors such as their own needs and resource heterogeneity.

[0085] Among them, atomic task AT i Self-calculated resource weight W i The specific calculation formula is as follows:

[0086] W i =λ cpu W i,cpu +λ gpu W i,gpu +λ mem W i,mem +λ Stor W i,stor (2)

[0087] Among them, W i,cpu Indicates atomic task AT i CPU demand weight, W i,gpu Indicates atomic task AT i GPU demand weight, W i,mem Indicates atomic task AT i Memory requirement weight, W i,stor Indicates atomic task AT iExternal storage demand weight, λ cpu This represents the CPU demand adjustment parameter, λ. gpu This represents the GPU demand adjustment parameter, λ. mem λ represents the parameter for adjusting memory requirements. stor This indicates the parameter for adjusting external storage requirements.

[0088] By characterizing the computational resource requirements of each atomic task in detail across all dimensions, a weight can be established for each atomic task element that is related to its computational resource requirements, ensuring that the differences in computational resource requirements of atomic tasks can be fully considered during clustering.

[0089] 4) Calculate the correlation parameters between atomic tasks;

[0090] When constructing a representation model, it is also necessary to consider the metrics of relationships between atomic tasks. These relationships manifest in multiple dimensions, particularly in data exchange, bandwidth requirements, data dependencies, and communication latency. Properly considering these relationships helps optimize the clustering results of atomic tasks, avoids excessive bottlenecks and uneven resource allocation during communication and data transmission, and ultimately improves the overall performance of the system.

[0091] To quantify and characterize the relationships between atomic tasks, the following parameters are used to describe the dependencies and connections between them: amount of interactive data, data dependency, bandwidth requirements, and special requirements. Detailed parameters are shown in Table 2.

[0092] Table 2 Correlation parameters among atomic tasks

[0093]

[0094] Therefore, the correlation coefficient (ATR) between atomic tasks can be calculated. ij The specific calculation formula is as follows:

[0095]

[0096] in, Indicates atomic task AT i and Atomic Task AT j Special needs between them, λ SD SD represents the adjustment parameter for the normalized interactive data volume. ij Indicates atomic task AT i and Atomic Task AT j The normalized interaction data volume between them, λ DD DD represents the adjustment parameter for normalized data dependency. ij Indicates atomic task AT i and Atomic Task AT jNormalized data dependency between them, λ BD BD represents the adjustment parameter for normalized bandwidth demand. ij Indicates atomic task AT i and Atomic Task AT j The normalized bandwidth requirement between them.

[0097] Based on the dependencies and relationships between atomic tasks, for any two atomic tasks AT i ,AT j The correlation coefficient (ATR) between them was calculated. ij This provides a foundation for subsequent calculations of the distances and local densities between atomic task elements. These correlation coefficients reflect the closeness between atomic task elements and are the core basis for subsequent calculations of atomic task distance metrics.

[0098] The computational resource requirement vector for each atomic task obtained through the above calculations illustrates the computational resource requirements of each dimension of the atomic task. Simultaneously, based on the properties of each atomic task, correlation parameters between atomic tasks are obtained. According to scenario requirements, necessary adjustment parameters are set to complete the construction of the industrial internet computing task representation model.

[0099] Step 2: Density peak clustering multi-granularity task decomposition based on the industrial internet computing task representation model;

[0100] Task decomposition in the Industrial Internet involves considering the functional components, resource requirements, and interrelationships of each atomic task, making the process quite complex. Therefore, an effective method is needed to classify and group tasks, forming composite subtasks from related atomic tasks to create task clusters, thus optimizing scheduling and resource allocation. Clustering algorithms can help divide tasks into different clusters, identifying connections and trends between atomic tasks even in the absence of supervised information.

[0101] Clustering algorithms are unsupervised learning methods used to group data into different clusters, ensuring high similarity within the same cluster and low similarity between different clusters. Density Peaks Clustering (DPC) is a density-based clustering algorithm. Its core idea is to determine cluster centers based on the distance between local density points and points with higher density. Its advantages include the ability to determine the number of clusters based on density distribution, better handling of noise and outliers, and effective processing of data clusters with different shapes and densities, uncovering the natural structure inherent in the data. These advantages are well-suited to the complexity of atomic tasks and the heterogeneity of resources in the Industrial Internet, offering significant improvements over other traditional clustering algorithms. Therefore, this paper proposes a multi-granularity task decomposition algorithm based on a task representation model using Density Peaks Clustering.

[0102] 1) Calculate the distance metric between atomic tasks;

[0103] In density peak clustering algorithms, the distance metric d ij This is a crucial step, used to measure the similarity between various data points (atomic tasks in this invention). In task decomposition and resource scheduling, a reasonable distance metric can effectively reflect the interrelationships between tasks and the differences in their resource requirements, thereby helping to cluster related tasks into the same cluster. A good distance metric not only needs to reflect the differences in resource requirements between atomic tasks, but also needs to capture the dependencies between tasks. Therefore, this invention proposes a representation-based distance metric, the specific calculation formula of which is as follows:

[0104]

[0105] Where, d ij Represents any two atomic tasks AT i ,AT j Distance metric between them, W i Indicates atomic task AT i The resource weights are calculated by itself, W j Indicates atomic task AT j Self-compute resource weights, ATR ij Represents any two atomic tasks AT i ,AT j The correlation coefficient between them.

[0106] Distance metrics can simultaneously consider the properties of the atomic tasks themselves as well as the correlations between atomic tasks. This ensures that tasks with high resource requirements have a greater impact on distance calculations, while also making the distance smaller the stronger the correlation between tasks, thus more accurately reflecting the interrelationships between tasks.

[0107] 2) Initialize the cutoff distance;

[0108] Cutoff distance d c Equally crucial, it determines which atomic tasks are considered closely related neighbors, thus affecting the calculation of local density. In traditional density peak clustering algorithms, the cutoff distance d... c Typically, the selection is done manually or based on experience, and this subjectivity can lead to inaccurate or unstable clustering results. Therefore, this invention introduces the Gini coefficient to optimize the selection of the cutoff distance. The specific formula for calculating the Gini coefficient G is as follows:

[0109]

[0110] Where, ρ i Indicates atomic task AT i Local density; δ i Indicates atomic task AT i The center offset distance; N represents the total number of data points, i.e., the total number of atomic tasks; S represents the sum of the products of local density and center offset distance, used to normalize the contribution value of atomic tasks and facilitate the calculation of the Gini coefficient. The specific calculation formula is as follows:

[0111]

[0112] By minimizing the Gini coefficient, an optimal cutoff distance can be found, which balances the local density distribution of atomic tasks and thus yields reasonable clustering results.

[0113] 3) Calculate the local density of each atomic task;

[0114] Local density ρ i Local density is also a core parameter used to measure the data distribution density of an atomic task within its neighborhood. In density-peak clustering algorithms, local density reflects the density of other atomic tasks surrounding an atomic task, determining whether that atomic task is likely to become a cluster center. The introduction of local density has two important functions: first, it helps the algorithm find regions with high local density, making atomic tasks in these regions more likely to be selected as cluster centers; second, it enhances the algorithm's robustness to noise, because noise points in sparse regions usually have low local density, thus avoiding being mistakenly identified as cluster centers. Local density ρ i The specific calculation formula is as follows:

[0115]

[0116] Where, d ij Represents any two atomic tasks ATi ,AT j Distance metric between them, d c The cutoff distance represents the range within which atomic tasks contribute significantly to the density, while those outside this range have a smaller impact. In this way, the density peak clustering algorithm can identify high-density atomic tasks that are far from other high-density atomic tasks as cluster centers.

[0117] 4) Calculate the minimum center offset distance;

[0118] With local density ρ i One relevant parameter is the center offset distance, which determines the distance of each atomic task relative to atomic tasks with higher local density. This center offset distance helps distinguish atomic tasks with high local density that are far from other high-density atomic tasks, thus making it easier to identify cluster centers. Specifically, the minimum center offset distance is defined as: for any atomic task AT i Its minimum center offset distance δ i This refers to the atomic task AT i With all local densities greater than that of the atomic task AT i The minimum distance between atomic tasks is calculated using the following formula:

[0119]

[0120] Where, d ij Represents any two atomic tasks AT i ,AT j Distance metric between them, ρ i Indicates atomic task AT i The local density, ρ j Indicates atomic task AT j Local density.

[0121] 5) Obtain cluster centers;

[0122] Cluster centers are selected based on the optimal local density and minimum center offset distance obtained above.

[0123] 6) Obtain the clustering results;

[0124] Based on the selected cluster centers, all atomic tasks are assigned. Each atomic task is added to the nearest cluster center according to its distance from each cluster center, thus forming several clusters and ultimately obtaining a multi-granularity task decomposition scheme.

[0125] This invention provides a density peak clustering multi-granularity task decomposition algorithm (MDPC-TM) based on a task representation model. To verify the effectiveness of the MDPC-TM algorithm, the K-Means algorithm and the IDPC-Dcom algorithm are used as comparison schemes, and the full decomposition (Full Dcom) method is used as the baseline reference. Clustering experiments are performed on all atomic tasks participating in the clustering decomposition, and the clustering decomposition results are compared.

[0126] In this embodiment, the test environment computer hardware is a 12th Gen Intel Core i7-12700H (2.30GHz), and the operating system version is Windows 1123H2. The algorithm is implemented based on Python 3.9, which depends on scikit-learn version 1.0.2.

[0127] (I) Experimental Methods and Evaluation Indicators

[0128] The experiment will evaluate the algorithm's performance using the following metrics: inter-cluster interaction data volume, inter-cluster bandwidth requirement, and inter-cluster data dependency. These metrics can effectively assess the performance of each sub-task group forming a cluster after task clustering decomposition.

[0129] 1) Inter-cluster data exchange volume

[0130] Explanation: After the algorithm completes, a series of clusters can be obtained. Each resulting cluster after task clustering decomposition is regarded as a set of subtasks. The amount of data exchanged between clusters refers to the total amount of data that needs to be exchanged between each cluster, used to characterize the amount of information that needs to be transferred between clusters after task classification.

[0131] Significance of evaluation: When a clustering algorithm is good, the amount of data exchanged between clusters is small. This is because tasks that require a lot of data transfer between atomic tasks can be clustered into the same cluster and deployed on the same server or the same cluster. This converts a large amount of network I / O into information transfer within the server or cluster, improving service performance while reducing the consumption of network resources and reducing the negative impact of network fluctuations on the service.

[0132] 2) Inter-cluster bandwidth requirements

[0133] Explanation: Inter-cluster bandwidth requirement measures the total network bandwidth needed between clustered task clusters. For each pair of clusters, if there is a data exchange requirement between their tasks, a certain amount of bandwidth is needed to transmit the data. The smaller the bandwidth requirement, the better the algorithm can group bandwidth-intensive tasks within the same cluster, thereby reducing the bandwidth consumption between clusters.

[0134] Significance of the evaluation: When the clustering algorithm performs well, the inter-cluster bandwidth requirement is low. This indicates that after task partitioning, tasks with high bandwidth requirements are effectively grouped together, reducing inter-cluster bandwidth needs and the potential for network bottlenecks caused by bandwidth demands, thus lowering the demand for bandwidth resources. Conversely, a large bandwidth requirement indicates more inter-cluster communication and a higher demand for network bandwidth resources. This may lead to network transmission delays or insufficient bandwidth, which is detrimental to the execution of real-time tasks, increases system load, and causes network resource strain.

[0135] 3) Inter-cluster data dependencies

[0136] Meaning Explanation: Inter-cluster data dependency measures the data dependencies between tasks within each cluster after task partitioning. Tasks with high data dependencies typically rely on the same input or upstream data streams. If these tasks are assigned to different clusters, it may lead to the generation of multiple unnecessary data copies to meet the needs of inter-cluster data exchange. Such data dispersion increases inter-cluster communication overhead and places additional burden on network resources due to transmission demands. Simultaneously, the generation of data copies also consumes additional CPU computing resources and network I / O, thereby reducing overall system efficiency. Therefore, inter-cluster data dependency can serve as an important indicator for evaluating the rationality of task clustering.

[0137] Evaluation Significance: Inter-cluster data dependency reflects the data dependency status between clusters after task clustering. Ideally, clustering algorithms should aggregate tasks with high data dependencies into the same cluster as much as possible, thereby reducing inter-cluster transmission requirements and improving system performance and resource utilization. If the inter-cluster data dependency is low, it indicates that the clustering result effectively reduces cross-cluster data coupling and the task allocation is reasonable; while high data dependency means increased cross-network data traffic, additional data replication, and potentially increased consumption of CPU, I / O, memory, and other resources, leading to resource fragmentation and low resource utilization.

[0138] The experimental dataset is the same as that used in the control algorithm, consisting of a set of computational tasks comprising nine sub-tasks. The dataset includes the CPU, GPU, memory, and external storage requirements for each task. It also includes the amount of interaction data, data dependencies, and bandwidth requirements between atomic tasks. The number of resource nodes was set to 3, resulting in cluster granularity of 2, 3, and 4. To balance the resources, the parameters were adjusted using an average method, without biasing towards specific resources or relationships. The specific experimental steps are as follows:

[0139] Step 1: Based on the relationships between individual atomic tasks, export a relationship diagram of all atomic tasks in the application, such as... Figure 2 As shown.

[0140] Step 2: Derive a suitable cutoff distance using the adaptive constraint distance dc based on Gini parameters optimized by the algorithm, form a decision graph, and obtain the constraint density and minimum relative distance.

[0141] Step 3: Generate multi-granularity results (2, 3, or 4 clusters), obtain cluster centers and clustering results, and compare the results with other algorithms at this granularity according to the selected evaluation index to verify the algorithm.

[0142] (II) Experimental Results

[0143] After running the algorithm, the decision graph of the clustering algorithm and the decomposition results at different granularities can be obtained. The decision graph is used in algorithms such as density peak clustering to help identify cluster centers. It visualizes the distribution of data points and clustering characteristics by showing the relationship between local density and distance of data points. The decision graph is as follows: Figure 3 As shown in the decision graph, points 0, 2, 4, and 5 are significantly far from other data points, indicating that they can serve as cluster centers. In selecting cluster centers, this invention selects different numbers of cluster centers according to different granularities, thereby achieving multi-granularity cluster decomposition.

[0144] Clustering results for granularities of 2, 3, and 4 are shown below. Figure 4 , Figure 5 , Figure 6 As shown.

[0145] Based on the evaluation metrics, the amount of inter-cluster interaction data, inter-cluster bandwidth requirements, and inter-cluster data dependency parameters of the MDPC-TM algorithm, K-Means algorithm, and IDPC-Dcom algorithm at different granularities were calculated. For ease of intuitive comparison, all results were normalized using the Baseline algorithm Full Dcom as the benchmark.

[0146] 1) Scenario 1: Clustering granularity is 2;

[0147] Table 3 presents the results for each algorithm evaluation metric at a granularity of 2. To more intuitively illustrate the performance of each algorithm under different evaluation metrics, a bar chart is shown below. Figure 7 As shown.

[0148] Table 3. Algorithm evaluation index results (granularity 2)

[0149] Algorithm Amount of data exchanged inter-cluster Data dependency inter-cluster Amount of bandwidth required inter-cluster MDPC-TM 0.192533 0.088235 0.168835 IDPC-Dcom 0.216666 0.071786 0.206186 K-Means 0.27416 0.098124 0.228133 FullDcom 1 1 1

[0150] From Table 3 and Figure 7As can be seen, all three algorithms, compared to Full Dcom decomposition, can optimize the amount of inter-cluster data exchange, inter-cluster data dependency, and inter-cluster bandwidth requirements. MDPC-TM's inter-cluster data exchange volume at a granularity of 2 is approximately 0.19, significantly smaller than IDPC-Dcom and K-Means. This smaller inter-cluster data exchange volume indicates that MDPC-TM can effectively group tasks requiring large amounts of data exchange within the same cluster, thereby reducing the amount of data exchange between different clusters.

[0151] Regarding inter-cluster data dependency, MDPC-TM's value is close to IDPC-Dcom, slightly higher than IDPC-Dcom, but significantly better than K-Means. The lower inter-cluster data dependency indicates that MDPC-TM effectively allocates tasks with strong data dependencies to the same cluster during task partitioning, thereby reducing cross-cluster data coupling. This helps reduce data traffic and avoids adding extra data replicas that consume various resources.

[0152] In the comparison of inter-cluster bandwidth requirements, MDPC-TM performed best, indicating that the low bandwidth requirement means that MDPC-TM can reduce bandwidth consumption between different clusters, which is beneficial to optimizing the overall system performance and saving network resources.

[0153] 2) Scenario 2: Clustering granularity is 3;

[0154] The results for each algorithm evaluation metric at a granularity of 3 are shown in Table 4, and the comparison chart is shown below. Figure 8 As shown.

[0155] Table 4. Algorithm evaluation index results (granularity 3)

[0156] Algorithm Amount of data exchanged inter-cluster Data dependency inter-cluster Amount of bandwidth required inter-cluster MDPC-TM 0.363636 0.264706 0.421127 IDPC-Dcom 0.383333 0.294416 0.486835 K-Means 0.473156 0.305354 0.548932 FullDcom 1 1 1

[0157] From Table 4 and Figure 8 As can be seen, with the granularity increasing to 3, the amount of inter-cluster data exchanged by MDPC-TM increases compared to granularity 2, but it is still superior to IDPC-Dcom and K-Means. This result further verifies that MDPC-TM can maintain a relatively low data exchange volume while increasing the task allocation granularity, thus maintaining its advantage in controlling the amount of inter-cluster data exchange.

[0158] In terms of inter-cluster data dependencies, MDPC-TM outperforms IDPC-Dcom and K-Means. This indicates that as the granularity increases, MDPC-TM's multi-dimensional processing of the properties of atomic tasks and their relationships during the representation model construction process can reasonably divide the dependencies between tasks, reducing cross-cluster dependencies.

[0159] MDPC-TM performs best in terms of bandwidth requirements, demonstrating its excellent task clustering capabilities and ability to reduce bandwidth consumption between different task clusters.

[0160] 3) Scenario 3: Clustering granularity is 4;

[0161] The results for each algorithm evaluation metric at a granularity of 4 are shown in Table 5, and the comparison chart is shown below. Figure 9 As shown.

[0162] Table 5. Algorithm evaluation index results (granularity 4)

[0163] Algorithm Amount of data exchanged inter-cluster Data dependency inter-cluster Amount of bandwidth required inter-cluster MDPC-TM 0.533427 0.395214 0.695463 IDPC-Dcom 0.633333 0.413625 0.747621 K-Means 0.785108 0.531288 0.781027 FullDcom 1 1 1

[0164] With a particle size of 4, we can refer to Table 6 and... Figure 9 It was observed that MDPC-TM significantly outperformed other comparative algorithms in terms of inter-cluster data exchange volume, inter-cluster data dependency, and inter-cluster bandwidth requirements.

[0165] In summary, the rationality and superiority of the density peak clustering multi-granularity task decomposition algorithm based on the task representation model of the present invention have been verified. In particular, when the granularity is higher, MDPC-TM can still control the data exchange, data dependency and bandwidth requirements between clusters well, and has excellent performance.

[0166] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A density peak clustering multi-granularity task decomposition algorithm based on a task representation model, characterized in that, Includes the following steps: Step S1: Construct an industrial internet computing task representation model; Step S1.1: Based on the information of each atomic task, obtain the computational resource requirement vector for each atomic task; For each atomic task The specific formula for calculating the resource demand vector is as follows: ； in, Represents the CPU demand vector. Represents the GPU demand vector. Represents the memory requirement vector. Represents the external storage demand vector; Step S1.2: Normalize the computational resource requirement vector for each atomic task; The Min-max method is used for normalization. By centering the data x according to its minimum value and then scaling it according to its range, the data is made dimensionless and converges to a uniform size, forming normalized weights. (1)； in, Representing atomic tasks Normalized weights of data x; Representing atomic tasks The demand for a certain resource; This represents the maximum value of the data x; This represents the minimum value of data x; the denominator in equation (1) is... This represents the average CPU, GPU, memory, and external storage requirements for all tasks. Step S1.3: Calculate the computational resource weight coefficient of the atomic task itself; The atomic task itself calculates resource weights. The specific calculation formula is as follows: (2)； in, Representing atomic tasks CPU demand weight, Representing atomic tasks GPU demand weight, Representing atomic tasks Memory requirement weight, Representing atomic tasks External storage demand weight, This indicates the CPU demand adjustment parameters. This indicates the parameters that require GPU adjustment. This indicates the parameters for adjusting memory requirements. This indicates the parameter for adjusting external storage requirements; Step S1.4: Calculate the correlation parameters between atomic tasks; Based on the dependencies and relationships between atomic tasks, for any two atomic tasks The correlation coefficient between them was calculated. The specific calculation formula is as follows: ； in, Representing atomic tasks and atomic tasks Special needs between them This parameter represents the adjustment parameter for the amount of normalized interactive data. Representing atomic tasks and atomic tasks Normalized interaction data volume between them The adjustment parameter representing the dependence of normalized data. Representing atomic tasks and atomic tasks Normalized data dependency between them The adjustment parameter represents the normalized bandwidth requirement. Representing atomic tasks and atomic tasks Normalized bandwidth requirements between; Step S2: Perform density peak clustering multi-granularity task decomposition based on the constructed industrial internet computing task representation model; Step S2.1: Calculate the distance metric between atomic tasks; The specific formula for calculating the distance metric between the atomic tasks is as follows: (4)； in, Represents any two atomic tasks Distance metric between Representing atomic tasks It calculates resource weights itself. Representing atomic tasks It calculates resource weights itself. Represents any two atomic tasks The correlation coefficient between them; Step S2.2: Initialize the cutoff distance; Step S2.3: Calculate the local density of each atom task; Step S2.4: Calculate the minimum center offset distance; Step S2.5: Obtain cluster centers; Step S2.6: Obtain clustering results.

2. The density peak clustering multi-granularity task decomposition algorithm based on the task representation model according to claim 1, characterized in that, In step S2.2, a suitable cutoff distance is found by minimizing the Gini coefficient; the Gini coefficient The specific calculation formula is as follows: ； in, Representing atomic tasks Local density; Representing atomic tasks The center offset distance; N represents the total number of data points, i.e., the total number of atomic tasks; S represents the sum of the products of local density and center offset distance, used to normalize the contribution value of atomic tasks and facilitate the calculation of the Gini coefficient. Its specific calculation formula is as follows: 。 3. The density peak clustering multi-granularity task decomposition algorithm based on the task representation model according to claim 1, characterized in that, In step S2.3, the specific formula for calculating the local density of each atomic task is as follows: ； in, Represents any two atomic tasks Distance metric between The cutoff distance indicates that atomic tasks within a certain range contribute significantly to the density, while atomic tasks outside this range have a smaller impact on the density.

4. The density peak clustering multi-granularity task decomposition algorithm based on the task representation model according to claim 1, characterized in that, In step S2.4, the specific formula for calculating the minimum center offset distance is as follows: (7)； in, Represents any two atomic tasks Distance metric between Representing atomic tasks Local density, Representing atomic tasks Local density.

5. The density peak clustering multi-granularity task decomposition algorithm based on the task representation model according to claim 1, characterized in that, In steps S2.5 and S2.6, cluster centers are selected based on the obtained optimal local density and minimum center offset distance; all atomic tasks are assigned based on the selected cluster centers; each atomic task is added to the nearest cluster based on its distance from each cluster center, forming several clusters, thus obtaining a multi-granularity task decomposition scheme.

Images