Resource matching method, device, equipment, storage medium and product

Through the methods of fuzzy clustering processing and bilateral comprehensive matching degree generation, the problem of difficulty in matching tasks and resources in cross-domain environments is solved, and real-time, reasonable matching and dynamic scheduling of tasks and resources in cross-domain scenarios is achieved.

CN118567868BActive Publication Date: 2024-11-26CHINA MOBILE INFORMATION TECHNOLOGY CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411040494.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-26
Estimated Expiration
2044-07-31

AI Technical Summary

Technical Problem

The existing technology cannot effectively deal with the complex and changeable situations of state changes during task operation in a cross-domain environment, resulting in resource matching strategies that cannot meet dynamic scheduling requirements, resulting in difficulty in matching tasks and resources.

Method used

By performing fuzzy clustering processing based on the task demand model and the resource supply model, a candidate target set is generated, and a preferred scheduling target set is generated based on the bilateral comprehensive matching degree to achieve dynamic matching between tasks and resources.

Benefits of technology

In cross-domain scenarios, real-time and reasonable matching of tasks and resources is achieved, and it can cope with system changes and resource balance, ensuring the normal execution of tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118567868B_ABST
    Figure CN118567868B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of cloud computing technology, and discloses a resource matching method, device, equipment, storage medium and product. The present invention performs fuzzy clustering processing according to a task requirement model and a resource supply model to obtain a candidate target set corresponding to the task to be matched in the task requirement model; determines the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set; generates a preferred scheduling target set corresponding to the task to be matched based on the bilateral comprehensive matching degree and the candidate target set; and performs resource matching for the task to be matched based on the preferred scheduling target set. Since clustering is performed according to a task requirement model that can characterize the real-time task state and a resource supply model of the real-time computing center resource information, and a preferred scheduling target set is generated according to the clustered results for resource matching, system changes and the balance between resources and tasks are taken into account in resource matching, ensuring that tasks and resources can be matched in real time and reasonably in cross-domain scenarios.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and in particular to a resource matching method, device, equipment, storage medium and product. Background Art

[0002] Big data processing involves massive amounts of data and huge amounts of computing, so sufficient resources are needed to support the execution of tasks. In distributed computing, resources are allocated in the form of containers, and each container is an independent execution environment with a certain amount of resources. However, in the process of allocating resources to tasks, insufficient resources or unreasonable allocation may occur, resulting in the inability to execute tasks normally, especially in a cross-domain environment, which generally involves cross-domain matching of computing resources from multiple computing centers, making it more difficult to match tasks with resources.

[0003] Nowadays, the task and resource matching strategy is generally to perform an estimation analysis before the task starts, or to fix a certain matching strategy before the task starts, and then no adjustment is made during the entire task operation. The factors considered in the two-way matching process are relatively simple, but the cross-domain scenario is complex and changeable, and the state changes of the task during the operation cannot be predicted in advance. Therefore, static resource estimation and scheduling strategies cannot cope with such changeable cross-domain scenarios. Summary of the invention

[0004] The main purpose of the present invention is to provide a resource matching method, device, equipment, storage medium and product, aiming to solve the technical problem that the existing resource matching strategy cannot meet the resource matching requirements of complex and changeable cross-domain scenarios.

[0005] To achieve the above object, the present invention provides a resource matching method, which comprises the following steps:

[0006] Perform fuzzy clustering processing according to the task demand model and the resource supply model to obtain a set of candidate targets corresponding to the tasks to be matched in the task demand model, wherein the task demand model is used to characterize the task requirements corresponding to the current tasks, and the resource supply model is used to characterize the resource information corresponding to the current computing centers;

[0007] Determining a bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set;

[0008] Generate a preferred scheduling target set corresponding to the task to be matched based on the bilateral comprehensive matching degree and the candidate target set;

[0009] Resources are matched for the tasks to be matched based on the preferred scheduling target set.

[0010] Optionally, the step of determining the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set includes:

[0011] Traversing the candidate target set to obtain a current candidate target;

[0012] Obtaining a task-oriented matching degree and a resource-oriented matching degree between the current candidate target and the task to be matched;

[0013] Determine the bilateral comprehensive matching degree between the task to be matched and the current candidate target according to the resource matching degree and the task matching degree;

[0014] At the end of the traversal, the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set is obtained.

[0015] Optionally, the step of obtaining the task-oriented matching degree and resource-oriented matching degree between the current candidate target and the task to be matched includes:

[0016] Obtaining the affiliation similarity between the current candidate target and the task to be matched, and obtaining the task-oriented matching degree;

[0017] Obtain resource information corresponding to the current candidate target and task requirement resource information corresponding to the task to be matched;

[0018] The resource matching degree is determined according to the resource information and the required resource information.

[0019] Optionally, the step of determining the resource matching degree according to the resource information and the required resource information includes:

[0020] Calculate resource load according to the resource information and the required resource information;

[0021] Standardizing the resource load to obtain a standard resource load;

[0022] The absolute value of the standard resource load is calculated to obtain the resource matching degree.

[0023] Optionally, the step of determining the bilateral comprehensive matching degree between the task to be matched and the current candidate target according to the resource-oriented matching degree and the task-oriented matching degree includes:

[0024] Calculating the arithmetic mean of the resource matching degree and the task matching degree to obtain a first matching degree;

[0025] Calculating the geometric mean of the resource matching degree and the task matching degree to obtain a second matching degree;

[0026] The first matching degree and the second matching degree are weightedly summed based on a preset matching degree weight to obtain a bilateral comprehensive matching degree between the task to be matched and the current candidate target.

[0027] Optionally, the step of performing fuzzy clustering processing according to the task requirement model and the resource supply model to obtain a set of candidate targets corresponding to the task to be matched in the task requirement model includes:

[0028] Pairing the task demand model with the resource supply model to obtain at least one task-resource pair;

[0029] Eliminate the task resource pairs that do not meet the preset screening condition from the at least one task resource pair to obtain a resource pair to be processed;

[0030] Perform fuzzy clustering processing on the resource pairs to be processed to obtain a set of candidate targets corresponding to the tasks to be matched in the task requirement model.

[0031] In addition, to achieve the above object, the present invention also proposes a resource matching device, which includes the following modules:

[0032] A clustering module is used to perform fuzzy clustering processing according to a task requirement model and a resource supply model to obtain a set of candidate targets corresponding to the tasks to be matched in the task requirement model. The task requirement model is used to characterize the task requirements corresponding to the current tasks, and the resource supply model is used to characterize the resource information corresponding to the current computing centers.

[0033] A determination module, used to determine the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set;

[0034] A generation module, configured to generate a preferred scheduling target set corresponding to the task to be matched based on the bilateral comprehensive matching degree and the candidate target set;

[0035] A matching module is used to perform resource matching for the task to be matched based on the preferred scheduling target set.

[0036] In addition, to achieve the above-mentioned purpose, the present invention also proposes a resource matching device, which includes: a processor, a memory, and a resource matching program stored in the memory and executable on the processor, and the resource matching program implements the steps of the resource matching method described above when executed by the processor.

[0037] In addition, to achieve the above-mentioned purpose, the present invention further proposes a computer-readable storage medium, on which a resource matching program is stored, and when the resource matching program is executed, the steps of the resource matching method described above are implemented.

[0038] In addition, to achieve the above-mentioned purpose, the present invention also proposes a computer program product, which includes a resource matching program, and the resource matching program implements the steps of the resource matching method described above when executed.

[0039] The present invention performs fuzzy clustering processing according to the task requirement model and the resource supply model to obtain a candidate target set corresponding to the task to be matched in the task requirement model; determines the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set; generates a preferred scheduling target set corresponding to the task to be matched based on the bilateral comprehensive matching degree and the candidate target set; and performs resource matching for the task to be matched based on the preferred scheduling target set. Since clustering is performed based on the task requirement model that can characterize the real-time task state and the resource supply model of the real-time computing center resource information, and resource matching is performed based on the clustered results to generate a preferred scheduling target set, system changes and the balance between resources and tasks are taken into account in resource matching, ensuring that tasks and resources can be matched in real time and reasonably in cross-domain scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the structure of an electronic device in a hardware operating environment involved in an embodiment of the present invention;

[0041] Figure 2 It is a flowchart of the first embodiment of the resource matching method of the present invention;

[0042] Figure 3 A schematic diagram for generating a preferred scheduling target set according to an embodiment of the present invention;

[0043] Figure 4 It is a flowchart of the second embodiment of the resource matching method of the present invention;

[0044] Figure 5 A schematic diagram of the structure of a scheduling system according to an embodiment of the present invention;

[0045] Figure 6 A schematic diagram of the overall resource matching process according to an embodiment of the present invention;

[0046] Figure 7 It is a structural block diagram of the first embodiment of the resource matching device of the present invention.

[0047] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0049] Reference Figure 1 , Figure 1 The diagram is a schematic diagram of the structure of a resource matching device for a hardware operating environment according to an embodiment of the present invention.

[0050] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0051] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0052] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a resource matching program.

[0053] exist Figure 1 In the electronic device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the resource matching device, and the electronic device calls the resource matching program stored in the memory 1005 through the processor 1001, and executes the resource matching method provided by the embodiment of the present invention.

[0054] The embodiment of the present invention provides a resource matching method, referring to Figure 2 , Figure 2 The figure is a flow chart of a first embodiment of a resource matching method of the present invention.

[0055] In this embodiment, the resource matching method includes the following steps:

[0056] Step S10: Perform fuzzy clustering processing according to the task requirement model and the resource supply model to obtain a set of candidate targets corresponding to the tasks to be matched in the task requirement model.

[0057] It should be noted that the executor of this embodiment may be the resource matching device, and the resource matching device may be a device that can manage and control various devices in a distributed computing system, such as personal computers, servers and other electronic devices, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the resource matching method of the present invention is explained by taking the resource matching device as an example.

[0058] It should be noted that the task demand model can be a data model used to characterize the task demand corresponding to each current task, and the resource supply model is a data model used to characterize the resource information corresponding to each computing center in the current distributed computing system. Among them, the task demand can include the number of computing cores required, storage capacity required, and estimated working time, etc., and the resource information corresponding to the computing center can include the total number of computing cores in the computing center, the number of running computing cores, the total number of computing cores of the task demand waiting to be processed, the total storage capacity of the computing center, the used storage capacity, the network bandwidth of the computing center, and other information.

[0059] In the specific implementation, the task requirement is a description of the amount of resources required for the submitted computing task, the expected resource status and strategy, etc., where the resource amount refers to the number of resources such as computing or storage.

[0060] If we assume that the task requirements formally described have the number of computing cores required C t , Storage capacity requirement S t and estimated working time C t,pred , then each task in the distributed computing system can use the computing core number requirement C t , Storage capacity requirement S t and estimated working time C t,pred The triple is described as:

[0061]

[0062] The task requirement model includes the task requirements of all current tasks in the distributed computing system, which can be represented by an M*3 matrix:

[0063]

[0064] In this matrix (i.e., task demand model), a row vector represents a task in the distributed computing system, M is the total number of tasks, and a column vector represents the demand of different tasks for the same resources.

[0065] A computing center (a computing center can be a platform composed of multiple physical servers or virtual servers) is formally described as an overall resource that includes multiple resource capabilities. Resource capabilities are used to describe the resource status and policies contained in the computing center. In order to more fully describe the resource status to support matching, the number of elements in the resource capability model will be more than the number of elements in the task-side resource requirement model.

[0066] Assume that the computing resources include the total number of cores C in the computing center p , the number of cores running in the computing center C p,run , and the total number of cores C required by the tasks queued in the platform p,wait , in terms of storage resources, it includes the total storage capacity S of the computing center p and the used storage capacity S p,used , in terms of network resources, it includes the network bandwidth W of the computing center p ;

[0067] Then the resource information of each computing center in the distributed computing system can be described by a six-tuple:

[0068]

[0069] The resource supply model contains the resource information corresponding to each computing center in the current distributed computing system, which can be represented by an N*6 matrix:

[0070]

[0071] In this matrix (i.e., resource supply model), each row vector represents the resource description of a computing center, and a column vector represents the situation of a certain type of resource characteristics in each computing center. Different types of demand column vectors in the resource demand model need to be matched by the corresponding resource column vectors in the resource supply model. The resource supply model can more intuitively reflect the global resource status and strategy, thereby more effectively supporting task scheduling. Among them, each computing center in the distributed computing system can collect its own resource information and send it to the resource matching device, which can summarize it to form a resource supply model.

[0072] In actual use, the fuzzy c-means clustering algorithm (FCM) can be used to perform fuzzy clustering processing on the task requirement model and the resource supply model to determine the candidate target set corresponding to the task to be matched in the task requirement model. The task requirement model can contain multiple tasks to be matched, and each task to be matched can correspond to a candidate target set.

[0073] Among them, FCM obtains the membership of each cross-domain task sample point to all class centers by optimizing the objective function, thereby determining the category of the task sample point to achieve the purpose of automatically classifying the task sample data. Specifically, each task sample will be assigned a membership function belonging to each cluster, and the classification of the task will be determined by the size of the membership value.

[0074] The objective function of FCM is as follows:

[0075]

[0076] Wherein, m is the number of clustering categories. In this embodiment, tasks are divided into six categories: computing-intensive tasks, storage-intensive tasks, bandwidth-intensive tasks, computing and storage-intensive tasks, computing and bandwidth-intensive tasks, and storage and bandwidth-intensive tasks. N is the total number of task samples, and C is the number of cluster centers. j represents the jth cluster center, which has the same feature dimension as the task sample, p i represents the i-th task sample, u ij Represents the task sample p i For cluster center C j The membership degree can be understood as p i Belongs to C j probability.

[0077] The membership matrix is ​​an N∗C-dimensional matrix, which indicates the degree to which each task sample point belongs to each class. i For each cluster, the closer the membership value is to 1, the higher the membership is, and vice versa. The sum is 1. The specific implementation is shown in the following formula:

[0078]

[0079] C j The specific implementation is shown in the following formula:

[0080]

[0081] For ease of understanding, an example is given below to illustrate that the implementation method of using the FCM algorithm for fuzzy clustering to determine the candidate target set corresponding to each task to be matched in the task requirement model can be expressed in the following pseudo code:

[0082] {Input: task side demand matrix P, center side supply matrix T

[0083] Output: coarse-grained candidate target set I

[0084] Set the cluster category m=6 and randomly initialize its cluster center coordinates

[0085] Initialize the task membership matrix U using random values ​​[0,1] t and the central resource membership matrix U p

[0086] for do:

[0087] Calculate the task membership matrix Ut by formula

[0088] Update the task cluster center C by calculating the formula j

[0089] end for

[0090] for do:

[0091] The center resource membership matrix U is calculated by the formula p

[0092] The update center resource cluster center C is calculated by formula j

[0093] end for

[0094]

[0095] return I}

[0096] In a specific implementation, in order to reduce unnecessary calculations, the above step S10 in this embodiment may include:

[0097] Pairing the task demand model with the resource supply model to obtain at least one task-resource pair;

[0098] Eliminate the task resource pairs that do not meet the preset screening condition from the at least one task resource pair to obtain a resource pair to be processed;

[0099] Perform fuzzy clustering processing on the resource pairs to be processed to obtain a set of candidate targets corresponding to the tasks to be matched in the task requirement model.

[0100] It should be noted that pairing the task demand model and the resource supply model to obtain at least one task resource pair can be performed by pairing the task demand model and the resource supply model in a cross-combination manner to generate at least one task resource pair. For example, assuming that the task demand model contains two tasks to be matched, A and B, and the resource supply model contains two computing centers, C and D, then four task resource pairs, AC, AD, BC, and BD, can be generated.

[0101] If the task-resource pair meets the preset screening conditions, it means that the task requirements and resource capabilities corresponding to the task-resource pair do not match. Therefore, the task-resource pair can be cleared, thereby effectively reducing the search domain for solving the matching problem, thereby improving the matching efficiency of the subsequent matching process.

[0102] The preset screening conditions can be pre-set by the administrator of the resource matching device. For example, for the task t described by the triple and the computing center p described by the sextuple, the preset screening conditions can be described by the following formula:

[0103]

[0104] The above formula represents the number of computing cores required for the task during the resource condition screening process. t Not more than , and the storage capacity requirement of the task is S t Must not exceed available storage capacity in the computing center .

[0105] Step S20: Determine the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set.

[0106] It should be noted that when the candidate target set corresponding to the task to be matched is determined, the candidate targets included in the candidate target set (i.e., candidate computing centers) can actually all execute the task to be matched. However, when actually matching resources, it is also necessary to consider the impact on the computing center, the impact on the overall distributed computing system, and the related impacts during task execution. It is also necessary to ensure that the most suitable computing center can be selected from the candidate target set. Therefore, the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set can be determined.

[0107] Among them, the bilateral comprehensive matching degree can be used to characterize the matching degree between tasks and resources. If the bilateral comprehensive matching degree between the task to be matched and the candidate target is higher, it means that the candidate target is more suitable for executing the task to be matched.

[0108] Step S30: generating a preferred scheduling target set corresponding to the task to be matched based on the bilateral comprehensive matching degree and the candidate target set.

[0109] It should be noted that generating a preferred scheduling target set corresponding to the task to be matched based on the bilateral comprehensive matching degree and the candidate target set can be by sorting the candidate targets in the candidate target set from large to small according to the corresponding bilateral comprehensive matching, and using the sorted candidate target set as the preferred scheduling set.

[0110] For example: assuming that for the task t1 to be matched, its corresponding candidate target set is {p1, p2, p3, p4, p5, p6}, where the bilateral comprehensive matching degrees between the task t1 to be matched and the candidate targets are ranked from large to small as p1>p2>p3>p6>p5>p4, then the preferred scheduling target set generated at this time is {p1, p2, p3, p6, p5, p4}.

[0111] For ease of understanding, now combined Figure 3 This is for illustration only, but not for limitation. Figure 3 A schematic diagram is generated for the preferred scheduling target set of this embodiment, such as Figure 3 As shown, if there are multiple tasks to be matched (including t1-tm), a corresponding preferred scheduling target set will be generated for each task to be matched.

[0112] Step S40: performing resource matching for the task to be matched based on the preferred scheduling target set.

[0113] It should be noted that performing resource matching for the task to be matched based on the preferred scheduling target set may be performing resource matching for the task to be matched according to the ranking of each computing center in the preferred scheduling target set as the matching priority.

[0114] In actual use, there may be multiple tasks to be matched. At this time, for multiple tasks to be matched, the priority can be set according to the time order in which the tasks are submitted to the distributed computing system (that is, early submission and early allocation), and resources can be matched for each task to be matched in order. After each resource allocation, the above steps S10-S40 can be re-executed to ensure that a more reasonable computing center can be selected when performing resource matching.

[0115] This embodiment performs fuzzy clustering processing according to the task requirement model and the resource supply model to obtain a candidate target set corresponding to the task to be matched in the task requirement model; determines the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set; generates a preferred scheduling target set corresponding to the task to be matched based on the bilateral comprehensive matching degree and the candidate target set; and performs resource matching for the task to be matched based on the preferred scheduling target set. Since clustering is performed based on the task requirement model that can characterize the real-time task status and the resource supply model of the real-time computing center resource information, and resource matching is performed based on the clustered results to generate a preferred scheduling target set, system changes and the balance between resources and tasks are taken into account in resource matching, ensuring that tasks and resources can be matched in real time and reasonably in cross-domain scenarios.

[0116] refer to Figure 4 , Figure 4 It is a flowchart diagram of a second embodiment of a resource matching method of the present invention.

[0117] Based on the first embodiment above, step S20 of the resource matching method of this embodiment includes:

[0118] Step S201: traverse the candidate target set to obtain the current candidate target.

[0119] It should be noted that traversing the candidate target set can be traversing the candidate target set in the index order of the candidate target set, and taking the traversed candidate target as the current candidate target. Of course, other methods can also be used for traversal, such as reverse traversal, etc., which is not limited in this embodiment.

[0120] Step S202: Obtaining the task-oriented matching degree and resource-oriented matching degree between the current candidate target and the task to be matched.

[0121] It should be noted that the task-oriented matching degree can be a quantitative value used to evaluate whether resources will lead to unbalanced use of computing and storage resources while meeting task requirements, which may affect the allocation of other tasks. The greater the task-oriented matching degree, the more balanced the use of computing and storage resources. The resource-oriented matching degree can be a quantitative value used to evaluate the adaptability between resources and tasks from the perspective of load balancing of the computing center's resource capabilities. The greater the resource-oriented matching degree, the higher the adaptability between resources and tasks.

[0122] In a specific implementation, in order to ensure that the task matching degree and resource matching degree can be reasonably calculated, step S202 in this embodiment may include:

[0123] Obtaining the affiliation similarity between the current candidate target and the task to be matched, and obtaining the task-oriented matching degree;

[0124] Obtain resource information corresponding to the current candidate target and task requirement resource information corresponding to the task to be matched;

[0125] The resource matching degree is determined according to the resource information and the required resource information.

[0126] It should be noted that the task-oriented matching degree is mainly used to evaluate whether resources will lead to unbalanced use of computing and storage resources while meeting task requirements, which may affect the allocation of other tasks. Therefore, it is necessary to measure the similarity between the task requirement vector and the resource capability vector, and the affiliation similarity can characterize this similarity. Therefore, the affiliation similarity between the current candidate target and the task to be matched can be used as the task-oriented matching degree.

[0127] Among them, since the clustering basis was also based on the affiliation similarity when the fuzzy clustering was performed before, the affiliation similarity between the current candidate target and the task to be matched is obtained at this time. The task-oriented matching degree can be obtained by extracting the affiliation similarity between the current candidate target and the task to be matched from the candidate target set and using it as the task-oriented matching degree.

[0128] For example: the final candidate target set I is obtained in the above pseudocode, and the candidate target set I can be traversed by a nested loop to read the affiliation similarity between each task to be matched and each computing center, so as to obtain the task-oriented matching degree between each task to be matched and each computing center.

[0129] In actual use, since the resource matching degree can be a quantitative value used to evaluate the adaptability between resources and tasks from the perspective of load balancing of the computing center's resource capabilities, it is necessary to calculate the impact of allocating tasks to the computing center on the load capacity of the computing center. Therefore, the resource information corresponding to the current candidate target and the task requirement resource information corresponding to the task to be matched can be obtained.

[0130] In specific applications, determining the resource matching degree according to the resource information and the required resource information may be to calculate the resource load of the computing center after allocation according to the resource information and the required resource information, thereby obtaining the resource matching degree.

[0131] In a specific implementation, in order to ensure that the value of the resource matching degree is normal, the step of determining the resource matching degree according to the resource information and the required resource information in this embodiment may include:

[0132] Calculate resource load according to the resource information and the required resource information;

[0133] Standardizing the resource load to obtain a standard resource load;

[0134] The absolute value of the standard resource load is calculated to obtain the resource matching degree.

[0135] It should be noted that during the standardization process, min-max standardization process may be used. Of course, other similar processing methods may also be used.

[0136] To facilitate understanding, an example is given below:

[0137] Resource matching mainly considers the resource load of each computing center and the global load balancing factor, and is used to evaluate whether allocating tasks to specific resources will cause an imbalance in the global resource load of the system. Resource load is mainly divided into computing resource load and storage resource load. Therefore, in terms of computing resource load, since each computing center has running tasks and queued tasks, it is necessary to comprehensively consider the computing resource load caused by all tasks. The computing resource load L of a specific computing center t,p,comp It is expressed as follows:

[0138]

[0139] Among them, C p represents the total number of cores in the computing center, C p,run represents the number of cores running in the computing center, C p,wait represents the total number of cores required by the tasks queued in the computing center, C t Indicates the number of computing cores required for the submitted task.

[0140] In terms of storage resource load, the storage resource occupancy rate after the task is assigned to the computing center is considered. If a copy of the data required by the task already exists in the computing center, the storage resource demand of the task is not considered in the storage resource load. t,p,stor It is expressed as follows:

[0141]

[0142] Among them, S p represents the total storage capacity of the computing center, S p,used represents the used storage capacity of the computing center, S t Indicates the storage capacity requirement of the task.

[0143] Finally, the computing resource load and storage resource load are processed by min-max normalization to obtain and ,The comprehensive idle space of the computing center is measured by the computing resource load and the storage resource load, and the specific form is shown in the following formula:

[0144]

[0145] Finally, resources are matched to the matching degree β t,p The value range of is (0,1), and the specific expression is as follows:

[0146]

[0147] Step S203: determining the bilateral comprehensive matching degree between the task to be matched and the current candidate target according to the resource matching degree and the task matching degree.

[0148] In actual use, after obtaining the resource-oriented matching degree and the task-oriented matching degree, the bilateral comprehensive matching degree between the task to be matched and the current candidate target can be calculated according to the resource-oriented matching degree and the task-oriented matching degree.

[0149] In a specific implementation, in order to ensure that the value of the bilateral comprehensive matching degree is reasonable, step S203 in this embodiment may include:

[0150] Calculating the arithmetic mean of the resource matching degree and the task matching degree to obtain a first matching degree;

[0151] Calculating the geometric mean of the resource matching degree and the task matching degree to obtain a second matching degree;

[0152] The first matching degree and the second matching degree are weightedly summed based on a preset matching degree weight to obtain a bilateral comprehensive matching degree between the task to be matched and the current candidate target.

[0153] It should be noted that the preset matching degree weight may be pre-set by the administrator of the resource matching device.

[0154] Specifically, the bilateral comprehensive matching degree M t,p The calculation formula can be shown as follows:

[0155]

[0156] Where ε is the preset matching weight parameter, and its value range is (0,1), M t,p The value range is (0,1), α t,p is the task-oriented matching degree, β t,p The matching degree of resources.

[0157] It should be noted that the bilateral comprehensive matching degree is expressed by the weighted sum of the arithmetic mean and geometric mean of the task-oriented matching degree and the resource-oriented matching degree, which can reflect the trade-off relationship between the two and take into account the bilateral consistency and bilateral complementarity of tasks and resources. This means that the task-oriented matching degree and the resource-oriented matching degree can make up for each other's shortcomings. Therefore, the subsequent resource matching based on the bilateral comprehensive matching degree can ensure that multiple factors between tasks and resources are fully considered in the matching process, so as to better meet the needs and performance requirements of the system.

[0158] Step S204: At the end of the traversal, the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set is obtained.

[0159] It can be understood that if the traversal is completed, then each candidate target in the candidate target set has been traversed once, that is, a bilateral comprehensive matching degree calculation has been performed once. Therefore, the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set can be obtained.

[0160] For ease of understanding, now combined Figure 5 and 6 This is for illustration only, but not for limitation. Figure 5 is a schematic diagram of the scheduling system structure of this embodiment, Figure 6 The figure is a schematic diagram of the overall resource matching process of this embodiment.

[0161] like Figure 5 As shown, the distributed computing system involved in this embodiment includes multiple computing centers (computing center 1-computing center N). In order to facilitate resource matching, an inter-center scheduler (i.e., the resource matching device of this embodiment) and an intra-center scheduler are set in the distributed computing system.

[0162] Among them, the center-to-center scheduler is used to coordinate and manage global resources and generate matching solutions, and mainly includes the following modules: resource information summary module, task queue module, resource matching module, scheduling decision module and execution optimization module. The main functional modules involved in the resource matching method of this embodiment are resource information summary module, task queue module and resource matching module. Among them, the resource information summary module and task queue module are the preliminary preparation stage of resource matching. The specific functions of each functional module are as follows:

[0163] Task queue module: When a task is submitted to a cross-domain distributed computing environment, this module collects all task information and describes the task in a unified manner, and then adds the task to the task requirement model.

[0164] Resource information aggregation module: This module collects resource information in the cross-domain distributed computing environment based on the feedback from each computing center, describes the resources in a unified manner, and then updates the resource information to the resource capability model.

[0165] Resource matching module: Dynamically optimize the matching strategy of demand and supply based on the bilateral comprehensive matching degree, fully consider the bilateral comprehensive matching degree between global task requirements and resource capabilities, and avoid complex matching processes, so as to achieve efficient matching of diversified task requirements and resource capabilities in cross-domain environments.

[0166] The central scheduler is distributed in each computing center and is used to manage and monitor the tasks and resources in the computing center. It includes the following two modules: scheduling module and resource information collection module. The specific functions of each module are as follows:

[0167] Scheduling module: This module further arranges and controls the tasks assigned to the local computing center and is the actual executor of scheduling decisions;

[0168] Resource information collection module: This module regularly collects resource information of the local computing center and sends it to the resource information aggregation module in the inter-center scheduler to generate a resource supply model on the center side.

[0169] When matching specific resources, the overall process is as follows: Figure 6 The steps shown may include:

[0170] 1. Basic resource condition screening (clearing the task resources that do not meet the preset screening conditions in the task to be built)

[0171] 2. Fuzzy clustering to obtain a coarse-grained candidate target set (i.e., fuzzy clustering to obtain a candidate target set corresponding to each task to be matched)

[0172] 3. Task-to-resource matching (i.e. obtaining task-to-resource matching)

[0173] 4. Resource matching calculation (i.e. obtaining resource matching)

[0174] 5. Bilateral comprehensive matching (i.e. obtaining the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set)

[0175] 6. Generate a preferred scheduling target set (i.e., generate a preferred scheduling target set corresponding to the task to be matched based on the bilateral comprehensive matching degree and the candidate target set).

[0176] This embodiment obtains the task item matching degree and the resource matching degree, and then calculates the bilateral comprehensive matching degree based on the task matching degree and the resource matching degree, so as to ensure that when resource matching is subsequently performed based on the calculated bilateral comprehensive matching degree, it can ensure that multiple factors between tasks and resources are fully considered in the matching process, thereby better meeting the needs and performance requirements of the system.

[0177] In addition, an embodiment of the present invention further provides a storage medium, on which a resource matching program is stored. When the resource matching program is executed by a processor, the steps of the resource matching method described above are implemented.

[0178] In addition, an embodiment of the present invention further provides a computer program product, which includes a resource matching program. When the resource matching program is executed, the steps of the resource matching method described above are implemented.

[0179] Reference Figure 7 , Figure 7 It is a structural block diagram of the first embodiment of the resource matching device of the present invention.

[0180] like Figure 7 As shown, the resource matching device proposed in the embodiment of the present invention includes:

[0181] The clustering module 10 is used to perform fuzzy clustering processing according to the task demand model and the resource supply model to obtain a set of candidate targets corresponding to the tasks to be matched in the task demand model, wherein the task demand model is used to characterize the task requirements corresponding to the current tasks, and the resource supply model is used to characterize the resource information corresponding to the current computing centers;

[0182] A determination module 20, used to determine the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set;

[0183] A generating module 30, configured to generate a preferred scheduling target set corresponding to the task to be matched based on the bilateral comprehensive matching degree and the candidate target set;

[0184] The matching module 40 is used to perform resource matching for the task to be matched based on the preferred scheduling target set.

[0185] This embodiment performs fuzzy clustering processing according to the task requirement model and the resource supply model to obtain a candidate target set corresponding to the task to be matched in the task requirement model; determines the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set; generates a preferred scheduling target set corresponding to the task to be matched based on the bilateral comprehensive matching degree and the candidate target set; and performs resource matching for the task to be matched based on the preferred scheduling target set. Since clustering is performed based on the task requirement model that can characterize the real-time task status and the resource supply model of the real-time computing center resource information, and resource matching is performed based on the clustered results to generate a preferred scheduling target set, system changes and the balance between resources and tasks are taken into account in resource matching, ensuring that tasks and resources can be matched in real time and reasonably in cross-domain scenarios.

[0186] Furthermore, the determination module 20 is also used to traverse the candidate target set to obtain the current candidate target; obtain the task-oriented matching degree and resource-oriented matching degree between the current candidate target and the task to be matched; determine the bilateral comprehensive matching degree between the task to be matched and the current candidate target based on the resource-oriented matching degree and the task-oriented matching degree; at the end of the traversal, obtain the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set.

[0187] Furthermore, the determination module 20 is also used to obtain the affiliation similarity between the current candidate target and the task to be matched, and obtain the task matching degree; obtain the resource information corresponding to the current candidate target and the task requirement resource information corresponding to the task to be matched; and determine the resource matching degree based on the resource information and the requirement resource information.

[0188] Furthermore, the determination module 20 is also used to calculate the resource load according to the resource information and the required resource information; standardize the resource load to obtain a standard resource load; calculate the absolute value of the standard resource load to obtain a resource matching degree.

[0189] Furthermore, the determination module 20 is also used to calculate the arithmetic mean of the resource matching degree and the task matching degree to obtain a first matching degree; calculate the geometric mean of the resource matching degree and the task matching degree to obtain a second matching degree; and perform weighted summation of the first matching degree and the second matching degree based on a preset matching degree weight to obtain a bilateral comprehensive matching degree between the task to be matched and the current candidate target.

[0190] Furthermore, the clustering module 10 is also used to pair the task requirement model and the resource supply model to obtain at least one task resource pair; remove the task resource pairs that do not meet the preset screening conditions in the at least one task resource pair to obtain the resource pairs to be processed; perform fuzzy clustering processing on the resource pairs to be processed to obtain a set of candidate targets corresponding to the tasks to be matched in the task requirement model.

[0191] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

[0192] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0193] In addition, for technical details not fully described in this embodiment, reference can be made to the resource matching method provided in any embodiment of the present invention, and will not be repeated here.

[0194] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0195] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0196] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0197] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A resource matching method, characterized in that: The resource matching method comprises the following steps: Perform fuzzy clustering processing according to the task demand model and the resource supply model to obtain a set of candidate targets corresponding to the tasks to be matched in the task demand model, wherein the task demand model is used to characterize the task demand corresponding to each current task, and the task demand is characterized by the number of computing cores required, the storage capacity required, and the estimated working time; the resource supply model is used to characterize the resource information corresponding to each current computing center, and the resource information is characterized by the total number of cores in the computing center, the number of cores running in the computing center, the total number of cores required by the queued tasks, the total storage capacity of the computing center, the used storage capacity, and the network bandwidth of the computing center; Determining a bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set; Generate a preferred scheduling target set corresponding to the task to be matched based on the bilateral comprehensive matching degree and the candidate target set; Performing resource matching for the task to be matched based on the preferred scheduling target set; The step of performing fuzzy clustering processing according to the task requirement model and the resource supply model to obtain a set of candidate targets corresponding to the task to be matched in the task requirement model includes: Pairing the task demand model and the resource supply model in a cross-combination manner to generate at least one task-resource pair; Eliminate the task resource pairs that do not meet the preset screening condition from the at least one task resource pair to obtain a resource pair to be processed; Performing fuzzy clustering processing on the resource pairs to be processed to obtain a set of candidate targets corresponding to the tasks to be matched in the task requirement model; The step of determining the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set includes: Traversing the candidate target set to obtain a current candidate target; Obtaining the affiliation similarity between the current candidate target and the task to be matched, and obtaining the task-oriented matching degree; Obtain resource information corresponding to the current candidate target and task requirement resource information corresponding to the task to be matched; Calculate the resource load of each computing center according to the resource information and the required resource information; Standardizing the resource load to obtain a standard resource load; Calculating the absolute value of the standard resource load to obtain the resource matching degree; Determine the bilateral comprehensive matching degree between the task to be matched and the current candidate target according to the resource matching degree and the task matching degree; At the end of the traversal, a bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set is obtained; Wherein, the resource load includes computing resource load and storage resource load; The calculation formula corresponding to the computing resource load is: The calculation formula corresponding to the storage resource load is: The calculation formula corresponding to the resource matching degree is: In the formula, C p represents the total number of cores in the computing center, C p,run represents the number of cores running in the computing center, C p,wait represents the total number of cores required by the tasks queued in the computing center, C t Indicates the number of computing cores required for the submitted task; S p represents the total storage capacity of the computing center, S p,used represents the used storage capacity of the computing center, S t represents the storage capacity requirement of the task; β t,p For resource matching, To standardize the computing resource load, This is the storage resource load after normalization.

2. The resource matching method according to claim 1, characterized in that: The step of determining the bilateral comprehensive matching degree between the task to be matched and the current candidate target according to the resource matching degree and the task matching degree comprises: Calculating the arithmetic mean of the resource matching degree and the task matching degree to obtain a first matching degree; Calculating the geometric mean of the resource matching degree and the task matching degree to obtain a second matching degree; The first matching degree and the second matching degree are weightedly summed based on a preset matching degree weight to obtain a bilateral comprehensive matching degree between the task to be matched and the current candidate target.

3. A resource matching device, characterized in that: The resource matching device comprises the following modules: A clustering module is used to perform fuzzy clustering processing according to a task demand model and a resource supply model to obtain a set of candidate targets corresponding to the tasks to be matched in the task demand model. The task demand model is used to characterize the task demand corresponding to each current task, and the task demand is characterized by the number of computing cores required, the storage capacity required, and the estimated working time. The resource supply model is used to characterize the resource information corresponding to each current computing center, and the resource information is characterized by the total number of cores in the computing center, the number of cores running in the computing center, the total number of cores required by queued tasks, the total storage capacity of the computing center, the used storage capacity, and the network bandwidth of the computing center; A determination module, used to determine the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set; A generation module, configured to generate a preferred scheduling target set corresponding to the task to be matched based on the bilateral comprehensive matching degree and the candidate target set; A matching module, used for performing resource matching for the task to be matched based on the preferred scheduling target set; The clustering module is further used to pair the task demand model and the resource supply model in a cross-combination manner to generate at least one task resource pair; remove the task resource pairs that do not meet the preset screening conditions in the at least one task resource pair to obtain a resource pair to be processed; perform fuzzy clustering processing on the resource pair to be processed to obtain a set of candidate targets corresponding to the task to be matched in the task demand model; The determination module is further used to traverse the candidate target set to obtain the current candidate target; Obtaining the affiliation similarity between the current candidate target and the task to be matched, and obtaining the task-oriented matching degree; Obtain resource information corresponding to the current candidate target and task demand resource information corresponding to the task to be matched; calculate the resource load of each computing center according to the resource information and the demand resource information; perform standardization on the resource load to obtain a standard resource load; calculate the absolute value of the standard resource load to obtain a resource-oriented matching degree; determine the bilateral comprehensive matching degree between the task to be matched and the current candidate target according to the resource-oriented matching degree and the task-oriented matching degree; at the end of the traversal, obtain the bilateral comprehensive matching degree between the task to be matched and each candidate target in the candidate target set; Wherein, the resource load includes computing resource load and storage resource load; The calculation formula corresponding to the computing resource load is: The calculation formula corresponding to the storage resource load is: The calculation formula corresponding to the resource matching degree is: In the formula, C p represents the total number of cores in the computing center, C p,run represents the number of cores running in the computing center, C p,wait represents the total number of cores required by the tasks queued in the computing center, C t Indicates the number of computing cores required for the submitted task; S p represents the total storage capacity of the computing center, S p,used represents the used storage capacity of the computing center, S t represents the storage capacity requirement of the task; β t,p For resource matching, To standardize the computing resource load, This is the storage resource load after normalization.

4. A resource matching device, characterized in that: The resource matching device comprises: a processor, a memory, and a resource matching program stored in the memory and executable on the processor. When the resource matching program is executed by the processor, the steps of the resource matching method according to any one of claims 1 to 2 are implemented.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a resource matching program, and when the resource matching program is executed, the steps of the resource matching method according to any one of claims 1 to 2 are implemented.

6. A computer program product, characterized in that The computer program product includes a resource matching program, and when the resource matching program is executed, the steps of the resource matching method according to any one of claims 1 to 2 are implemented.