Scheduling method and device of cooperative computing platform, equipment, medium and product

Abstract

The invention provides a scheduling method and device of a cooperative computing platform, equipment, a medium and a product. The method comprises the steps of obtaining waiting task lists of a plurality of computing platforms under the condition that computing resource idle information sent by a target computing platform is received; screening target tasks meeting constraint conditions from waiting task lists of the plurality of computing platforms; inserting the target task into a target position of a waiting task list of the target computing platform to execute the target task; based on a preset time interval, respectively calculating data access popularity and data storage cost of the task data in the plurality of computing platforms; and according to the data access popularity and the data storage cost of the task data, adjusting the storage positions of the task data in the plurality of computing platforms. According to the method provided by the invention, the technical problems of low global resource utilization rate and low system throughput of a plurality of computing platforms in related technologies can be solved.

Description

Technical Field

[0001] The present application belongs to the field of big data technology, and in particular relates to a scheduling method, device, equipment, medium and product for a collaborative computing platform. Background Art

[0002] In distributed collaborative computing, multiple computing platforms are involved in executing tasks simultaneously. In order to improve the operating efficiency of system computing on multiple computing platforms, it is necessary to schedule the multiple running computing platforms to optimize the global resource utilization of multiple computing platforms and minimize the task execution time.

[0003] In the related technology, the data layout method is usually adopted, but this method ignores the state of computing resources, and may cause unbalanced task allocation during the task scheduling process, thereby affecting the system throughput.

[0004] Therefore, there is an urgent need for a scheduling method for collaborative computing platforms that can effectively schedule multiple running computing platforms to optimize the global resource utilization of multiple computing platforms and improve system throughput. Summary of the invention

[0005] The embodiments of the present application provide a scheduling method, apparatus, device, storage medium and computer program product for a collaborative computing platform to solve the technical problems of low global resource utilization and long task execution time for multiple computing platforms in related technologies.

[0006] In a first aspect, an embodiment of the present application provides a scheduling method for a collaborative computing platform, the method comprising:

[0007] Upon receiving computing resource idle information sent by the target computing platform, obtaining waiting task lists of multiple computing platforms;

[0008] Selecting target tasks that meet constraint conditions from waiting task lists of multiple computing platforms, wherein the constraint conditions are determined according to computing resource occupancy of the computing platform, task priority, task computing time, and task data migration time;

[0009] Inserting the target task into a target position of a waiting task list of a target computing platform to execute the target task, wherein the target position is obtained by parsing resource idle information;

[0010] Based on a preset time interval, respectively calculate the data access heat and data storage cost of the task data in multiple computing platforms;

[0011] Adjust the storage location of task data on multiple computing platforms based on the data access popularity and data storage cost of the task data.

[0012] In a second aspect, an embodiment of the present application provides a scheduling device for a collaborative computing platform, the device comprising:

[0013] An acquisition module, used to acquire waiting task lists of multiple computing platforms upon receiving computing resource idle information sent by a target computing platform;

[0014] A screening module is used to screen target tasks that meet constraint conditions from waiting task lists of multiple computing platforms, wherein the constraint conditions are determined according to the computing resource occupancy rate of the computing platform, the task priority, the task computing time, and the task data migration time;

[0015] An insertion module, used for inserting a target task into a target position of a waiting task list of a target computing platform to execute the target task, wherein the target position is obtained by parsing resource idle information;

[0016] A calculation module, used to calculate the data access heat and data storage cost of task data in multiple computing platforms based on a preset time interval;

[0017] The adjustment module is used to adjust the storage location of task data on multiple computing platforms according to the data access popularity and data storage cost of the task data.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising:

[0019] A processor and a memory storing programs or instructions;

[0020] The above method is implemented when the processor executes the program or instruction.

[0021] In a fourth aspect, an embodiment of the present application provides a machine-readable storage medium, on which a program or instruction is stored, and the program or instruction implements the above method when executed by a processor.

[0022] In a fifth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the above method.

[0023] In an embodiment of the present application, by screening and retrieving tasks from multiple computing platforms and backfilling them into the target computing platform when there are idle computing resources on the target computing platform, the idle computing resources of the target computing platform are utilized, the load on the remaining computing platforms is reduced, and the global resource utilization of the system is improved. Furthermore, the present application can shorten the time required for task scheduling by adjusting the storage location of task data on multiple computing platforms based on preset time intervals, thereby improving the throughput of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0025] Figure 1 is a schematic diagram of expected execution states of a scheduling method for a collaborative computing platform according to an embodiment of the present invention;

[0026] Figure 2 is a schematic diagram of the actual execution state of a scheduling method for a collaborative computing platform according to an embodiment of the present invention;

[0027] Figure 3 is a schematic diagram of expected execution time of a scheduling method for a collaborative computing platform according to an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of actual execution time of a scheduling method for a collaborative computing platform according to an embodiment of the present invention;

[0029] Figure 5 is a flowchart of a scheduling method for a collaborative computing platform according to an embodiment of the present invention;

[0030] Figure 6 is a task scheduling flow diagram of a scheduling method for a collaborative computing platform according to an embodiment of the present invention;

[0031] Figure 7 is a schematic diagram of a data scheduling process of a scheduling method for a collaborative computing platform according to an embodiment of the present invention;

[0032] Figure 8 It is a schematic diagram of the overall process of a scheduling method for a collaborative computing platform according to an embodiment of the present invention;

[0033] Fig. 9 is a structural block diagram of a scheduling device of a collaborative computing platform according to an embodiment of the present invention;

[0034] Fig.10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0036] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0037] In addition, it should be noted that the acquisition, storage, use, and processing of data in the embodiments of this application are in compliance with the relevant provisions of national laws and regulations. It should be noted that in the embodiments of this application, some existing solutions in the industry such as certain software, components, and models may be mentioned, and they should be considered as exemplary. Their purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0038] For ease of understanding, before explaining the solution provided in the present application, the inventive concept of the solution provided in the present application is first explained.

[0039] In a wide area network environment, scheduling information delays will cause inaccurate time overheads such as task calculation time, data migration time, and queue waiting time predicted during the system scheduling decision process, resulting in inaccurate execution status of tasks during actual execution and the expected status during the scheduling decision process, which in turn leads to low task execution efficiency and affects system performance. Figure 1 is a schematic diagram of an expected execution state of a scheduling method for a collaborative computing platform according to an embodiment of the present invention, such as Figure 1 As shown in the figure, the system consists of four computing platforms, core 1, core 2, core 3, and core 4. At this time, the 23 tasks in the system fully utilize all computing resources and can complete all task execution within T time.

[0040] Figure 2 is a schematic diagram of the actual execution state of a scheduling method for a collaborative computing platform according to an embodiment of the present invention, such as Figure 2 As shown in the figure, if the scheduling information in the wide area network environment is delayed, the task t 5 ( Figure 2 The actual task completion time of the black block is longer than expected, and the task t 5 The extension of execution time may trigger a series of changes, which will eventually lead to the overall task completion time of the system increasing by T+ΔT. Figure 2 The blank blocks in the figure show that the computing resources in the system are not fully utilized, resulting in low efficiency in executing system tasks.

[0041] On the other hand, due to the high latency and low bandwidth of the WAN, system data migration will be slow, causing tasks to wait for data for a long time, which will reduce system performance. When a task is waiting for computing resources in the queue of the target computing platform, the task data required by the task is also being migrated to the target computing platform. After the task completes data migration and queue waiting, it can start computing. In an example, Figure 3 is a schematic diagram of the expected execution time of a scheduling method for a collaborative computing platform according to an embodiment of the present invention, such as Figure 3 As shown, at this time, the seven tasks in the system perform queue waiting, data migration and task calculation processes in sequence. Figure 4 is a schematic diagram of the actual execution time of a scheduling method for a collaborative computing platform according to an embodiment of the present invention, such as Figure 4 As shown in the figure, due to the slow data migration in the WAN environment, the task t 2 ,t 4 ,t 6 The actual data migration time is longer than expected. The actual execution of the system is as follows: Figure 4 The extended task data migration time may cause a series of task delays, and the later the task is ranked, the longer it will take to wait for data, up to Furthermore, since the tasks in the system wait for data for a long time, computing resources become idle.

[0042] In summary, the scheduling information delay problem and slow data migration result in long task execution time and idle computing resources, which leads to a decrease in system throughput, low global resource utilization of the computing platform, and low task execution efficiency.

[0043] In order to solve the problems of the prior art, an embodiment of the present application provides a scheduling method for a collaborative computing platform, which can be applied in the research and development or application scenarios of a cross-domain distributed collaborative computing platform. Figure 5FIG. 1 is a flow chart of a scheduling method for a collaborative computing platform according to an embodiment of the present invention. Figure 5 As shown, the scheduling method of the collaborative computing platform may include the following steps:

[0044] S510, upon receiving computing resource idle information sent by a target computing platform, obtaining waiting task lists of multiple computing platforms.

[0045] In S510, the target computing platform can be any one of multiple computing platforms. When any one of the multiple computing platforms sends computing resource idle information, the computing resource idle information sent by the target computing platform can be received through the scheduling system. The scheduling system can be connected with multiple computing platforms to form a distributed collaborative computing platform system (hereinafter referred to as the system). Each computing platform may include a task waiting list, and the task waiting list includes several tasks waiting for calculation.

[0046] It can be understood that the purpose of obtaining the waiting task lists of multiple computing platforms is to retrieve several tasks in the waiting state in the waiting task lists of multiple computing platforms, and to fill the retrieved tasks back into the target computing platform. In this way, by rescheduling the tasks in the system, the global utilization of system resources can be achieved, thereby improving the system throughput.

[0047] S520, screening target tasks that meet constraint conditions from waiting task lists of multiple computing platforms, wherein the constraint conditions are determined according to the computing resource occupancy rate of the computing platform, task priority, task computing time, and task data migration time.

[0048] In S520, before calling several tasks in the waiting state in the waiting task lists of multiple computing platforms, it is necessary to first determine the specific calling tasks, which involves screening the tasks in the waiting task lists of multiple computing platforms. For example, the constraints can be determined based on the computing resource occupancy of the computing platform, the task priority, the task calculation time, and the task data migration time, and the tasks that meet the constraints are used as target tasks.

[0049] It is understandable that the higher the computing resource occupancy rate of the computing platform, the heavier the resource load of the computing platform; the higher the task priority, the higher the urgency of the task; the shorter the task calculation time and the shorter the task data migration time, the easier it is to reschedule the task. Based on this, by considering the above factors to select the target tasks, we can comprehensively consider the needs of each computing platform and users in the system, so as to ensure the rationality of task rescheduling.

[0050] S530, inserting the target task into the target position of the waiting task list of the target computing platform to execute the target task, wherein the target position is obtained by parsing the resource idle information.

[0051] In S530, after the target task is retrieved, the target task can be filled back into the waiting task list of the target computing platform to complete the task rescheduling. Specifically, the specific time when the target computing platform resources are idle can be obtained by parsing the resource idle message. After obtaining the specific time when the resources are idle, the retrieved target task can be filled back into the waiting task list position indicated by the specific time. In an example, Figure 6 FIG. 1 is a schematic diagram of a task scheduling process of a scheduling method for a collaborative computing platform according to an embodiment of the present invention. Figure 6 As shown, P n Platform t 9 Tasks are inserted into P 1 The target position of the platform.

[0052] In this way, by inserting the target task into the target position in the waiting task list of the target computing platform, the idle computing resources of the target computing platform are rationally utilized, while the computing load of the initial computing platform is reduced and the global resource utilization of the system is improved.

[0053] S540: Based on a preset time interval, respectively calculate the data access heat and data storage cost of the task data in the plurality of computing platforms.

[0054] In S540, task data is data corresponding to the task. The type of task data is not limited here. Task data consists of original data and several data copies. When one computing platform stores task data, other computing platforms store data copies of the task data. Among them, data access heat can be used to characterize the number of times task data is accessed and the time interval between accesses.

[0055] It is understandable that in the process of scheduling the target task from the initial computing platform to the target computing platform, the task data of the target task also needs to be transmitted to the target computing platform, and the transmission of the task data takes time. Based on this, it can be thought of storing the task data in the target computing platform first, but this requires a certain data storage cost. Based on this, the data access heat and data storage cost can be calculated at preset time intervals, and then the storage method of the task data in multiple computing platforms can be weighed according to the data access heat and data storage cost. In this way, the task data can be more reasonably arranged in multiple computing platforms.

[0056] S550, adjusting the storage location of the task data on multiple computing platforms according to the data access popularity and data storage cost of the task data.

[0057] In S550, the storage location of task data on multiple computing platforms can be dynamically adjusted so that during the task rescheduling process, the task data copy on the computing platform can be directly used for calculation without waiting for the task data to be transmitted between computing platforms, thereby improving the system throughput efficiency.

[0058] In one example, Figure 7 FIG. 1 is a schematic diagram of a data scheduling process of a scheduling method for a collaborative computing platform according to an embodiment of the present invention. Figure 7 As shown, initially DC1, DC2, DC3, and DC4 include 4, 3, 2, and 2 task data respectively. After adjusting the storage locations of the task data on multiple computing platforms, DC1, DC2, DC3, and DC4 include 3, 2, 1, and 2 task data respectively.

[0059] In an embodiment of the present application, by screening and retrieving tasks from multiple computing platforms and backfilling them into the target computing platform when there are idle computing resources on the target computing platform, the idle computing resources of the target computing platform are utilized, the load on the remaining computing platforms is reduced, and the global resource utilization of the system is improved. Furthermore, the present application can shorten the time required for task scheduling by adjusting the storage location of task data on multiple computing platforms based on preset time intervals, thereby improving the throughput of the system.

[0060] In some embodiments, screening target tasks that meet the constraint conditions from waiting task lists of multiple computing platforms includes:

[0061] Traversing waiting task lists of multiple computing platforms, adding tasks having task data copies stored in the target computing platform to the empty task set, to obtain a first task set;

[0062] If there are tasks in the first task set, traverse the first task set and determine the tasks in the first task set that meet the constraint conditions as target tasks;

[0063] When there is no task in the first task set or there is no task satisfying the constraint condition in the first task set, waiting task lists of multiple computing platforms are traversed to determine tasks satisfying the constraint condition in the waiting task lists of multiple computing platforms as target tasks.

[0064] In this embodiment, the waiting task lists of multiple computing platforms can be determined as the total available task set T p,all To improve the efficiency of traversal and find suitable available tasks in a shorter time, a greedy selection algorithm can be used to select the appropriate available tasks in the set of available tasks T. p,all Find the first task set in .

[0065] If T p,all If a task has a task data copy stored in the target computing platform, the task can be executed immediately after being retrieved and backfilled without going through the data migration process and queue waiting process. Therefore, the retrieval task set T p,all All such tasks in the first task set T p,data , so, in the task set T p,data When the tasks in are selected as the call and backfill targets, the system execution efficiency can be effectively improved.

[0066] In this embodiment, the available task set T p,all The other tasks in are set as task set T p,comp , that is, T p,comp =T p,all -T p,data . Task set T p,comp Although the requirements of all tasks in can be met by computing platform p, since there is no data copy required by the task in computing platform p, such tasks need to go through a data migration phase after being called and backfilled, at the cost of a certain amount of data migration overhead, to optimize task allocation and improve system throughput.

[0067] It is understandable that the task set T can be called p,all Define the task selection set T in p,select , to narrow the search domain. First, let T p,select =T p,data , task set T p,data After being retrieved and backfilled, the tasks in can be executed immediately by skipping the data migration process and queue waiting process. Therefore, the task set T is preferred. p,data As the calling target, the task in can effectively reduce the waiting time of the task and optimize the resource utilization of the computing platform p. p,data When it is an empty set or all the tasks in it do not meet the call constraints, the task selection set T p,select Redefine as T p,select =T p,comp .

[0068] In this way, by giving priority to tasks with task data copies stored in the target computing platform as target tasks, the data migration process and queue waiting process can be skipped, thereby shortening the computing waiting time of the task. At the same time, using a greedy algorithm to filter the task list can improve the search efficiency.

[0069] In some embodiments, the constraint condition includes an adaptability parameter, which is used to characterize the calling priority of the task. Before screening the target task that meets the constraint condition from the waiting task lists of the multiple computing platforms, the method further includes:

[0070] Obtain computing resource occupancy rates of multiple computing platforms, where the computing resource occupancy rates are used to characterize the idleness of computing resources of the computing platforms;

[0071] Obtain priority data corresponding to all tasks in waiting task lists of multiple computing platforms;

[0072] For any task in the waiting task lists of multiple computing platforms, the adaptability parameter of the task is calculated according to the priority data corresponding to the task and the computing resource occupancy rate of the computing platform corresponding to the task.

[0073] In this embodiment, the priority data corresponding to the task can adopt different data according to the needs of the user. In one example, if the user adopts a first-in-first-out strategy, the priority data can be based on the submission time of the task. If the user adopts a small task priority strategy, the priority data can be based on the task completion time.

[0074] In this embodiment, for each task t∈T p,select , set its adaptability parameter F(t) as shown in formula (1). The task with the larger adaptability parameter F(t) is selected as the priority target.

[0076] Calculate platform p s It is understandable that when the initial computing platform p s The greater the computing resource occupancy rate of the computing platform, the smaller the adaptability parameter, and the lower the possibility of tasks being called in the computing platform, so that the load between the computing platforms can be balanced. In addition, when it is necessary to select tasks from the task set T p,comp When selecting tasks, the calculation of adaptability parameters needs to include the data migration time T d,p,tran , that is, the time required for the task data d of task t to be migrated to the target computing platform p. It can be understood that when the data migration overhead is larger, the smaller the adaptability parameter is, the lower the possibility of the task being called, thereby avoiding the generation of large additional data migration overhead.

[0077] In this way, by calculating the adaptability parameters and screening the target tasks according to the adaptability parameters as constraint conditions, it can be ensured that the screened target tasks meet the user's priority strategy and better meet the user's needs.

[0078] In some embodiments, the constraints include: the target task is the task with the largest adaptability parameter in the target set, and the sum of the migration time of the target task's task data to the target computing platform and the computing time of the target task on the target computing platform is less than the expected waiting time of any task in the real-time task list of the target computing platform, and the migration time of the target task's task data to the target computing platform is less than the expected waiting time of the target task on the initial computing platform.

[0079] In this embodiment, constraints such as those shown in equations (2), (3), and (4) can be set during the search process for the target task:

[0080]

[0081] Among them, formula (3) can ensure that the task is rescheduled without increasing the waiting time of the queued tasks on the target computing platform, that is, the called target task t 0 Computation time on the target computing platform p With its task data d 0 Data migration time to the target computing platform p The sum is less than any task t in the real-time task list of the target computing platform p p Expected waiting time Formula (4) ensures that the start execution time of the called target task will not be delayed, that is, the target task t 0 Data migration time after rescheduling Less than the target task t 0 On the initial calculation platform p s Queue waiting time Due to T p,data The data required by all tasks in the target computing platform has a copy of the data. Therefore, formula (4) is mainly for T p,comp in the tasks.

[0082] In some embodiments, the data access heat includes global access heat, which is used to characterize the overall access heat of the task data on multiple computing platforms. According to the data access heat and data storage cost of the task data, the storage location of the task data on multiple computing platforms is adjusted, including:

[0083] Based on the global access popularity and data storage cost, the storage evaluation indicators of multiple task data in multiple computing platforms are calculated respectively;

[0084] For storage evaluation indicators of multiple task data in multiple computing platforms, when the storage evaluation indicators are not within a first preset threshold range, storage locations of the task data corresponding to the storage evaluation indicators in the multiple computing platforms are adjusted.

[0085] In this embodiment, upper and lower thresholds (Score l , Score u ), that is, the first preset threshold range. For data exceeding the first preset threshold range, the storage locations of the task data corresponding to the storage evaluation index on multiple computing platforms can be adjusted. At the same time, in order to ensure the accuracy of the storage evaluation index threshold, the replica threshold can be adaptively adjusted during the execution process.

[0086] After setting the first preset threshold range, all task data on multiple computing platforms can be sorted from large to small according to the storage evaluation index, and the task data with a storage evaluation index less than Score can be selected. l The task data or storage evaluation index is greater than Score u Redistribute the task data.

[0087] In this way, by using the first preset threshold range to screen the storage evaluation index, the adjustment range is limited, and excessive system overhead caused by adjusting too much task data is avoided.

[0088] In some embodiments, for storage evaluation indicators of multiple task data in multiple computing platforms, when the storage evaluation indicators are not within a first preset threshold range, adjusting the storage locations of the task data corresponding to the storage evaluation indicators in the multiple computing platforms includes:

[0089] Determining whether the storage resource occupancy rate of the plurality of computing platforms is greater than or equal to a second preset threshold;

[0090] When the storage resource occupancy rate is greater than or equal to the second preset threshold, for task data whose storage evaluation index is less than the first preset threshold, reduce R task data copies on multiple computing platforms, where R is a positive integer.

[0091] In this embodiment, to ensure the normal use of storage resources of multiple computing platforms, it is possible to determine whether the storage resource occupancy rate of multiple computing platforms is greater than or equal to a second preset threshold, where the second preset threshold is set according to user needs and is not limited.

[0092] When the storage resource occupancy rate is greater than or equal to the second preset threshold, it indicates that the storage resource occupancy rates of multiple computing platforms are too high. At this time, data redistribution may be performed only on the task data whose storage evaluation index is less than the first preset threshold, and no data redistribution may be performed on the task data whose storage evaluation index is greater than or equal to the first preset threshold.

[0093] When the storage resource occupancy rate is less than the second preset threshold, it indicates that the storage resource occupancy rates of multiple computing platforms are low. At this time, data redistribution can be performed on task data whose storage evaluation index is less than the first preset threshold, or data redistribution can be performed on task data whose storage evaluation index is greater than or equal to the first preset threshold.

[0094] In this way, by not redistributing task data whose storage evaluation index is greater than or equal to the first preset threshold when the system's storage resource occupancy is greater than or equal to the second preset threshold, the system can avoid adding data copies, thereby avoiding excessive load on the system's storage resources.

[0095] In some embodiments, the data access heat further includes local access heat, which is used to characterize the access heat of the task data on a single computing platform. For the storage evaluation index of multiple task data in multiple computing platforms, when the storage evaluation index is not within the first preset threshold range, the storage location of the task data corresponding to the storage evaluation index on the multiple computing platforms is adjusted, including:

[0096] According to the local access popularity and data storage cost of multiple task data, the ranking scores of multiple computing platforms are calculated based on the weighted average algorithm;

[0097] For multiple task data in multiple computing platforms, task data copies are stored on designated computing platforms according to the sorting scores of the multiple computing platforms, where the designated computing platforms are the first M computing platforms sorted from high to low according to the sorting scores, and M is a positive integer.

[0098] In this embodiment, after storing the task data copy on the designated computing platform according to the ranking scores of multiple computing platforms, the copy storage status of the task data can be checked according to user needs, and the computing platform designated by the user as the copy location but not in the copy plan can be replaced to a lower ranked position.

[0099] In this way, the target computing platform for storing the task data copy is determined by the ranking scores of multiple computing platforms calculated based on the weighted average algorithm, and the target computing platform for storage can be adjusted according to user needs, thereby improving the accuracy of task data copy storage.

[0100] In some embodiments, storage evaluation indicators of multiple task data in multiple computing platforms are calculated respectively according to global access popularity and data storage cost, including:

[0101]

[0102] Among them, Score d is the storage evaluation index of task data d, H dis the global access popularity of task data d, C d is the data storage cost of task data d.

[0103] In this embodiment, the storage evaluation index Score of each task data in multiple computing platforms can be calculated based on the global access popularity and data storage cost of the data. d ,

[0104] The storage evaluation index can be a value in the range of (0,1). For data with low data storage cost and high global access popularity, the storage evaluation index is high, and it is more inclined to add new copies of the data in multiple computing platforms. Conversely, when the storage evaluation index is low, it means that the data storage cost is high and the global access popularity is low, and it is more inclined to reduce the number of copies of the data in the system.

[0105] C d is the data storage cost of task data d, which mainly includes the data copy storage cost of task data d. For example, the data copy cost of task data d mainly includes the storage cost C of the copy d,stor , Data migration cost when creating a copy C d,tran and the cost of maintaining consistency between multiple copies C d,cons .

[0106] The data copy storage cost of task data d can be calculated using formula (6):

[0107] C d =C d,stor +C d,tran +C d,cons (6)

[0108] Since the consideration of replica cost is mainly from the perspective of storage and network resource occupation, each cost in formula (6) can be summarized as the product of occupied resource amount and occupied time, which is called resource occupation cost. For example, in a system composed of multiple computing platforms, the amount of data d is S d , there are n copies of the data in the system, and the xth copy d x Existing time is

[0109] It is understandable that the storage cost of the replica, C d,stor The main reason is that multiple redundant copies of data occupy storage resources, which may affect the layout of other data and thus affect system performance. Therefore, too many copies will cause performance degradation. Copy storage cost C d,stor It can be calculated using formula (7):

[0110]

[0111] Data migration cost when creating a replica C d,tran It is a one-time operation, which is generated only by data migration during the creation of replicas. For data that does not have multiple replicas in the system, the data adaptive layout strategy will be triggered after the data is migrated, and it will be determined whether to generate a persistent copy of the data on the computing platform that initiated the access based on the access popularity. Since the data migration has been completed at this time, triggering the data adaptive layout strategy can avoid additional data migration costs while obtaining the benefits of replicas. Data migration cost C d,tran It can be calculated according to formula (8):

[0112]

[0113] In formula (8) Indicates data copy d x The data migration time required to migrate to the target computing platform p is the data volume S d and on the initial computing platform p s Data migration between the target computing platform p occupies WAN bandwidth Formula (8) can express the cost of network resources occupied by data migration from the second to the nth copy in the system. It can be understood that, during the data migration process, the data migration overhead of the local area network in the computing platform is very small compared to the data migration overhead of the wide area network. Therefore, the calculation of data migration time in formula (8) can only consider the initial computing platform p s The time to migrate to the target computing platform p.

[0114] Cost of consistency maintenance C d,cons The main source of this cost is the network resource cost of synchronizing multiple copies. The commonly used broadcast protocol and master-slave replica mechanism for consistency maintenance in storage systems can be used, that is, after a piece of data is changed, the data is broadcast to all its copies and synchronizes the data changes. The data adaptive layout strategy assumes that the data in the system will be synchronized once every τ time with a data volume of ΔS. d If there is a change, then the sum of the costs of maintaining consistency of all data copies during their existence is calculated based on the existence time of the existing copies in the system. The calculation of the consistency maintenance cost C is consistent with the calculation of the data migration cost when establishing a copy. d,cons When we only consider the data migration within the WAN, we can calculate it according to formula (9):

[0115]

[0116] C d is the data storage cost of task data d. The copy cost of data d can be C d Perform min-max normalization.max and C min Express the maximum and minimum copy cost values ​​of all data in the system. d It represents the level of data d copy cost in all data copy costs and can be calculated according to formula (10):

[0117]

[0118] In this way, the storage evaluation index of the task data is calculated according to the global access popularity and the data storage cost, and the layout of the task data can be adjusted according to the storage evaluation index.

[0119] In some embodiments, based on a preset time interval, respectively calculating data access heat and data storage cost of multiple task data in multiple computing platforms includes:

[0120] Get the access times and access time intervals of multiple task data;

[0121] The data access heat of the multiple task data is calculated according to the access times and access time intervals of the multiple task data.

[0122] In this embodiment, in order to calculate the data access heat of the task data, the number of accesses and the time interval of accesses of the task data can be obtained first, wherein the data access heat can include global access heat and local access heat. For example, the data access heat of data d can be divided into global data access heat H d and the local data access heat h in computing platform p d,p , the global data access heat can be calculated according to formula (11), and the local data access heat can be calculated according to formula (12):

[0123]

[0124] In equations (11) and (12), the global data access ratio F d is the ratio of the number of accesses to data d to the number of accesses to all global data, f d,p It is the ratio of the number of accesses to data d in platform p to the number of global accesses to the data. d and Δt d,p They represent the last global access time interval and the last local access time interval of data d, respectively.

[0125] In this embodiment, when calculating the data access heat, the access time interval may be normalized to the min-max value so that the data access heat can be uniformly measured. min,dIndicates the minimum time interval for global access to data d, Δt max,d Indicates the maximum time interval for global access of data d, Δt min,d,p It represents the minimum time interval of local access to data d on computing platform p, Δt max,d,p Indicates the maximum time interval of local access of data d on computing platform p. According to formula (11) and formula (12), it can be understood that when the time interval of data access is smaller and the number of accesses is greater, the data access heat of the data is greater.

[0126] In some embodiments, in order to avoid large-scale adjustment of the storage location of task data on multiple computing platforms, which causes excessive network load, only small-scale adjustments can be made to the task data, and then the data layout of multiple computing platforms can be optimized by continuous polling. For example, a γ increase / decrease scheme can be adopted, that is, for a system with N computing platforms, data d has n copies laid out in the system, and the number of copies added or reduced in the data redistribution phase is γ, as shown in formula (13):

[0127]

[0128] Figure 6 FIG. 1 is a schematic diagram of a task scheduling process of a scheduling method for a collaborative computing platform according to an embodiment of the present invention. Figure 6 As shown, it includes: S610, receiving computing resource idle information feedback of the target computing platform; S620, obtaining system resource load conditions and task data distribution characteristics; S630, selecting a set of callable tasks from the task waiting list of multiple computing platforms; S640, determining the target task to be called from the set of callable tasks based on heuristic task selection conditions; S650, calling the target task from the initial computing platform to the target computing platform.

[0129] Figure 7 FIG. 1 is a schematic diagram of a data scheduling process of a scheduling method for a collaborative computing platform according to an embodiment of the present invention. Figure 7 As shown, it includes: S710, obtaining data access heat and data storage cost; S720, integrating data access heat and data storage cost into a data trade-off model; S730, adjusting the number and location of data copies of multiple computing platforms according to the output results of the trade-off model and the storage resource occupancy rate.

[0130] Figure 8 is a schematic diagram of the overall process of a scheduling method for a collaborative computing platform according to an embodiment of the present invention; Figure 8As shown, it includes: S810, the system allocates tasks to multiple computing platforms; S820, a wide-area task retrieval and backfill strategy based on location perception, when the computing resource idle information of the target computing platform is received through the scheduling system, the task is selected among the waiting task queues of multiple computing platforms according to whether the computing resource idleness of the target computing platform meets the task requirements, the distribution characteristics of the task data, the task priority, the resource load of the computing platform, the data transmission of the task data, etc., to obtain the target task; and the target task is retrieved from the initial computing platform and backfilled to the target computing platform to optimize the task allocation; S830, the system performs data layout on the task data of multiple computing platforms; S840, A data adaptive layout strategy based on heat feedback and cost perception is proposed. First, the data to be laid out is selected. Specifically, the data access heat can be calculated based on the number of data accesses and the frequency of data access. Then, the data storage cost is calculated based on the number of data copies and the size of the data. Finally, a storage evaluation index is generated based on the trade-off between data access heat and data storage cost. After the storage evaluation index is generated, the data can be redistributed on multiple computing platforms based on the storage evaluation index. Specifically, the number of data copies in multiple computing platforms can be adjusted based on the storage evaluation index, and the copy location can be adjusted based on the global computing resource status and the local access heat of the data, thereby optimizing the data layout. The proposed method optimizes the global utilization of resources by collaboratively and dynamically adjusting task allocation and data layout during the system execution process, thereby improving the system throughput.

[0131] The entire scheduling technology can be divided into two parts: task scheduling and data layout. When idle resources appear in the system, task scheduling will be triggered, and task backfilling operations will be performed based on the data layout in the current system to optimize task execution. At the same time, the system will trigger data scheduling at a regular interval and perform data redistribution based on task access popularity to optimize data access.

[0132] By sharing system resource information (data access / data distribution and other information), task scheduling and data scheduling are used together. The task scheduling results will affect data access and thus data scheduling. The data scheduling results will affect data layout and thus task scheduling. By collaboratively scheduling tasks and data, the task execution and data access efficiency are jointly optimized to improve system performance. The specific algorithm execution process is as follows.

[0133] Fig. 9 A schematic diagram of the structure of a scheduling device for a collaborative computing platform provided in another embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0134] Reference Fig. 9 , the scheduling device 900 of the collaborative computing platform may include:

[0135] The acquisition module 901 is used to acquire the waiting task lists of multiple computing platforms connected to the scheduling system when the scheduling system receives the computing resource idle information sent by the target computing platform;

[0136] A screening module 902 is used to screen target tasks that meet constraint conditions from waiting task lists of multiple computing platforms, wherein the constraint conditions are determined according to the computing resource occupancy rate, task priority, task computing time, and task data migration time of the computing platform;

[0137] Insertion module 903, used for inserting the target task into the target position of the waiting task list of the target computing platform to execute the target task, wherein the target position is obtained by parsing the resource idle information;

[0138] A calculation module 904, used to calculate the data access heat and data storage cost of the task data in multiple computing platforms respectively based on a preset time interval;

[0139] The adjustment module 905 is used to adjust the storage location of the task data on multiple computing platforms according to the data access popularity and data storage cost of the task data.

[0140] In some embodiments, the screening module 902 may include the following units:

[0141] A storage unit, used for traversing waiting task lists of multiple computing platforms, adding tasks having task data copies stored in the target computing platform to an empty task set, and obtaining a first task set;

[0142] A determination unit, configured to, if there are tasks in the first task set, traverse the first task set and determine the tasks in the first task set that meet the constraint conditions as target tasks;

[0143] The determination unit is also used to traverse the waiting task lists of multiple computing platforms when there is no task in the first task set or there is no task that meets the constraint conditions in the first task set, and determine that the task that meets the constraint conditions in the waiting task lists of multiple computing platforms is the target task.

[0144] In some embodiments, the constraint condition includes an adaptability parameter, which is used to characterize the calling priority of the task. Before screening the target task that meets the constraint condition from the waiting task list of multiple computing platforms, the scheduling device 900 of the collaborative computing platform may further include:

[0145] The acquisition module is also used to obtain the computing resource occupancy rate of multiple computing platforms, and the computing resource occupancy rate is used to characterize the computing resource idleness of the computing platform;

[0146] The acquisition module is also used to obtain priority data corresponding to all tasks in the waiting task lists of multiple computing platforms;

[0147] The calculation module is also used to calculate the adaptability parameter of any task in the waiting task list of multiple computing platforms according to the priority data corresponding to the task and the computing resource occupancy rate of the computing platform corresponding to the task.

[0148] In some embodiments, the data access heat includes global access heat, which is used to characterize the overall access heat of task data on multiple computing platforms. The adjustment module 905 may include the following units:

[0149] A computing unit, used to calculate storage evaluation indicators of multiple task data in multiple computing platforms according to global access popularity and data storage cost;

[0150] The adjustment unit is used to adjust the storage location of the task data corresponding to the storage evaluation index in the multiple computing platforms when the storage evaluation index is not within a first preset threshold range.

[0151] In some embodiments, the adjustment unit may include the following subunits:

[0152] A determination subunit, used to determine whether the storage resource occupancy rate of the plurality of computing platforms is greater than or equal to a second preset threshold;

[0153] The adjustment subunit is used to reduce R task data copies on multiple computing platforms for task data whose storage evaluation index is less than a first preset threshold when the storage resource occupancy rate is greater than or equal to a second preset threshold, where R is a positive integer.

[0154] In some embodiments, the data access heat also includes local access heat, which is used to characterize the access heat of the task data on a single computing platform. The above adjustment unit may include the following subunits:

[0155] A computing subunit, used to calculate the ranking scores of multiple computing platforms based on a weighted average algorithm according to the local access popularity and data storage cost of multiple task data;

[0156] The storage subunit is used to store copies of the task data in multiple computing platforms on a designated computing platform according to the sorting scores of the multiple computing platforms, wherein the designated computing platform is the first M computing platforms sorted from high to low according to the sorting scores, and M is a positive integer.

[0157] In some embodiments, the above-mentioned computing unit may be specifically used for:

[0158]

[0159] Among them, Score d is the storage evaluation index of task data d, H d is the global access popularity of task data d, C d is the data storage cost of task data d.

[0160] In some embodiments, the calculation module 904 may include the following units:

[0161] The acquisition unit is further used to acquire the number of accesses and the access time intervals of the plurality of task data;

[0162] The calculation unit is further used to calculate the data access heat of the multiple task data according to the number of times the multiple task data are accessed and the access time interval.

[0163] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application, and are devices corresponding to the scheduling method of the above-mentioned collaborative computing platform. All implementation methods in the above-mentioned method embodiment are applicable to the embodiment of the device. Its specific functions and technical effects can be found in the method embodiment part, which will not be repeated here.

[0164] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0165] Fig.10 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Fig.10 As shown:

[0166] The device may include a processor 1001 and a memory 1002 storing programs or instructions.

[0167] When the processor 1001 executes the program, the steps in any of the above method embodiments are implemented.

[0168] Exemplarily, the program may be divided into one or more modules / units, one or more modules / units are stored in the memory 1002, and executed by the processor 1001 to complete the present application. One or more modules / units may be a series of program instruction segments capable of completing a specific function, and the instruction segments are used to describe the execution process of the program in the device.

[0169] Specifically, the processor 1001 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0170] The memory 1002 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 1002 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 1002 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 1002 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 1002 is a non-volatile solid-state memory.

[0171] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) machine-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0172] The processor 1001 implements any one of the methods in the above embodiments by reading and executing the program or instruction stored in the memory 1002 .

[0173] In one example, the electronic device may further include a communication interface 1003 and a bus 1004. The processor 1001, the memory 1002, and the communication interface 1003 are connected via the bus 1004 and communicate with each other.

[0174] The communication interface 1003 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0175] Bus 1004 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 1004 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0176] In addition, in combination with the method in the above embodiment, the embodiment of the present application may provide a machine-readable storage medium for implementation. The machine-readable storage medium stores a program or instruction; when the program or instruction is executed by a processor, any one of the methods in the above embodiment is implemented. The machine-readable storage medium can be read by a machine such as a computer.

[0177] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0178] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0179] An embodiment of the present application provides a computer program product, which is stored in a machine-readable storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0180] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0181] The functional modules shown in the above block diagram can be implemented as hardware, software, firmware or their combination. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), suitable firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer grids such as the Internet, intranets, etc.

[0182] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0183] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by a computer program or instruction. These programs or instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0184] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A scheduling method for a collaborative computing platform, characterized in that: The method comprises: Upon receiving computing resource idle information sent by the target computing platform, obtaining waiting task lists of multiple computing platforms; Filtering target tasks that meet the constraint conditions from the waiting task lists of the multiple computing platforms, wherein the constraint conditions are determined according to the computing resource occupancy rate, task priority, task computing time, and task data migration time of the computing platform; Inserting the target task into a target position of a waiting task list of the target computing platform to execute the target task, wherein the target position is obtained by parsing the resource idle information; Based on a preset time interval, respectively calculating the data access heat and the data storage cost of the task data in the multiple computing platforms; According to the data access popularity and data storage cost of the task data, the storage locations of the task data on the multiple computing platforms are adjusted.

2. The method according to claim 1, characterized in that: The step of selecting target tasks that meet the constraint conditions from the waiting task lists of the plurality of computing platforms includes: Traversing the waiting task lists of the multiple computing platforms, adding the tasks having task data copies stored in the target computing platform to the empty task set, to obtain a first task set; If there are tasks in the first task set, traverse the first task set and determine the task in the first task set that meets the constraint condition as the target task; When there is no task in the first task set or there is no task satisfying the constraint condition in the first task set, the waiting task lists of the multiple computing platforms are traversed to determine that the task satisfying the constraint condition in the waiting task lists of the multiple computing platforms is the target task.

3. The method according to claim 1, characterized in that The constraint condition includes the adaptability parameter, and the adaptability parameter is used to characterize the calling priority of the task. Before screening the target task that meets the constraint condition from the waiting task list of the multiple computing platforms, the method further includes: Obtaining computing resource occupancy rates of the multiple computing platforms, where the computing resource occupancy rates are used to characterize the computing resource idleness of the computing platforms; Obtaining priority data corresponding to all tasks in waiting task lists of the multiple computing platforms; For any task in the waiting task list of the plurality of computing platforms, the adaptability parameter of the task is calculated according to the priority data corresponding to the task and the computing resource occupancy rate of the computing platform corresponding to the task.

4. The method according to claim 2, characterized in that: The constraints include: the target task is the task with the largest adaptability parameter in the target set, and the sum of the migration time of the target task's task data to the target computing platform and the computing time of the target task on the target computing platform is less than the expected waiting time of any task in the real-time task list of the target computing platform, and the migration time of the target task's task data to the target computing platform is less than the expected waiting time of the target task on the initial computing platform.

5. The method according to claim 1, characterized in that The data access heat includes global access heat, which is used to characterize the overall access heat of the task data on the multiple computing platforms. The step of adjusting the storage location of the task data on the multiple computing platforms according to the data access heat and the data storage cost of the task data includes: Calculating storage evaluation indicators of multiple task data in the multiple computing platforms respectively according to the global access popularity and the data storage cost; With respect to storage evaluation indicators of multiple task data in the multiple computing platforms, when the storage evaluation indicators are not within a first preset threshold range, storage locations of the task data corresponding to the storage evaluation indicators in the multiple computing platforms are adjusted.

6. The method according to claim 5, characterized in that The storage evaluation indicators for the plurality of task data in the plurality of computing platforms, when the storage evaluation indicators are not within a first preset threshold range, adjusting the storage locations of the task data corresponding to the storage evaluation indicators in the plurality of computing platforms, comprises: Determining whether the storage resource occupancy rate of the plurality of computing platforms is greater than or equal to a second preset threshold; When the storage resource occupancy rate is greater than or equal to the second preset threshold, for task data whose storage evaluation index is less than the first preset threshold, reduce R task data copies of the task data on the multiple computing platforms, where R is a positive integer.

7. The method according to claim 5, characterized in that The data access heat also includes local access heat, which is used to characterize the access heat of the task data on a single computing platform. The storage evaluation index for the multiple task data in the multiple computing platforms, when the storage evaluation index is not within a first preset threshold range, adjusts the storage location of the task data corresponding to the storage evaluation index on the multiple computing platforms, including: Calculate the ranking scores of the multiple computing platforms based on a weighted average algorithm according to the local access popularity and data storage cost of the multiple task data; For multiple task data in the multiple computing platforms, a copy of the task data is stored on a designated computing platform according to the sorting scores of the multiple computing platforms, wherein the designated computing platform is the first M computing platforms sorted from high to low according to the sorting scores, and M is a positive integer.

8. The method according to claim 5, characterized in that The calculating, according to the global access heat and the data storage cost, storage evaluation indicators of the plurality of task data in the plurality of computing platforms respectively includes: Among them, Score d is the storage evaluation index of task data d, H d is the global access popularity of task data d, C d is the data storage cost of task data d.

9. The method according to claim 1, characterized in that: The calculating, based on the preset time interval, respectively the data access heat and the data storage cost of the plurality of task data in the plurality of computing platforms comprises: Get the access times and access time intervals of multiple task data; The data access heat of the multiple task data is calculated according to the access times and access time intervals of the multiple task data.

10. A scheduling device for a collaborative computing platform, characterized in that: The device comprises: An acquisition module, used to acquire waiting task lists of multiple computing platforms upon receiving computing resource idle information sent by a target computing platform; A screening module, used to screen target tasks that meet the constraint conditions from the waiting task lists of the multiple computing platforms, wherein the constraint conditions are determined according to the computing resource occupancy rate, task priority, task computing time, and task data migration time of the computing platform; An inserting module, used for inserting the target task into a target position of a waiting task list of the target computing platform to execute the target task, wherein the target position is obtained by parsing the resource idle information; A calculation module, used to calculate the data access heat and data storage cost of the task data in the multiple computing platforms respectively based on a preset time interval; The adjustment module is used to adjust the storage location of the task data on the multiple computing platforms according to the data access heat and data storage cost of the task data.

11. An electronic device, characterized in that: The device comprises: a processor and a memory storing a computer program or instruction; When the processor executes the computer program instructions, the method according to any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

13. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the method according to any one of claims 1 to 9.

Images