Cross-cluster operation method and device and storage medium

Abstract

The invention relates to the field of cluster data processing, and discloses a cross-cluster operation method and device and a storage medium, the method comprises the following steps: establishing a data transmission channel between a current cluster and a target cluster, and sending target data corresponding to a target operation in the current cluster to the target cluster through the data transmission channel; obtaining job parameters corresponding to the target job in the current cluster, determining a target script according to the job parameters, the cluster identifier of the target cluster and a pre-constructed script template library, and sending the target script to the target cluster, so that the target cluster executes the target job based on the target script and the target data to obtain a target result; and in response to determining that the target cluster completes the target job, establishing a result transmission channel between the current cluster and the target cluster, and sending the target result in the target cluster to the current cluster through the result transmission channel, thereby improving the data transmission efficiency and security, adaptively generating a script, and reducing the technical difficulty of cross-cluster job.

Description

Technical Field

[0001] The present invention relates to the technical field of cluster data processing, and in particular to a cross-cluster operation method, device and storage medium. Background Art

[0002] With the rapid development of technologies such as cloud computing, big data, and artificial intelligence, the demand for computing power has shown explosive growth. However, due to the limitations of various factors such as geographical location, resource distribution, and network conditions, computing power resources are often unable to efficiently and flexibly meet user needs.

[0003] Therefore, the research and application of cross-cluster computing power scheduling technology is of great significance for improving computing power resource utilization, optimizing resource allocation, and reducing energy consumption. However, the current cross-cluster scheduling technology has problems such as slow data transmission, poor security, and difficulty in building scheduling algorithms.

[0004] In view of this, the present invention is proposed. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a cross-cluster operation method, device and storage medium, which realize the cross-cluster transmission of operation data, the automatic submission and operation of jobs across clusters, and the return of results.

[0006] An embodiment of the present invention provides a cross-cluster operation method, the method comprising:

[0007] Establishing a data transmission channel between the current cluster and the target cluster, and sending the target data corresponding to the target job in the current cluster to the target cluster through the data transmission channel;

[0008] Obtaining job parameters corresponding to the target job in the current cluster, determining a target script according to the job parameters, a cluster identifier of the target cluster, and a pre-built script template library, and sending the target script to the target cluster, so that the target cluster executes the target job based on the target script and the target data to obtain a target result corresponding to the target job;

[0009] In response to determining that the target cluster has completed the target job, a result transmission channel is established between the current cluster and the target cluster, and the target result in the target cluster is sent to the current cluster through the result transmission channel.

[0010] An embodiment of the present invention provides an electronic device, the electronic device comprising:

[0011] Processor and memory;

[0012] The processor is used to execute the steps of the cross-cluster operation method described in any embodiment by calling the program or instruction stored in the memory.

[0013] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program or instruction, wherein the program or instruction enables a computer to execute the steps of the cross-cluster operation method described in any embodiment.

[0014] The embodiments of the present invention have the following technical effects:

[0015] A data transmission channel is established between the current cluster and the target cluster, and the target data corresponding to the target job in the current cluster is sent to the target cluster through the data transmission channel, so as to establish a separate data transmission channel for transmitting the target data, thereby ensuring the security of the data. Then, the job parameters corresponding to the target job in the current cluster are obtained, and the target script is determined according to the job parameters, the cluster identifier of the target cluster and the pre-built script template library, and the target script is sent to the target cluster, so that the target cluster executes the target job based on the target script and the target data to obtain the target result corresponding to the target job. The script generation steps are simplified by adapting to different systems. In response to determining that the target cluster has completed the target job, a result transmission channel is established between the current cluster and the target cluster, and the target result in the target cluster is sent to the current cluster through the result transmission channel, thereby realizing the cross-cluster transmission of job data, automatic submission and operation of jobs across clusters and result transmission, thereby improving the efficiency and security of data transmission, and being able to adaptively generate corresponding scripts to reduce the technical difficulty of cross-cluster jobs. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a flow chart of a cross-cluster operation method provided by an embodiment of the present invention;

[0018] Figure 2 It is a schematic diagram of a tree structure in which clusters are mapped according to physical locations, provided by an embodiment of the present invention;

[0019] Figure 3 is a flow chart of determining a target cluster corresponding to a current cluster provided by an embodiment of the present invention;

[0020] Figure 4It is a sequence diagram of establishing a data transmission channel and transmitting target data provided by an embodiment of the present invention;

[0021] Figure 5 is a flow chart of determining a target transmission channel provided by an embodiment of the present invention;

[0022] Figure 6 It is a flow chart of generating a target script based on a template provided by an embodiment of the present invention;

[0023] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0025] The cross-cluster operation method provided by the embodiment of the present invention is mainly applicable to the case where the target cluster is used to execute the job and return the result when the resources of the current cluster cannot support the job execution. The cross-cluster operation method provided by the embodiment of the present invention can be executed by an electronic device.

[0026] With the rapid development of technologies such as cloud computing, big data, and artificial intelligence, the demand for computing power has shown explosive growth. However, due to the limitations of various factors such as geographical location, resource distribution, and network conditions, computing power resources are often unable to efficiently and flexibly meet user needs. Therefore, the research and application of cross-cluster computing power scheduling technology is of great significance for improving computing power resource utilization, optimizing resource allocation, and reducing energy consumption. At present, the main problems faced in implementing cross-domain task job scheduling are as follows:

[0027] Regarding network latency and bandwidth limitations, cross-cluster jobs require data transmission across geographical regions, and network latency and bandwidth limitations become key bottlenecks. When the input data and result data to be calculated are large, even if data is transmitted across regions through dedicated data lines, users cannot tolerate the time wasted due to data transmission. In addition, network problems may lead to inefficient data transmission, affecting the response time and completion efficiency of computing tasks.

[0028] The hardware and software environments of different regions and HPC (High Performance Computing) centers may differ, which will lead to heterogeneity of computing resources. This heterogeneity increases the complexity of cross-cluster scheduling, and it is necessary to ensure the compatibility and efficiency of jobs in different environments. In more cases, users are required to adapt the system and software themselves, which is very difficult and unfriendly for users.

[0029] Regarding scheduling algorithms and strategies, cross-cluster HPC job scheduling needs to consider more factors, such as task type, resource requirements, priority, deadline, etc., to ensure the rationality and fairness of the scheduling strategy. Therefore, it is necessary to design intelligent scheduling algorithms and strategies to cope with the complexity and uncertainty in cross-cluster scheduling, especially the differences in scheduling strategies and scheduling algorithms between different scheduling systems such as Slurm (Simple Linux Utility for Resource Management, a highly scalable and fault-tolerant cluster manager and job scheduling system that can be used for large clusters of computing nodes) and LSF (Load Sharing Facility, a distributed cluster management system). For example, a job running on a Slurm cluster may fail to run on LSF.

[0030] In the cross-cluster scheduling process, a large amount of sensitive data and computing results are transmitted and shared, and the security and privacy of the data need to be ensured. Therefore, effective security measures and technical means, such as data encryption, access control, and audit tracking, need to be adopted to ensure the security of data and computing processes. In particular, the data retention issue may have different protection levels on different clusters.

[0031] In addition, cross-cluster HPC job scheduling needs to balance multiple factors such as computing power, network, and cost to achieve optimal resource allocation and cost minimization.

[0032] Embodiment 1

[0033] Figure 1 is a flow chart of a cross-cluster operation method provided by an embodiment of the present invention. Figure 1 , the cross-cluster operation method specifically includes:

[0034] S110 , establishing a data transmission channel between the current cluster and the target cluster, and sending target data corresponding to the target job in the current cluster to the target cluster through the data transmission channel.

[0035] The current cluster is the cluster selected by the user to execute the target job. The target cluster is the cluster that replaces the current cluster to execute the target job when the current cluster has insufficient resources to execute the target job. The target job is the job waiting to be completed. The data transmission channel is the data channel between the current cluster and the target cluster, which is used to transmit the target data. The target data is the data required to execute the target job, which is all the data identified and used by the target job.

[0036] Specifically, in order to ensure that the target job can be executed in the target cluster, it is necessary to obtain the target data corresponding to the target job from the current cluster. Therefore, a data transmission channel for transmitting the target data is established between the current cluster and the target cluster, so as to control the current cluster to transmit the target data to the target cluster through the data transmission channel, so as to provide a data basis for the target cluster to execute the target job later.

[0037] It is understandable that in response to the target data transmission completion, the data transmission channel is revoked to avoid the mistransmission of other irrelevant data. Specifically, when the target cluster receives the target data, the target data can be verified, and after determining that the target data is complete and correct, a signal of transmission completion is generated. If the verification fails after the target data is transmitted, the data can be cleared and the target data can be retransmitted based on the data transmission channel.

[0038] Based on the above example, before establishing the data transmission channel between the current cluster and the target cluster, since only the current cluster is known, the target cluster needs to be determined from clusters other than the current cluster. The specific method for determining the target cluster is as follows:

[0039] In response to the current cluster not satisfying the resource condition of the target job, determining a physical space corresponding to the current cluster, and judging whether there is a first cluster satisfying the resource condition in the physical space;

[0040] If so, determining a target cluster corresponding to the current cluster from each first cluster;

[0041] If not, determine the current domain corresponding to the current cluster and determine whether there is a second cluster that meets the resource conditions in the current domain;

[0042] If so, determining a target cluster corresponding to the current cluster from each second cluster;

[0043] If not, traverse and obtain each third cluster that meets the resource condition, and determine a target cluster corresponding to the current cluster from each third cluster according to the inter-domain distance between each third cluster and the current cluster.

[0044] Among them, resource conditions are resource-related conditions required for the execution of the target job, which are used to determine whether the resources in the cluster are sufficient to support the execution of the target job. The physical space can be a space such as a computer room. The first cluster is a cluster in the same physical space as the current cluster and meets the resource conditions of the target job. The current domain is a set of clusters with a security protection boundary. The second cluster is a cluster in the same domain as the current cluster and meets the resource conditions of the target job. The third cluster is a cluster in other domains outside the current domain to which the current cluster belongs and meets the resource conditions of the target job. The inter-domain distance is the distance between the domain where the current cluster is located and the domain where the third cluster is located.

[0045] Specifically, if the current cluster meets the resource conditions of the target job, the current cluster can be used directly to execute the target job without executing the solution of this example. If the current cluster does not meet the resource conditions of the target job, it is necessary to determine the target cluster corresponding to the current cluster, specifically: analyze the current cluster to determine the physical space corresponding to the current cluster, which can be pre-identified in the basic information of the cluster by identifying the physical space corresponding to each cluster. Further, it is determined whether there is a first cluster that meets the resource conditions in the physical space corresponding to the current cluster. If there is, it means that there is at least one first cluster that is physically close to the current cluster. Based on this, a target cluster corresponding to the current cluster can be determined from each first cluster. If the number of first clusters is one, the first cluster is directly used as the target cluster. If the number of first clusters is at least two, one can be randomly determined from each first cluster as the target cluster, or a first cluster with the closest physical distance to the current cluster can be used as the target cluster, or a first cluster with the most available resources among the first clusters can be used as the target cluster. If it does not exist, it means that the available resources of other clusters in the same physical space as the current cluster cannot meet the resource conditions of the target job. Further expand the search scope, determine the current domain corresponding to the current cluster, search in the current domain, and determine whether there is a second cluster that meets the resource conditions in the current domain. If it does exist, it means that there is at least one second cluster in the same domain as the current cluster. Based on this, a target cluster corresponding to the current cluster can be determined from each second cluster. If the number of second clusters is one, the second cluster is directly used as the target cluster. If the number of second clusters is at least two, one can be randomly determined from each second cluster as the target cluster. The second cluster with the closest physical distance to the current cluster can also be used as the target cluster, or the second cluster with the most available resources among the second clusters can be used as the target cluster. If it does not exist, it means that the available resources of other clusters in the same domain as the current cluster cannot meet the resource conditions of the target job. The search scope is further expanded. By traversing the clusters in other domains outside the current domain, the third clusters that meet the resource conditions are obtained, and the inter-domain distance between each third cluster and the current cluster is calculated respectively. At least one third cluster with the smallest inter-domain distance with the current cluster is determined. If the number of third clusters with the smallest inter-domain distance with the current cluster is one, the third cluster is directly used as the target cluster. If the number of third clusters with the smallest inter-domain distance with the current cluster is at least two, one can be randomly determined from these third clusters as the target cluster, or the one with the most available resources among these third clusters can be used as the target cluster.

[0046] The middle layer may include a cross-cluster job scheduling layer, which is used to link resources between different clusters, different software architectures, and different hardware architectures. Through automatic identification and perception algorithms, it determines the target cluster corresponding to the current cluster, so as to schedule the user's target job to the computing power resources (target cluster) suitable for computing. Figure 2 This is a diagram of the tree structure of each cluster mapped according to the physical location. Domain A and Domain B are two different domains, Computer Room-Module 2-2 and Computer Room-Module 3-1 under Domain B are two physical spaces, Cluster A and Cluster B belong to the same physical space (Computer Room-Module 2-1) and the same domain, and Cluster C and Cluster D belong to different physical spaces under the same domain. Figure 3 It is a flowchart for determining the target cluster corresponding to the current cluster.

[0047] It is understandable that when the target job submitted by the user cannot meet the running conditions (resource conditions, such as the number of available nodes, etc.) in the current cluster or when it is determined that queuing is required in the current cluster, the cross-cluster job process is triggered.

[0048] Based on the above example, you can establish a data transmission channel between the current cluster and the target cluster in the following ways:

[0049] Create a first microservice corresponding to the current cluster, and create a second microservice corresponding to the target cluster;

[0050] Based on the first microservice, perform verification processing with the current cluster, so that when the verification passes, the current cluster creates a third microservice corresponding to the first microservice;

[0051] In response to the current cluster verification being passed, a first transmission channel between the first microservice and the third microservice is established based on the first microservice, and a first encryption algorithm is determined based on the first transmission channel;

[0052] Based on the second microservice, perform verification processing with the target cluster, so that when the verification passes, the target cluster creates a fourth microservice corresponding to the second microservice;

[0053] In response to the target cluster being verified successfully, a second transmission channel between the second microservice and the fourth microservice is established based on the second microservice, and a second encryption algorithm is determined based on the second transmission channel;

[0054] The first transmission channel, the first encryption algorithm, the second transmission channel, and the second encryption algorithm are used as the data transmission channel between the current cluster and the target cluster.

[0055] Among them, the first microservice is a microservice related to the current cluster established in the current middle layer. The second microservice is a microservice related to the target cluster established in the current middle layer. The third microservice is a microservice related to the current middle layer established in the current cluster. The fourth microservice is a microservice related to the current middle layer established in the target cluster. The first transmission channel is the transmission channel between the first microservice and the third microservice, that is, the transmission channel between the current middle layer and the current cluster. The first encryption algorithm is the encryption algorithm used when transmitting data in the first transmission channel. The second transmission channel is the transmission channel between the second microservice and the fourth microservice, that is, the transmission channel between the current middle layer and the target cluster. The second encryption algorithm is the encryption algorithm used when transmitting data in the second transmission channel.

[0056] It can be understood that this solution is applied in the middle layer to connect the current cluster and the target cluster to facilitate cross-cluster processing.

[0057] Specifically, a first microservice corresponding to the current cluster is created in the middle layer, and a second microservice corresponding to the target cluster is created. Based on the first microservice, a verification process is performed with the current cluster, which may specifically be a token verification request issued to the data security access API (Application Programming Interface) of the current cluster, mainly for identity authentication and authorization. ‌‌ If the token verification in the current cluster passes, a third microservice corresponding to the first microservice is created in the current cluster. Further, a first transmission channel is established between the first microservice and the third microservice, and an encryption algorithm negotiation is performed between the middle layer and the current cluster through the data security access API of the current cluster to determine the first encryption algorithm used in the first transmission channel. Based on the second microservice, a verification process is performed with the target cluster, which may specifically be a token verification request issued to the data security access API of the target cluster, mainly for identity authentication and authorization. ‌‌ If the token verification in the target cluster passes, a fourth microservice corresponding to the second microservice is created in the target cluster. Further, a second transmission channel is established between the second microservice and the fourth microservice, and an encryption algorithm negotiation is performed between the middle layer and the target cluster through the data security access API of the target cluster to determine the second encryption algorithm used in the second transmission channel. The first transmission channel and the second transmission channel are combined into a data transmission channel between the current cluster and the target cluster, and the data transmitted in the first transmission channel is encrypted using a first encryption algorithm, and the data transmitted in the second transmission channel is encrypted using a second encryption algorithm.

[0058] The middle layer can include a secure data transmission layer, which is connected by a data security access API link between computing power clusters in different domains. The transmission tasks can be managed through the asynchronous service management system to automatically complete the transmission process without the user waiting. Data access is authorized by the user, services are established, tokens are delegated, and users are granted specified data access rights. The access rights are for job input data transmission, job calculation and / or job result feedback. During the transmission process, progress and speed can be obtained through standardized APIs.

[0059] Figure 4 It is a sequence diagram for establishing a data transmission channel and transmitting target data. This method can be used in both the target data transmission stage and the target result transmission stage. Specifically, it can be: dynamically create user data access microservices (first microservice, second microservice, third microservice and fourth microservice), and the data resource permissions are granted through the authorization service in each cluster (current cluster and target cluster). The user's security data is dynamically mounted through user authorization, and the token returned by the authorization service is used to build a secure access channel (first transmission channel and second transmission channel). After the data is copied, the user data access microservice is automatically closed, the data access token is automatically invalidated, and the data is automatically uninstalled. By building an encrypted data transmission channel (the first transmission channel corresponding to the first encryption algorithm and the second transmission channel corresponding to the second encryption algorithm) in a secure and trusted network service area trusted by the user, the token authorized by the user is used to access the API, and the token is valid for single-time single-file access. Optionally, the encrypted data transmission channel can be encrypted using RSA-256 (Rivest-Shamir-Adleman-256, an asymmetric encryption algorithm), and the encrypted key pair is dynamically established through mTLS (MutualTransport Layer Security, a two-way authentication method) through the API services at both ends of the data transmission.

[0060] Based on the above example, the target data corresponding to the target job in the current cluster can be sent to the target cluster through the data transmission channel in the following ways:

[0061] Determine the task type of the target data transmission task corresponding to the target job in the current cluster;

[0062] Determine the acceleration value based on the task type and the transfer size corresponding to the transfer task;

[0063] According to the task type and the acceleration value, the target transmission channel is determined, and the target job is sent to the target cluster based on the target transmission channel to complete the transmission task.

[0064] Among them, the transmission task is a task for transmitting target data. Task types include new tasks and continued transmission tasks. New tasks are tasks for which target data has not been transmitted before, and continued transmission tasks are tasks for which part of the target data has been transmitted before. The transmission size is the size of the data corresponding to the transmission task, that is, the size of the data to be transmitted. The acceleration value is a value used to measure the low-speed transmission channel and the high-speed transmission channel, which can be understood as the ratio of the low-speed channel transmission time to the high-speed channel transmission time. The target transmission channel is the transmission channel used to complete the transmission task later, which can be a low-speed transmission channel or a high-speed transmission channel.

[0065] Specifically, a transmission task of the target data corresponding to the target job is constructed, and the task type of the transmission task is determined, that is, whether it is a new task or a continued transmission task. Furthermore, according to different task types, the transmission size corresponding to the transmission task is calculated, and the acceleration value is determined to measure the transmission time ratio between the low-speed transmission channel and the high-speed transmission channel. For example: the new task has no historical transmission data and cannot be used for reference. The acceleration value is calculated using a universal calculation method. The continued transmission task has historical transmission data for reference. The acceleration value is calculated in combination with the historical transmission data. According to different task types, the thresholds corresponding to different task types can be determined, and the thresholds are compared with the acceleration values ​​to determine whether the low-speed transmission channel or the high-speed transmission channel is the target transmission channel. Furthermore, the target job can be sent to the target cluster through the determined target transmission channel to complete the transmission task.

[0066] Based on the above example, if the task type is a new task, the acceleration value can be determined according to the task type and the transfer size corresponding to the transfer task in the following way:

[0067] In response to the task type being a new task, the size of the target data is used as the transmission size corresponding to the transmission task, and the high-speed value is determined according to the transmission size, the high-speed transmission response coefficient, and the high-speed channel transmission rate;

[0068] The low speed value is determined according to the transmission size, the low speed response coefficient and the low speed channel transmission rate, and the ratio of the low speed value to the high speed value is determined as the acceleration value.

[0069] Among them, the high-speed transmission response coefficient is a parameter value for the high-speed transmission channel to respond in advance. The high-speed channel transmission rate is the transmission rate of the high-speed transmission channel measured when the data transmission channel is started. The high-speed value is the estimated score for completing the transmission task using the high-speed channel. The low-speed response coefficient is a parameter value for the low-speed transmission channel to respond in advance. The low-speed channel transmission rate is the transmission rate of the low-speed transmission channel measured when the data transmission channel is started. The low-speed value is the estimated score for completing the transmission task using the low-speed channel.

[0070] Specifically, if the task type is a new task, it is determined that the target data has not been transmitted, so the size of the target data is used as the transmission size corresponding to the transmission task. Further, according to a preset calculation formula, the transmission size, the high-speed transmission response coefficient, and the high-speed channel transmission rate are calculated to obtain a high-speed value. Correspondingly, according to a preset calculation formula, the transmission size, the low-speed response coefficient, and the low-speed channel transmission rate are calculated to obtain a low-speed value. The ratio of the low-speed value to the high-speed value is determined as the acceleration value.

[0071] Based on the above example, if the task type is a new task, the target transmission channel can be determined according to the task type and acceleration value in the following way:

[0072] In response to the task type being a new task, obtaining a first threshold corresponding to the new task;

[0073] In response to the acceleration value being greater than a first threshold, determining that the target transmission channel is a high-speed transmission channel;

[0074] In response to the acceleration value being less than or equal to the first threshold, it is determined that the target transmission channel is a low-speed transmission channel.

[0075] The first threshold is a value preset for distinguishing acceleration values ​​and determining whether to use a high-speed transmission channel or a low-speed transmission channel when the task type is a new task.

[0076] Specifically, if the task type is a new task, the first threshold corresponding to the new task can be obtained, and the magnitude relationship between the acceleration value and the first threshold can be determined. When the acceleration value is greater than the first threshold, it can be determined that the transmission time using the high-speed transmission channel is much shorter than the transmission time using the low-speed transmission channel, and therefore the target transmission channel is determined to be the high-speed transmission channel. When the acceleration value is less than or equal to the first threshold, it can be determined that the transmission time using the high-speed transmission channel is not much shorter than the transmission time using the low-speed transmission channel, and therefore the target transmission channel is determined to be the low-speed transmission channel.

[0077] Based on the above example, if the task type is a resume task, the acceleration value can be determined according to the task type and the transfer size corresponding to the transfer task in the following way:

[0078] In response to the task type being a resume transmission task, the resume transmission data in the target data is determined, the size of the resume transmission data is used as the transmission size corresponding to the transmission task, and the high-speed value is determined according to the transmission size, the high-speed transmission response coefficient, and the historical high-speed transmission rate;

[0079] The low speed value is determined according to the transmission size, the low speed response coefficient and the low speed channel transmission rate, and the ratio of the low speed value to the high speed value is determined as the acceleration value.

[0080] The continued transmission data is data in the target data other than the already transmitted data. The historical high-speed transmission rate is based on the average transmission rate of the already transmitted data in the target data using the high-speed transmission channel.

[0081] Specifically, if the task type is a resume task, it is determined that the target data has been partially transmitted, and therefore, the size of the resume data in the target data other than the transmitted data is used as the transmission size corresponding to the transmission task. In addition, the relevant transmission information of the transmitted data, specifically, the transmission rate of the high-speed transmission channel used when transmitting the transmitted data, can be used to calculate the average value of these transmission rates as the historical high-speed transmission rate. Further, according to a preset calculation formula, the transmission size, the high-speed transmission response coefficient, and the historical high-speed transmission rate are calculated to obtain a high-speed value. Correspondingly, according to a preset calculation formula, the transmission size, the low-speed response coefficient, and the low-speed channel transmission rate are calculated to obtain a low-speed value. The ratio of the low-speed value to the high-speed value is determined as the acceleration value.

[0082] Based on the above example, if the task type is a resume task, the target transmission channel can be determined according to the task type and acceleration value in the following way:

[0083] In response to the task type being a resume task, obtaining a second threshold corresponding to the resume task;

[0084] In response to the acceleration value being greater than a second threshold, determining that the target transmission channel is a high-speed transmission channel;

[0085] In response to the acceleration value being less than or equal to the second threshold, it is determined that the target transmission channel is a low-speed transmission channel.

[0086] The second threshold is a value preset to distinguish the acceleration value and determine whether to use a high-speed transmission channel or a low-speed transmission channel when the task type is a resume transmission task.

[0087] Specifically, if the task type is a resume task, the second threshold corresponding to the resume task can be obtained, and the magnitude relationship between the acceleration value and the second threshold can be determined. When the acceleration value is greater than the second threshold, it can be determined that the transmission time using the high-speed transmission channel is much shorter than the transmission time using the low-speed transmission channel, so the target transmission channel is determined to be a high-speed transmission channel. When the acceleration value is less than or equal to the second threshold, it can be determined that the transmission time using the high-speed transmission channel is not much shorter than the transmission time using the low-speed transmission channel, so the target transmission channel is determined to be a low-speed transmission channel.

[0088] The middle layer can provide dynamic data acceleration function. The virtual data transmission bus can build two data transmission channels (high-speed transmission channel and low-speed transmission channel), which can realize automatic acceleration of transmission tasks according to user needs and network environment. Figure 5 It is a flow chart for determining the target transmission channel.

[0089] Among them, the first threshold It is the selection threshold for whether the new task uses the high-speed transmission channel. The new task is calculated based on the transmission size, the high-speed transmission response coefficient, and the high-speed channel transmission rate. ,in, is the transfer size, To use the high-speed transmission channel, High-speed transmission response coefficient, for example: 0< <0.3 (0.3 is an empirical value, adjustable), It is the transmission rate of the high-speed channel, which is measured when the client is started. ,in, To use the low speed value of the low speed transmission channel, is the low-speed transmission response coefficient, for example: 0< <0.1 (0.1 is an empirical value, adjustable), is the transmission rate of the low-speed channel, measured when the client starts up. , J is the value of the low-speed channel transmission time divided by the high-speed channel transmission time, that is, the acceleration value. If the acceleration value is greater than the first threshold That is, when the transmission time of the high-speed channel is much shorter than that of the low-speed channel, the high-speed transmission channel will be used, otherwise the low-speed transmission channel will be used.

[0090] The retransmission task calculates the acceleration value J based on the remaining file size (the size of the retransmission data in the target data except the already transmitted data), the low-speed channel transmission rate, and the historical high-speed transmission rate. , is the size of the retransmission data, where To use the high-speed transmission channel, High-speed transmission response coefficient, for example: 0< <0.3 (0.3 is an empirical value, adjustable), This is the historical high-speed transmission rate of the task, which can be recorded in the task information table. *T, where is the size of the retransmission data. To use the low speed value of the low speed transmission channel, is the low-speed transmission response coefficient, for example: 0< <0.1 (0.1 is an empirical value, adjustable), is the transmission rate of the low-speed channel, measured when the client starts up. , J is the value of the low-speed channel transmission time divided by the high-speed channel transmission time, that is, the acceleration value. If the acceleration value is greater than the second threshold That is, when the transmission time of the high-speed channel is much shorter than that of the low-speed channel, the high-speed transmission channel will be used, otherwise the low-speed transmission channel will be used.

[0091] S120. Obtain the job parameters corresponding to the target job in the current cluster, determine the target script according to the job parameters, the cluster identifier of the target cluster, and a pre-built script template library, and send the target script to the target cluster, so that the target cluster executes the target job based on the target script and target data to obtain the target result corresponding to the target job.

[0092] Among them, the job parameters can be the parameters that need to be configured when executing the target job, such as: the partition of the target job, the number of nodes, the number of node CPU (Central Processing Unit) cores, the number of GPU (Graphics Processing Unit) cores, (cross-cluster) input files, working directories, etc. The cluster identifier is an identifier used to distinguish the target cluster from other clusters. The script template library is a pre-built database for storing script templates corresponding to different clusters. The script template can be understood as a script without parameters. The target script is a script obtained by substituting the job parameters into the script template corresponding to the cluster identifier of the target cluster in the script template library. The target result is the result data of executing the target script in the target cluster using the target data to perform the job.

[0093] Specifically, the target job in the current cluster is analyzed and parameters are obtained to obtain the job parameters corresponding to the target job. The pre-built script template library is searched according to the cluster ID of the target cluster to obtain the script template corresponding to the cluster ID of the target cluster. The obtained job parameters are written into the script template to obtain the target script. Further, the target script is sent to the target cluster. After receiving the target script and target data, the target cluster executes the target script and calls the target data to complete the target job.

[0094] Based on the above example, you can determine the target script according to the job parameters, the cluster ID of the target cluster, and the pre-built script template library in the following way:

[0095] According to the cluster identifier of the target cluster, a target template corresponding to the cluster identifier is determined from a pre-built script template library;

[0096] Add the job parameters to the target template to obtain the target script.

[0097] The target template is a script template corresponding to the cluster identifier of the target cluster in a pre-built script template library, and is used for subsequent construction of the target script.

[0098] Specifically, a search is performed in a pre-built script template library according to the cluster identifier of the target cluster to obtain a target template corresponding to the cluster identifier of the target cluster, and then the obtained job parameters of the target job are written into the target template to obtain the target script.

[0099] Based on the above example, you can add job parameters to the target template in the following way to get the target script:

[0100] According to the target template, determine the template content and input items;

[0101] Render the task submission page based on the template content and input items;

[0102] Based on the task submission page, parameter operations for each job parameter are received, and a target script is generated according to each parameter operation.

[0103] The template content is the part of the target template that does not need to be modified, and the input item is the part used to add the corresponding job parameters, which can be adjusted and input. The task submission page is a page used to display the target template, and the template content and input items can be displayed separately. The parameter operation can be a visual execution operation of entering the corresponding job parameters in the input item, for example: an operation based on the text input box, drop-down selection box, file tree selection box, etc. corresponding to the input item, and can also be drag, drop, and drop operations.

[0104] Specifically, the target template is divided into template content and input items, and the template content and input items are rendered on the page to obtain a task submission page, so that the user can add job parameters to the input items based on the task submission page. Based on the task submission page, the parameter operation triggered by the user to input the job parameters for each input item can be received, the parameter operations are parsed, the job parameters are added to the input items, and the target script is generated to integrate the job parameters with the target template.

[0105] The script template is built based on low-code technology. Users only need to drag and drop on the web to generate a script template. The script template contains the partitions, number of nodes, number of node CPU cores, number of GPU cores, (cross-cluster) input files, working directories, etc. that need to run the task. All parameters (job parameters) support customization, and support text input boxes, drop-down selection boxes, file tree selection boxes, etc.

[0106] The process of generating target scripts based on templates is as follows Figure 6As shown in the figure, the user first selects a target template created by himself or built into the system from the script template library. The template engine automatically renders the low-code platform into a form-based task submission page based on the input items and template content of the target template. Through this task submission page, the user enters the job parameters and running content to automatically generate the target script of the final task. Then, the target script is stored through the storage interface of the fusion computing power API Server (API service), and the task is submitted to the cross-cluster task scheduler through cross-domain task scheduling for cross-cluster operations.

[0107] S130 . In response to determining that the target cluster has completed the target job, a result transmission channel is established between the current cluster and the target cluster, and the target result in the target cluster is sent to the current cluster through the result transmission channel.

[0108] The result transmission channel is a data channel between the current cluster and the target cluster, and is used for transmitting target results.

[0109] Specifically, the target cluster executes the target script, calls the target data, completes the target job and obtains the target result. The middle layer can monitor the generation of the target result. After determining that the target cluster has completed the target job and obtained the complete target result, it establishes a result transmission channel between the current cluster and the target cluster in the same way as establishing a data transmission channel, so as to control the target cluster to send the target result to the current cluster through the result transmission channel to complete the cross-cluster execution of the target job.

[0110] It is understandable that in response to the target result transmission completion, the result transmission channel is revoked to avoid the mistransmission of other irrelevant data. Specifically, when the current cluster receives the target result, the target result can be verified, and after determining that the target result is complete and correct, a signal of transmission completion is generated. If the verification fails after the target result is transmitted, the result data can be cleared and the target result can be retransmitted based on the result transmission channel.

[0111] Generate cross-cluster target job scripts through templates. Read the script templates and job parameters submitted by users, automatically parse the cluster information, scheduling system information, software and version information in the API, and use the job content generation algorithm to automatically generate a new target script for job submission. Write the target script through the data security access API, and then call the job submission API to submit the job. When the job is completed or fails, call the cross-cluster data security transmission (result transmission channel) to return the data (target result).

[0112] Optionally, a job monitoring API is provided to the outside world. First, the cross-cluster tasks are saved in the database where the task scheduling initiation domain is located. The scheduled task query service (resident daemon) periodically queries the cross-cluster task list from the database, and queries the real-time status of the task from the fusion computing power API server of this domain according to the cross-cluster task ID (identifier) ​​and updates it to the database, and updates the task submission and task completion status. According to whether the task is a cross-cluster task, the data flow direction of the cross-cluster task (target data and target result) is automatically called by the data transmission API of this domain to initiate asynchronous cross-cluster data transmission (data transmission channel and result transmission channel), which is responsible for transmitting the pre-data (target data) and result data (target result) of the task.

[0113] The technical solution of the above example realizes cross-cluster and cross-hardware heterogeneous computing resource platform job scheduling and adaptive platform architecture. It can automatically complete the process of target data transmission, job submission and target result return. In addition, safe and reliable cross-cluster data transmission technology is used. The cross-cluster job submission technology based on the script template library simplifies the script construction across clusters and scheduling systems. It can be built using standardized API interfaces, supports modular calls, and can submit cross-cluster jobs in a variety of ways such as scripts, APIs, and command lines.

[0114] The present invention has the following technical effects: establishing a data transmission channel between the current cluster and the target cluster, sending the target data corresponding to the target job in the current cluster to the target cluster through the data transmission channel, so as to establish a separate data transmission channel for transmitting the target data, thereby ensuring the security of the data, and then obtaining the job parameters corresponding to the target job in the current cluster, determining the target script according to the job parameters, the cluster identifier of the target cluster and a pre-built script template library, and sending the target script to the target cluster, so that the target cluster executes the target job based on the target script and the target data to obtain the target result corresponding to the target job, so as to adapt to different systems and simplify the script generation steps, and in response to determining that the target cluster has completed the execution of the target job, establishing a result transmission channel between the current cluster and the target cluster, and sending the target result in the target cluster to the current cluster through the result transmission channel, thereby realizing the cross-cluster transmission of job data, automatic submission and operation of jobs across clusters and result transmission, thereby improving the efficiency and security of data transmission, and being able to adaptively generate corresponding scripts to reduce the technical difficulty of cross-cluster jobs.

[0115] Embodiment 2

[0116] Figure 7 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 7 As shown, the electronic device 400 includes one or more processors 401 and a memory 402 .

[0117] The processor 401 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0118] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may run the program instructions to implement the cross-cluster operation method of any embodiment of the present invention described above and / or other desired functions. Various contents such as initial external parameters, thresholds, etc. may also be stored in the computer-readable storage medium.

[0119] In one example, the electronic device 400 may further include: an input device 403 and an output device 404, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). The input device 403 may include, for example, a keyboard, a mouse, etc. The output device 404 may output various information to the outside, including early warning prompt information, braking force, etc. The output device 404 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0120] Of course, to simplify, Figure 7 Only some of the components related to the present invention in the electronic device 400 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 400 may also include any other appropriate components.

[0121] In addition to the above methods and devices, an embodiment of the present invention may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the cross-cluster operation method provided by any embodiment of the present invention.

[0122] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the embodiments of the present invention, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0123] In addition, an embodiment of the present invention may also be a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the processor executes the steps of the cross-cluster operation method provided by any embodiment of the present invention.

[0124] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0125] It should be noted that the terms used in the present invention are only for describing specific embodiments, rather than limiting the scope of the present application. As shown in the present specification, unless the context clearly indicates an exception, the words "one", "a", "a kind of" and / or "the" do not specifically refer to the singular, but may also include the plural. The terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the presence of other identical elements in the process, method or device including the elements.

[0126] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A cross-cluster operation method, characterized in that: include: Establishing a data transmission channel between the current cluster and the target cluster, and sending the target data corresponding to the target job in the current cluster to the target cluster through the data transmission channel; Obtaining job parameters corresponding to the target job in the current cluster, determining a target script according to the job parameters, a cluster identifier of the target cluster, and a pre-built script template library, and sending the target script to the target cluster, so that the target cluster executes the target job based on the target script and the target data to obtain a target result corresponding to the target job; In response to determining that the target cluster has completed the target job, a result transmission channel is established between the current cluster and the target cluster, and the target result in the target cluster is sent to the current cluster through the result transmission channel.

2. The method according to claim 1, characterized in that Before establishing the data transmission channel between the current cluster and the target cluster, the method further includes: In response to the current cluster not satisfying the resource condition of the target job, determining a physical space corresponding to the current cluster, and judging whether there is a first cluster satisfying the resource condition in the physical space; If so, determining a target cluster corresponding to the current cluster from each first cluster; If not, determining the current domain corresponding to the current cluster, and judging whether there is a second cluster meeting the resource condition in the current domain; If so, determining a target cluster corresponding to the current cluster from each second cluster; If not, traverse and obtain each third cluster that meets the resource condition, and determine a target cluster corresponding to the current cluster from each third cluster according to the inter-domain distance between each third cluster and the current cluster.

3. The method according to claim 1, characterized in that The establishing of a data transmission channel between the current cluster and the target cluster includes: Create a first microservice corresponding to the current cluster, and create a second microservice corresponding to the target cluster; Based on the first microservice, perform verification processing with the current cluster, so that when the verification passes, the current cluster creates a third microservice corresponding to the first microservice; In response to the current cluster being verified, establishing a first transmission channel between the first microservice and the third microservice based on the first microservice, and determining a first encryption algorithm based on the first transmission channel; Based on the second microservice, perform verification processing with the target cluster, so that the target cluster creates a fourth microservice corresponding to the second microservice when the verification passes; In response to the target cluster being verified successfully, a second transmission channel between the second microservice and the fourth microservice is established based on the second microservice, and a second encryption algorithm is determined based on the second transmission channel; The first transmission channel, the first encryption algorithm, the second transmission channel, and the second encryption algorithm are used as a data transmission channel between the current cluster and the target cluster.

4. The method according to claim 1, characterized in that: The sending the target data corresponding to the target job in the current cluster to the target cluster through the data transmission channel includes: Determine a task type of a target data transmission task corresponding to a target job in the current cluster; Determining an acceleration value according to the task type and the transmission size corresponding to the transmission task; A target transmission channel is determined according to the task type and the acceleration value, and the target job is sent to the target cluster based on the target transmission channel to complete the transmission task.

5. The method according to claim 4, characterized in that The determining of the acceleration value according to the task type and the transmission size corresponding to the transmission task includes: In response to the task type being a new task, the size of the target data is used as the transmission size corresponding to the transmission task, and a high-speed value is determined according to the transmission size, a high-speed transmission response coefficient, and a high-speed channel transmission rate; Determine a low speed value according to the transmission size, the low speed response coefficient and the low speed channel transmission rate, and determine the ratio of the low speed value to the high speed value as the acceleration value; Accordingly, determining the target transmission channel according to the task type and the acceleration value includes: In response to the task type being a new task, obtaining a first threshold corresponding to the new task; In response to the acceleration value being greater than the first threshold, determining that the target transmission channel is a high-speed transmission channel; In response to the acceleration value being less than or equal to the first threshold, it is determined that the target transmission channel is a low-speed transmission channel.

6. The method according to claim 4, characterized in that The determining of the acceleration value according to the task type and the transmission size corresponding to the transmission task includes: In response to the task type being a resumed transmission task, the resumed transmission data in the target data is determined, the size of the resumed transmission data is used as the transmission size corresponding to the transmission task, and the high-speed value is determined according to the transmission size, the high-speed transmission response coefficient, and the historical high-speed transmission rate; wherein the historical high-speed transmission rate is based on the average transmission rate of the transmitted data in the target data using the high-speed transmission channel; Determine a low speed value according to the transmission size, the low speed response coefficient and the low speed channel transmission rate, and determine the ratio of the low speed value to the high speed value as the acceleration value; Accordingly, determining the target transmission channel according to the task type and the acceleration value includes: In response to the task type being a resume task, obtaining a second threshold corresponding to the resume task; In response to the acceleration value being greater than the second threshold, determining that the target transmission channel is a high-speed transmission channel; In response to the acceleration value being less than or equal to the second threshold, it is determined that the target transmission channel is a low-speed transmission channel.

7. The method according to claim 1, characterized in that The determining the target script according to the job parameters, the cluster identifier of the target cluster and a pre-built script template library includes: According to the cluster identifier of the target cluster, determining a target template corresponding to the cluster identifier from a pre-built script template library; The job parameters are added to the target template to obtain a target script.

8. The method according to claim 7, characterized in that The step of adding the job parameters to the target template to obtain a target script includes: Determine template content and input items according to the target template; Rendering a task submission page according to the template content and the input items; Based on the task submission page, parameter operations for each job parameter are received, and a target script is generated according to each parameter operation.

9. An electronic device, characterized in that: The electronic device comprises: Processor and memory; The processor is used to execute the steps of the cross-cluster operation method according to any one of claims 1 to 8 by calling the program or instruction stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and the program or instruction enables a computer to execute the steps of the cross-cluster operation method according to any one of claims 1 to 8.

Images