A cross-platform, cross-cluster hybrid operation method and system

Abstract

This invention relates to the field of data processing technology, specifically disclosing a cross-platform, cross-cluster hybrid job method and system. By constructing an initial hybrid environment adaptation model, this invention can pre-adjust job scheduling strategies based on the resource characteristics and job attributes of physical machine clusters and cloud environment clusters. This enables full collaboration of resources in different environments, improving the flexibility and efficiency of job scheduling. By dynamically adjusting the execution order and resource allocation of jobs, and utilizing cross-environment data sharing space, seamless sharing and flow of job data and status between physical machines and cloud environments are achieved. Optimization factors are generated by real-time collection of the operating status and resource change data of sub-job units, ensuring the real-time performance and accuracy of job scheduling. Through a hybrid environment job execution evaluation report feedback mechanism, the initial hybrid environment adaptation model can be continuously optimized to form a closed-loop optimization, ensuring the stability and efficiency of job execution.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a cross-platform, cross-cluster hybrid operation method and system. Background Technology

[0002] With the rapid development of cloud-native technologies, enterprise IT infrastructure is increasingly exhibiting hybrid characteristics, encompassing both physical machine environments and various cloud platform environments. In this hybrid environment, business tasks also exhibit different characteristics: stateless tasks are suitable for running in elastic and scalable cloud environments, while stateful tasks are more suitable for execution in stable physical machine environments. However, in real-world business scenarios, a complete job workflow often requires the simultaneous use of resources from both types of environments and the transfer of data and state between them. Currently, the market primarily uses Apache Airflow (a visual distributed task scheduling platform that provides workflow orchestration based on DAGs) for job scheduling. However, this approach mainly focuses on workflow orchestration rather than cross-environment execution and context sharing of jobs, and its data transfer mechanism suffers from performance bottlenecks during large-scale data transfers. Summary of the Invention

[0003] The main objective of this invention is to provide a cross-platform, cross-cluster hybrid operation method, which aims to solve the technical problems in the prior art.

[0004] This invention proposes a cross-platform, cross-cluster hybrid operation method, comprising:

[0005] Obtain the attribute parameters of the job to be executed, the first resource characteristic parameters of the physical machine cluster, and the second resource characteristic parameters of the cloud environment cluster, and construct an initial hybrid environment adaptation model based on the first resource characteristic parameters, the second resource characteristic parameters, and the attribute parameters;

[0006] The job scheduling strategy is pre-adjusted based on the initial hybrid environment adaptation model, and the job is decomposed into multiple sub-job units based on the pre-adjusted job scheduling strategy.

[0007] Obtain the heterogeneity characteristics, job data interaction requirements, and hybrid environment collaborative scheduling objectives of physical machine clusters and cloud environment clusters, and construct a cross-environment data sharing space based on the aforementioned heterogeneity characteristics, job data interaction requirements, and hybrid environment collaborative scheduling objectives;

[0008] Each sub-job unit is assigned a corresponding context-aware module. The running status parameters of the corresponding sub-job unit and the dynamic resource change data of the environment are collected in real time according to each context-aware module. Based on each running status parameter and dynamic resource change data, a corresponding job scheduling optimization factor is generated.

[0009] Establish a dependency graph among multiple sub-job units, and dynamically adjust the execution order and resource allocation of the corresponding sub-job units according to the dependency graph and each job scheduling optimization factor, and update the context information in the cross-environment data sharing space in real time to obtain the corresponding execution results;

[0010] A hybrid environment job execution evaluation report is generated based on multiple execution results and historical interaction data in the cross-environment data sharing space. The parameter weights of the initial hybrid environment adaptation model are optimized based on the execution evaluation report to obtain the final hybrid environment adaptation model, so as to achieve seamless collaborative execution of jobs between physical machines and cloud environments.

[0011] Preferably, the steps of pre-adjusting the job scheduling strategy according to the initial hybrid environment adaptation model and decomposing the job into multiple sub-job units based on the pre-adjusted job scheduling strategy include:

[0012] The basic allocation ratio of jobs in physical machines and cloud environments is obtained based on the physical machine adaptation degree and cloud environment adaptation degree output by the initial hybrid environment adaptation model, and a job scheduling strategy is generated based on the basic allocation ratio.

[0013] Obtain the first computing load, first average computing power, unit time operation and maintenance cost, and first execution time of the physical machine, as well as the second computing load, second average computing power, unit time rental cost, and second execution time of the cloud environment;

[0014] Obtain cross-environment data transmission time and resource load constraints, and obtain the estimated total execution time of the job based on the cross-environment data transmission time, the first computational load, the first average computational capacity, the second computational load, and the second average computational capacity;

[0015] The total scheduling cost is obtained based on the unit time operation and maintenance cost, the first execution duration, the unit time rental cost, and the second execution duration. The job scheduling strategy is then verified and pre-adjusted based on the total scheduling cost, resource load constraints, and the estimated total job execution time.

[0016] Obtain the parallelism parameters of the pre-adjusted job scheduling strategy, and decompose the job into multiple sub-job units according to the parallelism parameters using a hierarchical clustering algorithm.

[0017] Preferably, the step of constructing a cross-environment data sharing space based on the heterogeneous characteristics, job data interaction requirements, and hybrid environment collaborative scheduling objectives includes:

[0018] Based on the heterogeneity characteristics, obtain the hardware architecture difference value, network latency coefficient, storage protocol adaptability and computing resource elasticity coefficient, and construct an environment difference matrix based on the hardware architecture difference value, network latency coefficient, storage protocol adaptability and computing resource elasticity coefficient;

[0019] Based on the data interaction requirements of the operation, obtain the data interaction frequency, data transmission volume and data timeliness requirements, and construct a data transmission weight vector based on the data interaction frequency, data transmission volume and data timeliness requirements;

[0020] Based on the hybrid environment collaborative scheduling objectives, resource utilization objectives, job response time objectives, and cost consumption objectives are obtained, and a scheduling priority matrix is ​​constructed based on the resource utilization objectives, job response time objectives, and cost consumption objectives.

[0021] The environmental difference matrix, data transmission weight vector, and scheduling priority matrix are input into the preset shared space basic model to generate an initial cross-environment data sharing space.

[0022] Obtain the real-time data transmission rate, data consistency deviation value, and resource utilization rate of the initial cross-environment data sharing space, and obtain the sharing space optimization coefficient based on the real-time data transmission rate, data consistency deviation value, and resource utilization rate;

[0023] The initial cross-environment data sharing space is dynamically adjusted based on the shared space optimization coefficient to obtain the target cross-environment data sharing space.

[0024] Preferably, the step of generating a corresponding job scheduling optimization factor based on each of the aforementioned operating status parameters and dynamic resource change data includes:

[0025] Obtain the sub-job real-time response latency, task dependency, and calculated load fluctuation coefficient of the running status parameters, and construct a sub-job status evaluation matrix based on the sub-job real-time response latency, task dependency, and calculated load fluctuation coefficient;

[0026] The environmental resource utilization rate, resource elastic scaling rate, cross-environment data transmission bandwidth, and resource competition coefficient of the dynamic resource change data are obtained, and an environmental resource adaptability vector is constructed based on the environmental resource utilization rate, resource elastic scaling rate, cross-environment data transmission bandwidth, and resource competition coefficient.

[0027] Initial scheduling optimization factors are generated based on the environmental resource adaptability vector and the sub-job status evaluation matrix using a multi-dimensional weighted fusion algorithm.

[0028] Obtain the sub-job completion time deviation, resource waste rate, and cross-environment data synchronization error corresponding to the initial scheduling optimization factor, and obtain the factor correction coefficient based on the sub-job completion time deviation, resource waste rate, and cross-environment data synchronization error;

[0029] The target job scheduling optimization factor is obtained by iteratively optimizing the initial scheduling optimization factor based on the factor correction coefficient.

[0030] Preferably, the step of dynamically adjusting the execution order and resource allocation of corresponding sub-job units based on the dependency graph and each job scheduling optimization factor includes:

[0031] Extract all predecessor job sets and successor job sets for each sub-job unit from the dependency graph, and obtain the dependency strength of the corresponding sub-job unit based on each predecessor job set;

[0032] The execution priority of each sub-job unit is obtained based on each set of subsequent jobs and the dependency strength, and all sub-job units are sorted from high to low according to the execution priority to obtain the initial execution order;

[0033] Determine whether the initial execution order satisfies the execution timing constraints in the dependency graph;

[0034] If the initial execution order does not satisfy the execution timing constraints in the dependency graph, the sub-job units that violate the execution timing constraints will be adjusted until all their predecessor jobs have been executed to obtain the final execution order;

[0035] The target execution environment for the corresponding sub-job unit is determined based on each job scheduling optimization factor, and the total available computing resources, total available storage resources, and total available network bandwidth of the target execution environment are obtained.

[0036] Obtain the resource allocation ratio for each sub-job unit, and obtain the initial computing resource allocation based on the resource allocation ratio and the total available computing resources;

[0037] The initial storage resource allocation is obtained based on the resource allocation ratio and the total available storage resources, and the initial network bandwidth allocation is obtained based on the resource allocation ratio and the total available network bandwidth.

[0038] Based on the data interaction weights in the dependency graph, the computational resource allocation, storage resource allocation, and network bandwidth allocation for sub-job units with strong dependencies are balanced and adjusted, and the adjusted computational resource allocation, storage resource allocation, and network bandwidth allocation are used as the final resource allocation for the sub-job units.

[0039] Preferably, the step of generating a hybrid environment job execution evaluation report based on multiple execution results and historical interaction data in a cross-environment data sharing space includes:

[0040] Obtain the job completion time, total resource consumption, and data transmission success rate for each execution result, and obtain the single-dimensional execution indicators of the corresponding sub-job unit based on the job completion time, total resource consumption, and data transmission success rate for each job.

[0041] The cross-environment data transmission delay, environment switching frequency, and state synchronization error of the historical interaction data are obtained, and the cross-environment collaboration index of the hybrid environment is obtained based on the cross-environment data transmission delay, environment switching frequency, and state synchronization error.

[0042] A comprehensive evaluation matrix is ​​constructed based on multiple single-dimensional performance indicators and cross-environmental collaborative indicators, and the weight coefficient of each indicator is determined by the analytic hierarchy process.

[0043] The comprehensive evaluation value of each sub-work unit and the overall evaluation value of the mixed environment are obtained based on the comprehensive evaluation matrix and weighting coefficients.

[0044] A hybrid environment operation performance evaluation report is generated based on each comprehensive evaluation value, overall evaluation value, single-dimensional performance indicator, and cross-environment collaborative indicator.

[0045] This application also provides a cross-platform, cross-cluster hybrid operating system, including:

[0046] The first construction module is used to obtain the attribute parameters of the job to be executed, the first resource characteristic parameters of the physical machine cluster and the second resource characteristic parameters of the cloud environment cluster, and to construct an initial hybrid environment adaptation model based on the first resource characteristic parameters, the second resource characteristic parameters and the attribute parameters.

[0047] The first adjustment module is used to pre-adjust the job scheduling strategy according to the initial hybrid environment adaptation model, and to decompose the job into multiple sub-job units based on the pre-adjusted job scheduling strategy.

[0048] The second construction module is used to obtain the heterogeneity characteristics, job data interaction requirements and hybrid environment collaborative scheduling objectives of physical machine clusters and cloud environment clusters, and to construct a cross-environment data sharing space based on the heterogeneity characteristics, job data interaction requirements and hybrid environment collaborative scheduling objectives.

[0049] The data acquisition and generation module is used to assign a corresponding context-aware module to each sub-job unit, collect the running status parameters of the corresponding sub-job unit and the dynamic resource change data of the environment in real time according to each context-aware module, and generate a corresponding job scheduling optimization factor based on each running status parameter and dynamic resource change data.

[0050] The second adjustment module is used to establish a dependency graph between multiple sub-job units, and dynamically adjust the execution order and resource allocation of the corresponding sub-job units according to the dependency graph and each job scheduling optimization factor, and update the context information in the cross-environment data sharing space in real time to obtain the corresponding execution results.

[0051] The optimization module is used to generate a hybrid environment job execution evaluation report based on multiple execution results and historical interaction data in the cross-environment data sharing space, and optimize the parameter weights of the initial hybrid environment adaptation model based on the execution evaluation report to obtain the final hybrid environment adaptation model, so as to achieve seamless collaborative execution of jobs between physical machines and cloud environments.

[0052] Preferably, the first adjustment module includes:

[0053] The generation unit is used to obtain the basic allocation ratio of jobs in the physical machine and cloud environments based on the physical machine adaptation degree and cloud environment adaptation degree output by the initial hybrid environment adaptation model, and to generate a job scheduling strategy based on the basic allocation ratio.

[0054] The first acquisition unit is used to acquire the first computing power, first average computing power, unit time operation and maintenance cost, and first execution time of the physical machine, as well as the second computing power, second average computing power, unit time rental cost, and second execution time of the cloud environment.

[0055] The second acquisition unit is used to acquire cross-environment data transmission time and resource load constraints, and to acquire the estimated total execution time of the job based on the cross-environment data transmission time, the first computational load, the first average computing power, the second computational load, and the second average computing power.

[0056] The verification and adjustment unit is used to obtain the total scheduling cost based on the unit time operation and maintenance cost, the first execution duration, the unit time rental cost, and the second execution duration, and to verify and pre-adjust the job scheduling strategy based on the total scheduling cost, resource load constraints, and the estimated total job execution time.

[0057] The decomposition unit is used to obtain the parallelism parameters of the pre-adjusted job scheduling strategy, and decompose the job into multiple sub-job units according to the parallelism parameters through a hierarchical clustering algorithm.

[0058] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described cross-platform and cross-cluster hybrid operation method.

[0059] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described cross-platform, cross-cluster hybrid operation method.

[0060] The beneficial effects of this invention are as follows: By constructing an initial hybrid environment adaptation model, this invention can pre-adjust the job scheduling strategy based on the resource characteristic parameters and job attributes of physical machine clusters and cloud environment clusters, enabling resources in different environments to fully coordinate and improving the flexibility and efficiency of job scheduling. By dynamically adjusting the execution order and resource allocation of jobs, and utilizing the cross-environment data sharing space, seamless sharing and flow of job data and status between physical machines and cloud environments are achieved, effectively overcoming the problem of information isolation between different platforms in traditional methods. By collecting the running status and resource change data of sub-job units in real time, optimization factors are generated to ensure the real-time performance and accuracy of job scheduling, thereby optimizing the overall performance of job execution. By leveraging the hybrid environment job execution evaluation report feedback mechanism, the initial hybrid environment adaptation model can be continuously optimized to form a closed-loop optimization, ensuring the stability and efficiency of job execution. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of a method flow according to an embodiment of the present invention.

[0062] Figure 2 This is a schematic diagram of the system structure according to an embodiment of the present invention.

[0063] Figure 3 This is a schematic diagram of the internal structure of a computer device according to an embodiment of this application.

[0064] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0065] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0066] like Figure 1 As shown, this application provides a cross-platform, cross-cluster hybrid operation method, including:

[0067] S1. Obtain the attribute parameters of the job to be executed, the first resource characteristic parameters of the physical machine cluster, and the second resource characteristic parameters of the cloud environment cluster, and construct an initial hybrid environment adaptation model based on the first resource characteristic parameters, the second resource characteristic parameters, and the attribute parameters.

[0068] S2. The job scheduling strategy is pre-adjusted according to the initial hybrid environment adaptation model, and the job is decomposed into multiple sub-job units based on the pre-adjusted job scheduling strategy.

[0069] S3. Obtain the heterogeneity characteristics, job data interaction requirements, and hybrid environment collaborative scheduling objectives of the physical machine cluster and the cloud environment cluster, and construct a cross-environment data sharing space based on the heterogeneity characteristics, job data interaction requirements, and hybrid environment collaborative scheduling objectives.

[0070] S4. Assign a corresponding context-aware module to each sub-job unit, collect the running status parameters of the corresponding sub-job unit and the dynamic resource change data of the environment in real time according to each context-aware module, and generate a corresponding job scheduling optimization factor based on each running status parameter and dynamic resource change data.

[0071] S5. Establish a dependency graph between multiple sub-job units, and dynamically adjust the execution order and resource allocation of the corresponding sub-job units according to the dependency graph and each job scheduling optimization factor, and update the context information in the cross-environment data sharing space in real time to obtain the corresponding execution results.

[0072] S6. Generate a hybrid environment job execution evaluation report based on multiple execution results and historical interaction data in the cross-environment data sharing space, and optimize the parameter weights of the initial hybrid environment adaptation model based on the execution evaluation report to obtain the final hybrid environment adaptation model, so as to achieve seamless collaborative execution of jobs between physical machines and cloud environments.

[0073] As described in steps S1-S6 above, the steps of constructing the initial hybrid environment adaptation model based on the first resource feature parameters, the second resource feature parameters, and attribute parameters include standardizing the core dimensions of the first resource feature parameters, the second resource feature parameters, and the core dimensions of the job attribute parameters; calculating the parameter weights of each core dimension using the analytic hierarchy process (AHP) to construct judgment moments; designing a physical machine-cloud environment feature mapping layer to map the core dimensions of the first resource feature parameters and the core dimensions of the second resource feature parameters of the cloud environment parameter vector to the same feature space; and then fusing them through an attention mechanism. The initial hybrid environment adaptation model uses a three-layer stacked restricted Boltzmann machine, followed by an improved cross-entropy loss. Finally, environmental collaborative correction is performed on the output layer results, and a three-dimensional verification index system is adopted, including resource matching accuracy, performance prediction error, and cost optimization rate. The initial hybrid environment adaptation model constructed through the above steps can achieve deep fusion of heterogeneous features of physical machine and cloud environment resources, solving the adaptation blind spot problem of traditional single environment optimization models. The model parameters are based on newly generated job data. Incremental updates are performed to ensure that the adaptation accuracy is dynamically optimized as the environment changes. The steps for obtaining the corresponding execution results by updating the context information in the cross-environment data sharing space in real time include constructing a context information matrix in the cross-environment data sharing space, where the matrix elements represent the context parameters of the sub-job unit in the corresponding environment. The real-time data transmission latency, resource utilization, execution progress, and environmental load of the sub-job unit in the corresponding environment are obtained, and the context information update factor is calculated by weighted summation based on the real-time data transmission latency, resource utilization, execution progress (value range 0-1), and environmental load. The context information matrix is ​​updated using the context information update factor. The update step is to add the product of the original context parameters in the matrix and (1 minus the update factor) to the product of the real-time data transmission latency, resource utilization, execution progress, and environmental load and the context information update factor to obtain the updated context information matrix elements. When the execution progress of the sub-job unit in the corresponding environment reaches 1, its context information can be marked as completed and synchronized to the subsequent job units associated in the dependency graph.

[0074] The steps for establishing a dependency graph among multiple sub-job units include: obtaining the input data identifier, output data identifier, pre-execution conditions, and resource exclusivity marker of each sub-job unit; comparing the input data identifier and output data identifier of all sub-job units and obtaining the corresponding data dependency strength; for sub-job units containing pre-execution conditions, parsing the atomic conditions in the pre-execution conditions and obtaining the corresponding condition dependency trigger probability; collecting the resource types occupied by sub-job units with resource exclusivity markers and obtaining the corresponding resource conflict cost; constructing a directed graph with sub-job units as nodes and data dependency, condition dependency, and resource dependency as directed edges; adding weight attributes to each edge; and performing loop detection on the graph to obtain the sub-job unit relationship graph.

[0075] This invention acquires attribute parameters of the job to be executed, first resource characteristic parameters of the physical machine cluster, and second resource characteristic parameters of the cloud environment cluster. Since the attribute parameters of the job to be executed provide basic data for the specific requirements of the job, and the resource characteristic parameters of the physical machine cluster and cloud environment cluster describe the resource status of the physical machine and cloud environment respectively, by quantifying the resource status of different environments (physical machine and cloud environment), the job scheduling system can flexibly adapt to different environments. Unlike existing technologies that typically rely on resource information from a single environment (e.g., only physical machine resource information or cloud resource information), this invention, through multi-dimensional resource characteristic parameters, can more accurately assess resource distribution in mixed environments, ensuring that jobs can be optimally configured in two different resource environments. This improves the comprehensiveness and accuracy of the scheduling model, avoids the one-sidedness of resource allocation, and by capturing the resource characteristics of both environments in detail, job scheduling no longer relies solely on the resource status of a single environment, but more accurately captures cross-environmental scenarios. This approach addresses resource requirements, avoiding the imbalances or mismatches in resource allocation common in existing technologies. An initial hybrid environment adaptation model is constructed based on first resource characteristic parameters, second resource characteristic parameters, and attribute parameters. By integrating multiple resource characteristic parameters and job attribute parameters, an initial hybrid environment adaptation model is built. Compared to existing technologies that optimize scheduling for a single environment, this method fully considers the heterogeneity of physical machine clusters and cloud environment clusters, constructing a hybrid scheduling model adapted to different resource environments. This model not only optimizes resource utilization efficiency in a single environment but also coordinates resource allocation between physical machines and cloud environments, achieving more efficient job scheduling. It solves the problem of existing technologies optimizing only for a single resource environment (physical machine or cloud environment), enabling jobs to achieve reasonable resource allocation in a hybrid environment. By uniformly modeling the resource characteristics of different environments, the problem of incompatibility between resource requirements in different environments is avoided, improving the system's adaptability and flexibility.

[0076] By pre-adjusting the job scheduling strategy using an initial hybrid environment adaptation model, the resource allocation and execution order of jobs can be optimized before actual execution. This pre-adjustment not only considers the basic resource requirements of jobs but also the differences in resources across different environments, avoiding incompatibility issues when the scheduling strategy is executed across environments. Unlike existing technologies that perform resource scheduling only during execution, this invention significantly improves the system's efficiency and resource utilization in actual operation by adjusting the scheduling strategy in advance. Optimizing the job scheduling strategy in advance reduces resource waste and time delays during subsequent execution, effectively predicting and avoiding potential resource conflicts and uneven load distribution during scheduling, thus improving overall execution efficiency. Based on the pre-adjusted job scheduling strategy, the job is broken down into multiple sub-job units. By breaking down the job into multiple sub-job units, the execution of the job is made more granular and modular. This allows for flexible adjustment of the resource requirements of each sub-task during job execution, making the entire job execution more flexible and refined. Compared with the existing technology that usually schedules the entire job as a unit, this decomposition method can better adapt to the different resource characteristics in heterogeneous environments. Through refined resource management and scheduling, the system's resource utilization is improved, and the flexibility of job scheduling is enhanced. Each sub-job unit can be independently scheduled and resource allocated according to different environments. Through modular decomposition, the resource allocation of each sub-job can be dynamically adjusted according to the actual execution status of the job, thereby optimizing the overall execution efficiency.

[0077] By acquiring the heterogeneity characteristics, job data interaction requirements, and hybrid environment collaborative scheduling objectives of physical machine clusters and cloud environment clusters, and constructing a cross-environment data sharing space based on these characteristics, the invention achieves data interaction and sharing between physical machine and cloud environment clusters. By considering the heterogeneity characteristics, job data interaction requirements, and hybrid environment collaborative scheduling objectives, the invention enables seamless job connection between the two environments, avoiding the problem of data sharing or synchronization between environments. Unlike existing technologies where data typically flows only within a single environment, this invention allows data flow between different environments, ensuring the efficiency and collaboration of the entire system. It solves the data isolation problem between different resource environments in traditional technologies, ensuring that data can be effectively shared during job execution across environments. By establishing a sharing space, the invention improves the coordination of job execution and reduces potential resource and data conflicts during cross-environment scheduling.

[0078] By assigning a corresponding context-aware module to each sub-job unit, each sub-job unit can receive targeted resource support and real-time status monitoring. The context-aware module can monitor the execution status of the sub-job unit and the dynamic resource changes in its environment in real time, thereby ensuring that the resource allocation of each sub-job during operation is optimized. Unlike the static resource allocation mode in the prior art, this invention achieves resource adaptation and scheduling optimization during job execution through dynamic perception and real-time adjustment. It dynamically perceives and adjusts the resource requirements and status of each sub-job unit, improving the real-time performance and adaptability of resource allocation. Through the context-aware module, resources can be adjusted in a timely manner according to changes in the environment, making job execution more stable and efficient. Based on the real-time collection of the corresponding sub-job unit's running status parameters by each context-aware module, The system collects dynamic resource change data of the environment and generates corresponding job scheduling optimization factors based on each operating status parameter and dynamic resource change data. By collecting the operating status and resource change data of sub-job units in real time, it can provide accurate and timely optimization factors for job scheduling. The introduction of this dynamic optimization factor can adjust the job scheduling strategy in real time according to the changes in resources and job execution status, avoiding the shortcomings of static and preset resource allocation strategies in traditional methods that cannot cope with changing environments. Through real-time data feedback, the invention can accurately grasp the changes in demand during job execution, thereby maximizing resource utilization and job completion efficiency. By dynamically adjusting the job scheduling strategy, resource utilization is more efficient, avoiding the resource waste caused by static scheduling, and can respond to system load changes in real time, ensuring efficient execution of jobs in dynamic environments.

[0079] By establishing a dependency graph among multiple sub-job units, and dynamically adjusting the execution order and resource allocation of corresponding sub-job units based on the dependency graph and each job scheduling optimization factor, this invention ensures that the execution order and resource allocation of each sub-job unit in the job are reasonable. Unlike existing technologies that may lack sufficient consideration of the dependencies between subtasks, this invention clarifies the dependencies and execution order between each sub-job unit through the dependency graph, avoiding job execution errors or resource conflicts caused by unreasonable scheduling. The dynamic adjustment based on optimization factors further improves the accuracy and efficiency of resource scheduling. By optimizing the execution order of sub-jobs through the dependency graph, this invention ensures that each subtask is executed in a reasonable manner. The system operates sequentially, thus avoiding potential execution conflicts or invalid calculations. By dynamically adjusting the execution order and resource allocation, it improves the overall efficiency of the job, preventing over-allocation or idle resources. It also updates contextual information in the cross-environment data sharing space in real time to obtain corresponding execution results. A hybrid environment job execution evaluation report is generated using multiple execution results and historical interaction data from the cross-environment data sharing space. This process not only facilitates immediate adjustments to job execution strategies but also provides crucial historical data support for subsequent job scheduling. By analyzing historical data and real-time execution results, the system can learn from the execution process. This invention learns and optimizes scheduling strategies to form an adaptive scheduling system. Unlike existing technologies that rely solely on post-execution statistical data, this invention ensures the continuity and accuracy of job scheduling optimization by combining real-time feedback and historical data. The combination of real-time updates and historical data ensures continuous optimization of job scheduling, enabling data feedback after each execution to form a closed-loop optimization process. This improves the intelligence and efficiency of job scheduling. Based on the execution evaluation report, the parameter weights of the initial hybrid environment adaptation model are optimized to obtain the final hybrid environment adaptation model, achieving seamless collaborative execution of jobs between physical machines and cloud environments. Optimizing the initial hybrid environment adaptation model through the execution evaluation report allows the model to gradually adapt to the actual environment and job requirements. Unlike existing technologies that typically rely solely on fixed algorithms or models, this invention continuously optimizes model parameters through evaluation reports, ensuring the scheduling model can self-adjust based on environmental changes and historical execution results. This improves the long-term performance and stability of the entire system. By using evaluation reports to provide feedback and optimize the model, it ensures the model continuously adapts to resource changes in different environments during multiple job executions, enhancing the model's flexibility and adaptability. This enables the job scheduling system to cope with various complex execution environments and maintain efficient collaborative execution. Through meticulous multi-step operations and real-time dynamic adjustments, this invention achieves seamless optimization of collaborative job execution between physical machines and cloud environments, avoiding the limitations of existing technologies that optimize for a single environment.It enables more efficient and flexible scheduling in hybrid environments.

[0080] In one embodiment, step S2, which involves pre-adjusting the job scheduling strategy according to the initial hybrid environment adaptation model and decomposing the job into multiple sub-job units based on the pre-adjusted job scheduling strategy, includes:

[0081] S21. Obtain the basic allocation ratio of jobs in physical machines and cloud environments based on the physical machine adaptation degree and cloud environment adaptation degree output by the initial hybrid environment adaptation model, and generate a job scheduling strategy based on the basic allocation ratio.

[0082] S22. Obtain the first computing load, first average computing power, unit time operation and maintenance cost, and first execution time of the physical machine, as well as the second computing load, second average computing power, unit time rental cost, and second execution time of the cloud environment;

[0083] S23. Obtain cross-environment data transmission time and resource load constraints, and obtain the estimated total execution time of the job based on the cross-environment data transmission time, the first computational load, the first average computational capacity, the second computational load, and the second average computational capacity.

[0084] S24. Obtain the total scheduling cost based on the unit time operation and maintenance cost, the first execution duration, the unit time rental cost, and the second execution duration, and verify and pre-adjust the job scheduling strategy based on the total scheduling cost, resource load constraints, and the estimated total job execution time.

[0085] S25. Obtain the parallelism parameter of the pre-adjusted job scheduling strategy, and decompose the job into multiple sub-job units according to the parallelism parameter using a hierarchical clustering algorithm.

[0086] As described in steps S21-S25 above, the job scheduling strategy is constrained and verified based on the total scheduling cost, resource load constraints, and estimated total job execution time. If any one of the total scheduling cost, resource load constraints, or estimated total job execution time is not satisfied, the job scheduling strategy needs to be optimized and iterated until the job scheduling strategy meets the above three constraints. After constraint, the particle swarm optimization algorithm is used to pre-adjust the parameters of the job scheduling strategy. The job execution process is parsed through abstract syntax tree, function call relationships, data dependencies, and control dependencies are extracted, and a job execution dependency matrix is ​​constructed. The dependency strength is calculated based on the dependency matrix. According to the parallelism parameter K in the pre-adjusted scheduling strategy, a hierarchical clustering algorithm is used to divide the sub-job units. Specifically, this includes calculating the similarity between job modules, selecting the K modules with the lowest dependency strength as the initial cluster centers, calculating the distance from each module to the cluster center, merging them into the nearest cluster until the number of clusters is K, and finally ensuring that the computational amount of each sub-job unit meets the preset requirements.

[0087] This invention obtains the basic allocation ratio of jobs in physical and cloud environments by using the physical machine adaptability and cloud environment adaptability output by an initial hybrid environment adaptation model. Based on this basic allocation ratio, a job scheduling strategy is generated. By using the initial hybrid environment adaptation model to output physical machine adaptability and cloud environment adaptability, the adaptability of jobs in physical and cloud environments can be accurately evaluated. Compared with existing single-environment adaptability evaluation methods, this invention comprehensively considers the characteristics of hybrid environments, enabling jobs to find the optimal allocation ratio in different environments according to their respective characteristics. This not only improves job execution efficiency but also fully utilizes the advantages of both environments (such as the stability of physical machines and the flexibility of cloud environments), thereby optimizing overall resource utilization. This is achieved by obtaining the first computational load, first average computational power, and unit-time maintenance of the physical machine. The invention estimates the total execution time of a job in a hybrid environment by taking into account factors such as cost, first execution time, second computational load, second average computing power, unit time rental cost, and second execution time in the cloud environment. This is achieved by acquiring cross-environment data transmission time and resource load constraints, and by obtaining the estimated total execution time based on the cross-environment data transmission time, first computational load, first average computing power, second computational load, and second average computing power. By integrating cross-environment transmission time, computational load, and computing power into the total execution time estimation model, the invention can accurately predict the overall execution time of a job in a hybrid environment. Unlike existing single-environment execution time estimation models, this invention comprehensively considers cross-environment interaction factors, making the job execution time more accurate. This comprehensive estimation provides a reliable basis for subsequent scheduling strategy optimization and reduces resource waste caused by execution time uncertainty.

[0088] The total scheduling cost is obtained by calculating the unit-time operation and maintenance cost, the first execution duration, the unit-time rental cost, and the second execution duration. The calculation of scheduling cost not only considers the cost of computing resources but also the operation and maintenance costs of physical machines and the rental costs of the cloud environment. This invention optimizes job scheduling schemes by comprehensively calculating various costs, maximizing resource utilization while reducing unnecessary cost expenditures. Traditional scheduling schemes often rely solely on computing power while neglecting cost control, potentially leading to resource waste or cost overruns. Therefore, this invention offers significant economic benefits, helping users reduce total costs while meeting performance requirements. The job scheduling strategy is verified and pre-adjusted based on the total scheduling cost, resource load constraints, and estimated total job execution time. During the scheduling process, the job scheduling strategy is verified and pre-adjusted based on the estimated total job execution time, cost, and resource load constraints. This ensures the stability and efficiency of the job during execution. Most existing technologies do not have this step. Pre-adjusting the strategy before scheduling allows for flexible adjustments to the job based on the actual situation of different environments, thereby avoiding resource waste and overload problems. Through pre-adjustment, potential risks and bottlenecks can be identified before actual execution, ensuring that the final scheduling strategy is more in line with actual operational needs.

[0089] By acquiring the parallelism parameters of the pre-adjusted job scheduling strategy and decomposing the job into multiple sub-job units based on these parameters using a hierarchical clustering algorithm, the resource allocation of the job in different environments is optimized through adjusting the parallelism parameters. Adjusting the parallelism significantly improves job execution efficiency, especially when facing resource bottlenecks. Reasonable parallelism settings can reduce job execution and waiting times. Compared to existing technologies, traditional scheduling algorithms may not fully consider the matching between parallelism and resource load. This invention, by meticulously adjusting the parallelism, can more flexibly respond to changes in environmental resources and optimize job scheduling performance. Decomposing the job using a hierarchical clustering algorithm can subdivide it into multiple sub-task units, thereby improving job flexibility and schedulability. In existing technologies, jobs are usually scheduled as a whole, which cannot flexibly address differences in resource requirements between different tasks. This invention, by using a hierarchical clustering method, can independently schedule each sub-job unit based on its resource requirements and execution time, maximizing resource utilization and reducing wasted job execution time. The decomposed sub-job units can be processed separately in different environments, improving scheduling accuracy and efficiency.

[0090] In one embodiment, step S3, which constructs a cross-environment data sharing space based on the heterogeneous characteristics, job data interaction requirements, and hybrid environment collaborative scheduling objectives, includes:

[0091] S31. Obtain hardware architecture difference values, network latency coefficients, storage protocol adaptability, and computing resource elasticity coefficients based on the heterogeneity characteristics, and construct an environment difference matrix based on the hardware architecture difference values, network latency coefficients, storage protocol adaptability, and computing resource elasticity coefficients.

[0092] S32. Obtain the data interaction frequency, data transmission volume, and data timeliness requirements according to the operation data interaction requirements, and construct a data transmission weight vector according to the data interaction frequency, data transmission volume, and data timeliness requirements;

[0093] S33. Obtain resource utilization target, job response time target and cost consumption target according to the hybrid environment collaborative scheduling target, and construct a scheduling priority matrix according to the resource utilization target, job response time target and cost consumption target;

[0094] S34. Input the environmental difference matrix, data transmission weight vector and scheduling priority matrix into the preset shared space basic model to generate an initial cross-environment data sharing space;

[0095] S35. Obtain the real-time data transmission rate, data consistency deviation value, and resource occupancy rate of the initial cross-environment data sharing space, and calculate the sharing space optimization coefficient based on the real-time data transmission rate, data consistency deviation value, and resource occupancy rate, wherein the calculation formula is: ;

[0096] in, This represents the shared space optimization coefficient. Indicates the real-time data transmission rate. This indicates the data consistency deviation value. Indicates resource utilization rate. Indicates the dynamic adjustment factor;

[0097] S36. The initial cross-environment data sharing space is dynamically adjusted according to the shared space optimization coefficient to obtain the target cross-environment data sharing space.

[0098] As described in steps S31-S36 above, in the calculation formula of the shared space optimization coefficient, the calculation parameters need to be normalized first to eliminate the differences in dimensions between different variables. The purpose is to ensure that all variables are on the same order of magnitude, thereby making the calculation more stable and effective. The real-time data transmission rate is obtained by obtaining the transmission rate of a preset number of consecutive times through a traffic monitoring tool, and then obtaining the ratio of the average transmission rate to the theoretical maximum transmission rate. The data consistency deviation value is obtained by comparing the hash values ​​to calculate the number of difference fields between the physical machine cluster data version and the cloud environment cluster data version. The resource utilization rate is calculated by statistically analyzing the CPU resource rate, memory resource rate, and storage resource rate occupied by the shared space. The hardware architecture difference value measures the degree of difference in hardware infrastructure between physical machine clusters and cloud environment clusters, mainly reflected in the differences in CPU architecture and memory type. It is calculated by obtaining the CPU architecture parameters and memory type parameters of the physical machine cluster and the corresponding CPU architecture parameters and memory type parameters of the cloud environment cluster, and comparing these parameters. The network latency coefficient reflects the relative degree of communication latency between the physical machine cluster and the cloud environment cluster, and is used to assess the impact of network transmission performance on cross-environment data sharing. It is obtained by repeatedly executing the ping command to test the communication latency between the two clusters, obtaining a series of latency data, and then calculating the network latency coefficient based on this data. Protocol compatibility indicates the degree of compatibility between the storage protocols supported by the physical machine cluster and the cloud environment cluster. A higher compatibility score results in smoother data storage interaction. This is achieved by first determining the set of storage protocols supported by the physical machine cluster and the cloud environment cluster, finding the intersection of these two sets, and then calculating the storage protocol compatibility score based on protocol version compatibility. The computational resource elasticity coefficient reflects the difference in flexibility and efficiency between the physical machine cluster and the cloud environment cluster in terms of computational resource expansion. A higher elasticity coefficient indicates more flexible and efficient cluster resource adjustments. This is calculated by obtaining the resource expansion time and maximum resource adjustment range for both the physical machine cluster and the cloud environment cluster, and then using this data. Interaction frequency refers to the number of times a job interacts with data between a physical machine cluster and a cloud environment cluster per unit of time. It reflects the frequency of data interaction and is determined by statistically analyzing the number of interactions between the two clusters per unit of time. Data transmission volume represents the amount of data transmitted during a single data interaction and is used to assess the data transmission load. It is obtained by calculating the average amount of data transmitted per interaction between the physical machine cluster and the cloud environment cluster. Data timeliness requirements reflect the maximum allowable latency of data transmission and demonstrate the data's sensitivity to transmission speed. It is determined by considering the time constraints imposed on data transmission by the job to determine the maximum allowable transmission latency, thereby measuring the data timeliness requirements.The resource utilization target measures how close the current resource utilization of the physical machine cluster and the cloud environment cluster is to the target utilization. The goal is to ensure full and rational resource utilization. This is achieved by acquiring the current resource utilization of the physical machine cluster, the current resource utilization of the cloud environment cluster, and the set target utilization, and calculating priority parameters related to the resource utilization target through comparison. The job response time target reflects the comparison between job response time in a hybrid environment and a single environment. The goal is to optimize job response speed in a hybrid environment. This is achieved by statistically analyzing the average response time of jobs in a single physical machine environment, the average response time in a single cloud environment, and the target response time in a hybrid environment. Based on this data, priority parameters related to the job response time target are calculated. The cost consumption target reflects the degree to which the cost of job operation in a hybrid environment matches the target cost. The goal is to reduce costs while ensuring performance. This is achieved by acquiring the unit time cost of the physical machine cluster, the unit time cost of the cloud environment cluster, and the set target cost, and calculating priority parameters related to the cost consumption target based on the current resource utilization of both clusters.

[0099] This invention obtains hardware architecture difference values, network latency coefficients, storage protocol compatibility, and computing resource elasticity coefficients by leveraging heterogeneity characteristics. It then constructs an environment difference matrix based on these parameters. Traditional job scheduling methods typically focus only on resource scheduling within a single environment, lacking consideration for differences in hardware architecture, network latency, and storage protocols across environments (such as physical machines and cloud environments). This invention, by introducing parameters such as hardware architecture difference values, network latency coefficients, storage protocol compatibility, and computing resource elasticity coefficients, enables a quantitative assessment of differences between different environments. This allows the system to perform more accurate resource allocation based on varying hardware and network characteristics. Source scheduling and optimization improve overall resource utilization and job execution efficiency. Compared with traditional methods, this parameterized approach can overcome the challenges brought about by resource differences between environments, providing more scientific support for multi-environment operations. The construction of the environment difference matrix allows the system to systematically model the differences between different environments and analyze the impact of different environments on job execution. This matrix visualizes the environmental differences such as hardware, network, storage, and computing resources, and can provide a comprehensive view of resource adaptability, latency, and elasticity. Unlike the existing technology that optimizes a single environment, the use of the environment difference matrix can better coordinate resource allocation in mixed environments and improve the system's cross-environment adaptability and scheduling flexibility.

[0100] This invention obtains data interaction frequency, data transmission volume, and data timeliness requirements by identifying the data interaction needs for each task. Based on these requirements, a data transmission weight vector is constructed. This refinement of task data interaction needs captures the interaction frequency, transmission volume, and timeliness requirements, which is crucial for optimizing cross-environment data sharing. In existing technologies, data transmission generally fails to fully consider differences in timeliness and frequency, leading to low data sharing efficiency or excessive latency. By analyzing these key parameters, data transmission strategies can be finely adjusted for the data needs of different tasks, improving overall system performance, ensuring timely response while reducing data latency, and meeting more complex cross-environment data exchange needs. Constructing a data transmission weight vector allows the system to assign weights to the transmission requirements of different tasks, thereby optimizing data transmission priority. The core of this step lies in quantifying the priority of each task through weights, combined with data interaction needs, effectively avoiding issues in mixed environments. When transmission bottlenecks or resource conflicts occur, compared to existing methods that rely solely on fixed transmission strategies, the introduction of weight vectors enables the system to dynamically adjust resource allocation based on the specific needs of the job, improving the flexibility and efficiency of data transmission. By using a hybrid environment collaborative scheduling objective, resource utilization, job response time, and cost consumption objectives are obtained. A scheduling priority matrix is ​​then constructed based on these objectives. The construction of the scheduling priority matrix considers the comprehensive factors of resource utilization, job response time, and cost consumption objectives. This matrix not only considers the physical requirements of resources but also incorporates multi-dimensional objectives of cost control and response time optimization. It can comprehensively weigh the relationship between different objectives and prioritize the scheduling of the most urgent and important jobs. Unlike existing single-objective optimization methods, the scheduling priority matrix provides a multi-objective, comprehensive scheduling framework that can handle more complex job scheduling scenarios and improve the accuracy and execution effect of cross-environment job scheduling.

[0101] By inputting the environmental difference matrix, data transmission weight vector, and scheduling priority matrix into a preset shared space base model, an initial cross-environment data sharing space is generated. This shared space integrates information on environmental differences, data interaction requirements, and scheduling priorities, ensuring that the system can perform balanced data sharing and resource scheduling across various environments. Compared to existing technologies that directly perform data sharing or scheduling, the initial shared space model provides more dynamic and personalized processing strategies, ensuring efficient data sharing and job coordination between different environments. This is achieved by obtaining the real-time data transmission rate of the initial cross-environment data sharing space. This invention dynamically adjusts the shared space configuration by considering data consistency deviation and resource utilization, and calculates the shared space optimization coefficient based on real-time data transmission rate, data consistency deviation, and resource utilization. The acquisition of these real-time parameters allows for real-time evaluation of system performance during execution, timely identification and handling of potential bottlenecks or problems, thereby ensuring data transmission consistency and efficient utilization of system resources. The calculation of the shared space optimization coefficient, by comprehensively considering data transmission rate, data consistency, and resource utilization, provides a basis for fine-tuning the shared space. Unlike the static configuration methods of existing technologies, this invention continuously adjusts the shared space configuration through dynamic calculation of the optimization coefficient to adapt to environmental changes and operational requirements. This real-time adjustment can optimize resource allocation, reduce transmission latency, and improve job execution efficiency based on specific operating conditions. By dynamically adjusting the initial cross-environment data sharing space through the shared space optimization coefficient, the target cross-environment data sharing space is obtained. Through dynamic adjustment of the shared space, the configuration of the data sharing space can be optimized in a timely manner according to the real-time operating status and environmental differences, ensuring that cross-environment jobs can work together efficiently and seamlessly. Compared with the traditional static configuration of the shared space, the dynamic adjustment method can adapt to different environments, job loads, and network conditions, improve the system's flexibility and resource utilization efficiency, and ensure that it can still provide high consistency and low latency job execution performance in complex hybrid environments.

[0102] In one embodiment, step S4, which generates a corresponding job scheduling optimization factor based on each of the said operating status parameters and dynamic resource change data, includes:

[0103] S41. Obtain the sub-job real-time response delay, task dependency, and calculated load fluctuation coefficient of the running status parameters, and construct a sub-job status evaluation matrix based on the sub-job real-time response delay, task dependency, and calculated load fluctuation coefficient.

[0104] S42. Obtain the environmental resource utilization rate, resource elastic scaling rate, cross-environment data transmission bandwidth, and resource competition coefficient of the dynamic resource change data, and construct an environmental resource adaptability vector based on the environmental resource utilization rate, resource elastic scaling rate, cross-environment data transmission bandwidth, and resource competition coefficient.

[0105] S43. Generate initial scheduling optimization factors based on the environmental resource adaptability vector and the sub-job status evaluation matrix using a multi-dimensional weighted fusion algorithm;

[0106] S44. Obtain the sub-job completion time deviation value, resource waste rate and cross-environment data synchronization error corresponding to the initial scheduling optimization factor, and obtain the factor correction coefficient based on the sub-job completion time deviation value, resource waste rate and cross-environment data synchronization error;

[0107] S45. The initial scheduling optimization factor is iteratively optimized according to the factor correction coefficient to obtain the target job scheduling optimization factor.

[0108] As described in steps S41-S45 above, the sub-job real-time response latency measures the deviation between the sub-job task response time and the theoretical standard response time, reflecting the real-time processing efficiency of the sub-job. This latency can be calculated by collecting the task response time of the sub-job for a preset number of consecutive times using a monitoring tool. Task dependency reflects the degree to which the sub-job depends on its predecessor tasks, including the proportion of completed predecessor tasks and the urgency of the dependency. This can be determined by statistically analyzing the total number and completed number of predecessor tasks dependent on the sub-job, combined with the time difference between the current sub-job and the latest dependent task, and the dependency urgency coefficient. The load fluctuation coefficient reflects the fluctuation of the CPU utilization rate of the sub-job within a certain period, reflecting the stability of the computing load. This can be calculated by collecting the CPU utilization rate of the sub-job for multiple time periods within a preset time period. Environmental resource utilization comprehensively reflects the CPU and memory resource usage of the physical machine cluster and the cloud environment cluster. This can be calculated by separately collecting the CPU utilization rate and memory utilization rate of the physical machine cluster and the cloud environment cluster, and then performing a comprehensive calculation. Resource elastic scaling rate measures the speed at which resources in the physical machine cluster and the cloud environment cluster can be expanded or reduced. This can be calculated by... The time taken to expand / shrink resources in both the physical machine cluster and the cloud environment cluster is obtained separately. The cross-environment data transmission bandwidth represents the effective bandwidth for bidirectional data transmission between the physical machine cluster and the cloud environment cluster, reflecting data transmission capability. This can be determined by testing the bidirectional transmission bandwidth between the physical machine and the cloud environment for a preset number of consecutive times using the iperf tool. The resource contention coefficient reflects the intensity of competition for similar resources with the target sub-job in the current hybrid environment. It can be calculated by statistically analyzing the number of jobs competing for similar resources with the target sub-job in the current hybrid environment and combining this with the overall resource utilization rate. The sub-job completion time deviation value measures the degree of deviation between the actual completion time and the planned completion time of the sub-job. This can be calculated by recording the actual completion time and the planned completion time of the sub-job. The resource waste rate reflects the proportion of the actual resource usage of a sub-job exceeding the theoretical resource requirement. This can be calculated by statistically analyzing the actual resource usage and the theoretical resource requirement of the sub-job. The cross-environment data synchronization error reflects the degree of difference in sub-job data versions between the physical machine cluster and the cloud environment cluster. This can be determined by comparing timestamps to calculate the duration of the difference in sub-job data versions between the physical machine and the cloud environment.

[0109] The steps for generating the initial scheduling optimization factor based on the environmental resource adaptability vector and the sub-job status evaluation matrix using a multi-dimensional weighted fusion algorithm include: calculating the matrix norm of the sub-job status evaluation matrix and the L2 norm of the environmental resource adaptability vector; obtaining the matrix-vector fusion weights using the L2 norm and matrix norm; calculating the correlation between the sub-job status evaluation matrix and the environmental resource adaptability vector using the Pearson correlation coefficient; obtaining the matrix column vector stacking operation and transpose operation of the sub-job status evaluation matrix; and then calculating the initial scheduling optimization factor. The calculation formula is as follows: ;in, Represents the initial scheduling optimization factor. Represents the matrix-vector fusion weights. This represents the matrix column vector stacking operation. This indicates the transpose operation. Represents the environmental resource adaptability vector. Indicates the degree of relevance. Represents the matrix norm. The L2 norm is used to represent the L2 norm. In the calculation formula of the initial scheduling optimization factor, the calculation parameters need to be normalized first to eliminate the difference in the dimensions between different variables. The purpose is to ensure that all variables are on the same order of magnitude, so as to make the calculation more stable and efficient. The matrix norm, also known as the Frobenius norm, is a norm for matrices and is used to measure the size of the matrix. It is calculated as the square root of the sum of the squares of all the elements of the matrix. The L2 norm is a norm for vectors, also known as the Euclidean norm, and is used to measure the length of a vector. It is calculated as the square root of the sum of the squares of all the elements of the vector.

[0110] This invention obtains the real-time response latency, task dependency, and computational load fluctuation coefficient of sub-jobs as runtime status parameters, and constructs a sub-job status evaluation matrix based on these parameters. Real-time response latency is a key factor in measuring the execution efficiency and performance of sub-jobs. In a hybrid environment, different computing resources (such as physical machines, cloud environments, etc.) can lead to different response times. The task dependency of a job indicates the relationship between different tasks and their execution order. The computational load fluctuation coefficient can measure the fluctuation of the load on different computing resources. In a multi-environment mixed computing resource environment, load fluctuation may lead to uneven resource utilization. By comprehensively evaluating the real-time response latency, task dependency, and computational load fluctuation coefficient of sub-jobs, a sub-job status evaluation matrix is ​​generated. This matrix provides a multi-dimensional basis for subsequent scheduling optimization. Compared with existing technologies, by constructing such a matrix, the status of each sub-job can be comprehensively and accurately evaluated, thereby providing a more scientific reference for optimizing scheduling decisions. This is often overlooked in existing technologies, which leads to job scheduling failing to fully consider the correlation between jobs and resource utilization.

[0111] By acquiring dynamic resource change data, including environmental resource utilization rate, resource elastic scaling rate, cross-environment data transmission bandwidth, and resource contention coefficient, and constructing an environmental resource adaptability vector based on these metrics, we can determine whether resources are being overused or underused. Environmental resource utilization rate reflects the actual utilization of resources. By monitoring environmental resource utilization rate, we can determine whether resources are being overused or underused, and adjust resource allocation in a timely manner to avoid resource waste or shortages. In dynamic environments, the elastic scaling capability of resources is crucial for ensuring system stability and efficiency. By evaluating the resource elastic scaling rate, we can predict the speed of resource expansion or contraction, thereby avoiding performance bottlenecks caused by excessively slow or fast resource expansion. In multi-environment hybrid computing, cross-environment data transmission bandwidth... Bandwidth directly impacts data synchronization efficiency. By dynamically monitoring bandwidth, data transmission paths and rates can be optimized, improving the efficiency of cross-environment data exchange and avoiding bandwidth bottlenecks. The resource contention coefficient reflects the degree of competition among different jobs for the same resource. By monitoring the resource contention coefficient, resources can be rationally allocated, avoiding resource contention and conflicts, and improving the concurrent utilization of resources. By integrating the utilization rate, elastic scaling rate, bandwidth, and resource contention coefficient of environmental resources, an environmental resource adaptability vector is generated. This vector can comprehensively reflect the adaptability of resources in different environments, which helps to optimize job scheduling strategies. Compared with existing technologies, the lack of comprehensive assessment of environmental resource adaptability leads to job scheduling failing to consider the actual adaptability of resources across multiple environments, which may result in resource waste or low scheduling efficiency.

[0112] An initial scheduling optimization factor is generated by using an environmental resource adaptability vector and a sub-job status evaluation matrix through a multi-dimensional weighted fusion algorithm. By weighted fusion of the sub-job status evaluation matrix and the environmental resource adaptability vector, an initial scheduling optimization factor that comprehensively considers multiple dimensions can be generated. Compared to existing technologies, which typically optimize scheduling based on only a single dimension and neglect the comprehensive consideration of multiple factors, this invention, through weighted fusion, can more accurately adjust job scheduling strategies, improving scheduling accuracy and resource utilization. This is achieved by obtaining the sub-job completion time deviation, resource waste rate, and cross-environment data synchronization error corresponding to the initial scheduling optimization factor, and then... Cross-environment data synchronization error is used to obtain factor correction coefficients. The initial scheduling optimization factor is then iteratively optimized using these correction coefficients to obtain the target job scheduling optimization factor. Based on the sub-job completion time deviation, resource waste rate, and cross-environment data synchronization error, correction coefficients are calculated to correct the initial scheduling optimization factor. This step dynamically adjusts the optimization factor, avoiding unnecessary errors during job execution and ensuring a more accurate and efficient scheduling strategy. By iteratively optimizing the initial scheduling optimization factor, iterative optimization can gradually converge to the optimal scheduling scheme. Compared to the application of fixed scheduling strategies in existing technologies, iterative optimization can cope with dynamically changing environmental conditions, improving the adaptability and optimization effect of job scheduling.

[0113] In one embodiment, step S5, which dynamically adjusts the execution order and resource allocation of corresponding sub-job units based on the dependency graph and each job scheduling optimization factor, includes:

[0114] S51. Extract all predecessor job sets and successor job sets for each sub-job unit from the dependency graph, and obtain the dependency strength of the corresponding sub-job unit based on each predecessor job set.

[0115] S52. Obtain the execution priority of the corresponding sub-job unit according to each set of subsequent jobs and the dependency strength, and sort all sub-job units from high to low according to the execution priority to obtain the initial execution order;

[0116] S53. Determine whether the initial execution order satisfies the execution timing constraints in the dependency graph;

[0117] If the initial execution order does not satisfy the execution timing constraints in the dependency graph, the sub-job units that violate the execution timing constraints will be adjusted until all their predecessor jobs have been executed to obtain the final execution order;

[0118] S54. Determine the target execution environment of the corresponding sub-job unit according to each job scheduling optimization factor, and obtain the total available computing resources, total available storage resources and total available network bandwidth of the target execution environment;

[0119] S55. Obtain the resource allocation ratio for each sub-job unit, and obtain the initial computing resource allocation amount based on the resource allocation ratio and the total available computing resources;

[0120] S56. Obtain the initial storage resource allocation amount based on the resource allocation ratio and the total available storage resources, and obtain the initial network bandwidth allocation amount based on the resource allocation ratio and the total available network bandwidth.

[0121] S57. Based on the data interaction weights in the dependency graph, the computational resource allocation, storage resource allocation, and network bandwidth allocation for sub-job units with strong dependencies are balanced and adjusted, and the adjusted computational resource allocation, storage resource allocation, and network bandwidth allocation are taken as the final resource allocation for the sub-job unit.

[0122] As described in steps S51-S57 above, the resource allocation ratio of a sub-job unit is determined by dividing the job scheduling optimization factor of that sub-job unit by the sum of the job scheduling optimization factors of all sub-job units allocated to the target execution environment. The dependency strength is calculated as follows: when a predecessor job exists, the data interaction weight (range 0-1) between the sub-job unit and each predecessor job is multiplied by the corresponding predecessor job's job scheduling optimization factor, the sum is obtained, and then divided by the number of predecessor jobs. When no predecessor job exists, the dependency strength is 0. The execution priority is calculated as follows... The resource allocation ratio for this sub-job unit is calculated by multiplying its job scheduling optimization factor (1 plus the sum of dependency strength) and time sensitivity coefficient. The time sensitivity coefficient is 1.5 for real-time execution, 1.0 for normal execution, and 0.5 for deferred execution. The resource allocation ratio for this sub-job unit is calculated by dividing its job scheduling optimization factor by the sum of the job scheduling optimization factors of all sub-job units allocated to the target execution environment. The initial resource allocation is calculated by multiplying the resource allocation ratio, the total available computing resources of the target execution environment, and... The computational resource requirement of this sub-job unit is multiplied by the sum of the computational resource requirements of all sub-job units allocated to the target execution environment. The initial storage resource allocation is calculated by multiplying the resource allocation ratio, the total available storage resources of the target execution environment, and the storage resource requirement of this sub-job unit by the sum of the storage resource requirements of all sub-job units allocated to the target execution environment. The initial network bandwidth allocation is calculated by multiplying the resource allocation ratio, the total available network bandwidth of the target execution environment, and the network bandwidth requirement of this sub-job unit by the sum of the network bandwidth requirements of all sub-job units allocated to the target execution environment. A strong dependency relationship means that the data interaction weight is not less than 0.8. The method for balancing and adjusting the computational resource allocation, storage resource allocation, and network bandwidth allocation is to multiply the initial computational resource allocation, initial storage resource allocation, and initial network bandwidth allocation by the corresponding resource balancing adjustment amount, multiplied by the ratio of the data interaction weight of this sub-job unit with the current associated job to the total data interaction weight of its data with all related jobs, and then summed to obtain the adjusted computational resource allocation, storage resource allocation, and network bandwidth allocation.

[0123] This invention extracts all predecessor and successor sets of each sub-job unit from a dependency graph. Extracting these sets clarifies the dependencies between jobs, and the dependency graph accurately displays the execution order of job units, ensuring its rationality. This effectively avoids situations where dependencies are not met during execution, ensuring jobs are executed in the correct order. Traditional methods often overlook complex dependencies between job units, potentially leading to unmanageable errors or unnecessary delays. This invention, by precisely dividing dependencies, more effectively controls the execution order of jobs. It obtains the dependency strength of each sub-job unit based on its predecessor set and the execution priority of each sub-job unit based on its successor set and dependency strength, then executes the sub-job units according to their priority from high to low. The initial execution order is obtained by sorting all sub-job units. Obtaining the dependency strength can help identify which job units have a high degree of dependence on the predecessor job during execution. This helps to prioritize the scheduling of jobs with strong dependencies, reduce possible waiting time, and thus improve overall execution efficiency. In the prior art, the dependency strength between jobs may not be fully considered, resulting in insufficient precision in the handling of dependency relationships, which may lead to low scheduling efficiency. However, by calculating the dependency strength, the execution order of jobs can be arranged more accurately. By considering the set of successor jobs and the dependency strength, the execution priority of job units can be dynamically adjusted. The adjustment of priority ensures the effective use of resources and avoids the delayed execution of jobs with high dependency strength. Traditional methods usually only rely on static execution order or priority and lack the ability to consider dynamic dependency relationships. By adjusting the priority in real time, this invention can schedule tasks more flexibly and efficiently.

[0124] By determining whether the initial execution order satisfies the execution timing constraints in the dependency graph, if the initial execution order does not satisfy the execution timing constraints in the dependency graph, the sub-job units that violate the execution timing constraints are adjusted until all their predecessor jobs are completed to obtain the final execution order. The timing constraint judgment ensures the accuracy of task scheduling. If the initial execution order violates the timing constraints, timely adjustment can avoid system errors or conflicts and ensure that the final execution order meets the requirements. In the prior art, many methods cannot flexibly handle changes in timing constraints, leading to execution errors or conflicts. However, by verifying the timing constraints, this invention ensures the legality of the execution order and avoids unnecessary errors and conflicts. By adjusting the sub-job units that violate the timing constraints until the predecessor jobs are completed, dependency conflicts can be avoided during job execution. This adjustment method ensures the rationality of the execution order and improves the stability and efficiency of the system. The prior art usually lacks a dynamic adjustment mechanism for timing conflicts. This invention ensures the accuracy and flexibility of job scheduling through this intelligent adjustment method.

[0125] The target execution environment for each sub-job unit is determined by a job scheduling optimization factor. This factor flexibly adjusts the job execution environment based on different environmental conditions (such as computing resources, storage resources, and network bandwidth) to ensure the job runs in the most suitable environment. This not only improves resource utilization but also ensures job stability. Traditional methods may only optimize jobs in a single environment, neglecting the rational allocation of resources in mixed environments. This invention's multi-environment support allows jobs to be optimized according to the characteristics of different environments, solving the deficiency of existing technologies in effectively supporting mixed environments. It also obtains the total available computing resources, total available storage resources, and total available network bandwidth of the target execution environment. By obtaining the resource allocation ratio for each sub-job unit, the initial computing resource allocation is obtained based on the resource allocation ratio and the total available computing resources. The initial storage allocation is also obtained based on the resource allocation ratio and the total available storage resources. This invention allocates resources based on resource allocation ratios and total available network bandwidth, obtaining the initial network bandwidth allocation and the total available resources of the target execution environment. This allows for precise understanding of the resource configuration of each environment, aiding in more rational resource allocation decisions. This ensures that resources are efficiently allocated to the required work units, thereby improving overall job execution efficiency. Traditional technologies may neglect precise understanding of environmental resources, leading to uneven resource allocation or waste. This invention can more finely adjust resource configuration, improving resource utilization efficiency. Through the initial allocation of computing, storage, and network resources, it ensures that each work unit receives an appropriate amount of resources. This not only avoids resource waste but also effectively supports the smooth execution of jobs. Existing methods may not consider multi-dimensional resource allocation or lack fine-tuning of resource allocation ratios. This invention adjusts resource allocation from multiple perspectives, supporting task execution more rationally and effectively.

[0126] By adjusting the data interaction weights in the dependency graph, the computational, storage, and network bandwidth allocations for strongly dependent sub-job units are balanced. These adjusted allocations are then used as the final resource allocations for the sub-job units. This balanced adjustment of data interaction weights ensures more equitable resource distribution among strongly dependent job units, preventing delays or blockages due to insufficient resources. This improves the overall system efficiency. Existing technologies may not fully consider the interaction weights between jobs, leading to insufficient resource allocation for certain critical tasks. This invention, through dynamic balancing, ensures that critical jobs receive sufficient resource support. By using the balanced resource allocations as the final execution resources, it ensures that each job unit can execute efficiently with sufficient resources, avoiding instability in task execution caused by uneven resource allocation. Traditional technologies lack flexibility in resource allocation, typically employing fixed resource allocation strategies, which cannot meet the needs of multi-task and multi-environment environments. This invention, through precise scheduling and balancing, better addresses the multi-task execution problem in complex environments.

[0127] In one embodiment, step S6, which generates a hybrid environment job execution evaluation report based on multiple execution results and historical interaction data in a cross-environment data sharing space, includes:

[0128] S61. Obtain the job completion time, total resource consumption, and data transmission success rate for each of the execution results, and obtain the single-dimensional execution indicators of the corresponding sub-job unit based on the job completion time, total resource consumption, and data transmission success rate for each of the execution results.

[0129] S62. Obtain the cross-environment data transmission delay, environment switching frequency, and state synchronization error of the historical interaction data, and obtain the cross-environment collaboration index of the hybrid environment based on the cross-environment data transmission delay, environment switching frequency, and state synchronization error.

[0130] S63. Construct a comprehensive evaluation matrix based on multiple single-dimensional performance indicators and cross-environmental collaborative indicators, and determine the weight coefficient of each indicator through the analytic hierarchy process.

[0131] S64. Based on the comprehensive evaluation matrix and weighting coefficients, obtain the comprehensive evaluation value of each sub-work unit and the overall evaluation value of the mixed environment;

[0132] S65. Generate a hybrid environment operation performance evaluation report based on each comprehensive evaluation value, overall evaluation value, single-dimensional performance indicator, and cross-environmental collaboration indicator.

[0133] As described in steps S61-S65 above, the single-dimensional execution indicators include efficiency indicators, resource utilization indicators, and data interaction quality indicators; the cross-environmental collaboration indicators include environmental adaptability indicators, data consistency indicators, and state synchronization stability indicators; the evaluation report includes an analysis of the execution shortcomings of each sub-job unit, collaborative optimization suggestions for the hybrid environment, and job scheduling strategy adjustment schemes. The efficiency indicator is calculated by obtaining the actual completion time, theoretical minimum completion time (determined based on job complexity and maximum computing power of the environment), and average completion time of the environment of the sub-job unit, and by using a preset first set of weighting coefficients (the sum of the two weighting coefficients is 1). The efficiency index is calculated by multiplying the results by their respective weighting coefficients (1 minus the ratio of the difference between the actual completion time and the theoretical minimum completion time to the theoretical minimum completion time) and (1 minus the ratio of the difference between the actual completion time and the environmental average completion time to the environmental average completion time). The resource utilization rate index is calculated by obtaining the actual computing resource consumption, actual storage resource consumption, and actual network bandwidth consumption of the sub-job unit, as well as the allocated total computing resources, total storage resources, and total network bandwidth. The calculations include: computational resource utilization (the ratio of actual computational resource consumption to the total allocated computational resources), storage resource utilization (the ratio of actual storage resource consumption to the total allocated storage resources), and network bandwidth utilization (the ratio of actual network bandwidth consumption to the total allocated network bandwidth). Based on a preset second set of weighting coefficients (the sum of the three weighting coefficients is 1, dynamically adjusted according to the job type: 0.6 for computational resource utilization in compute-intensive jobs, 0.6 for storage resource utilization in storage-intensive jobs, and 0.6 for network bandwidth utilization in transmission-intensive jobs), the computational resource utilization is... The resource utilization rate, storage resource utilization rate, and network bandwidth utilization rate are multiplied by their respective weight coefficients and then summed to obtain the resource utilization rate index. The data interaction quality index is calculated by obtaining the number of successful cross-environment data transmissions, the total number of transmissions, and the data transmission integrity score (value range 0-1, calculated based on checksum matching degree) of the sub-job unit. Based on the preset third set of weight coefficients (the sum of the two weight coefficients is 1, and the first weight coefficient is 0.7 to give priority to the transmission success rate), the ratio of the number of successful transmissions to the total number of transmissions and the data transmission integrity score are multiplied by their respective weight coefficients and then summed to obtain the data interaction quality index.

[0134] The environment adaptability index is calculated using the execution time percentage of the physical environment (the ratio of physical machine execution time to total execution time), the execution time percentage of the cloud environment (the ratio of cloud environment execution time to total execution time, with the sum of the physical machine environment execution time percentage and the cloud environment execution time percentage equal to 1), the physical machine environment adaptability score (normalized to 0-1 based on the physical machine environment adaptability calculation result in claim 2), and the cloud environment adaptability score (similarly normalized to 0-1). The environment adaptability index is obtained by multiplying the physical machine environment execution time percentage by the physical machine environment adaptability score, multiplying the cloud environment execution time percentage by the cloud environment adaptability score, and then adding the two products. The data consistency index is calculated by obtaining the timestamp deviation of cross-environment data synchronization (the maximum time difference among multiple synchronizations). The data consistency index is calculated by multiplying the data content deviation (based on the difference rate of hash value comparison, with a value range of 0-1) and the preset maximum allowable time deviation and maximum allowable content deviation. The state synchronization stability index is calculated by obtaining the error values ​​of n consecutive state synchronizations (based on the Euclidean distance of the state vectors) and calculating the standard deviation of these error values ​​(first calculating the average error, then calculating the square root of the sum of the squares of the differences between each error value and the average error, divided by n). The state synchronization stability index is obtained by obtaining the preset maximum allowable error standard deviation and calculating the ratio of (1 minus the error standard deviation to the maximum allowable error standard deviation).

[0135] This invention obtains the job completion time, total resource consumption, and data transmission success rate for each execution result, and then derives a single-dimensional execution indicator for the corresponding sub-job unit based on these metrics. By acquiring these data, the efficiency and resource consumption of each execution result can be comprehensively evaluated. The job completion time reflects the time required to execute the task, the total resource consumption measures the computing and storage resources used, and the data transmission success rate directly reflects the reliability of data transmission. Compared with existing technologies, this multi-dimensional monitoring method provides a more comprehensive performance evaluation of the job execution process, avoiding the limitation of considering only time or... A simple method for evaluating resource consumption, particularly adaptable to complex situations in hybrid environments, enhances the system's resource optimization and performance scheduling capabilities. Obtaining single-dimensional execution indicators for sub-job units can accurately quantify the efficiency of each execution unit, providing data support for subsequent comprehensive evaluation. This process helps to break down complex job processes into multiple easily analyzable and optimizable units. The execution effect of each unit can be evaluated and optimized independently. Compared to existing technologies that may only rely on simple job execution feedback (such as total execution time or resource consumption), this refined evaluation method can conduct a more comprehensive analysis of each sub-job unit from multiple perspectives, which is conducive to discovering potential bottlenecks and optimization points, thereby improving the overall job execution efficiency and resource utilization.

[0136] By acquiring historical interaction data on cross-environment data transmission latency, environment switching frequency, and state synchronization error, and based on these factors, cross-environmental collaboration indicators for hybrid environments are obtained. Cross-environment data transmission latency, environment switching frequency, and state synchronization error are key factors affecting job execution performance in hybrid environments. Acquiring this data helps in-depth analysis of potential problems encountered during job execution in hybrid environments, such as cross-environmental communication latency, resource waste caused by frequent environment switching, and data inconsistency caused by untimely state synchronization. This method of acquiring specific indicators for hybrid environments is lacking in existing technologies, which typically focus more on single environments. While optimizing the environment, this approach lacks consideration for interaction and switching between environments. Cross-environment collaboration metrics, by comprehensively considering the interaction and synchronization efficiency between multiple environments, can fully measure the collaboration and overall efficiency of job execution in a hybrid environment. By introducing cross-environment collaboration metrics, the coordination level between various sub-environments (such as physical machine and cloud environments) can be better evaluated, and potential interaction bottlenecks or data synchronization problems can be identified. Compared with the single environment optimization scheme in traditional technologies, this method can better solve the collaboration problem in a hybrid environment, improve the flexibility and efficiency of cross-environment resource allocation, and accurately evaluate the collaboration effect in a hybrid environment, which helps to make more reasonable decisions on job sharing and data synchronization between different environments.

[0137] A comprehensive evaluation matrix is ​​constructed by integrating multiple single-dimensional execution indicators and cross-environmental collaborative indicators into a unified evaluation framework. This matrix allows for systematic analysis of various aspects of job execution, providing a holistic view of overall performance and clear decision-making basis for optimization. Compared to traditional single-indicator evaluation, it integrates multiple factors, resulting in more accurate and comprehensive evaluation results. Especially in mixed environments, the comprehensive consideration of multiple dimensions effectively avoids over-optimization of certain single dimensions leading to neglect of others, thus achieving optimal balance in global optimization. The weight coefficients of each indicator are determined using the Analytic Hierarchy Process (AHP). AHP objectively determines the weight coefficients based on the importance of each indicator, incorporating the influence of different indicators into the comprehensive evaluation. Weight allocation using AHP allows for flexible adjustment of weight coefficients based on actual business needs and environmental characteristics, making the evaluation model more aligned with real-world application scenarios. The comprehensive evaluation value of each sub-job unit and the mixed evaluation value are obtained based on the comprehensive evaluation matrix and weight coefficients. The overall environmental assessment, based on a weighted coefficient-based comprehensive evaluation, provides specific comprehensive assessment values ​​for each sub-work unit and the hybrid environment. These assessment values ​​help decision-makers quickly understand the system's operational status and job execution quality, enabling them to take corresponding optimization measures. Compared to traditional single-dimensional assessments or simple evaluation methods, this comprehensive assessment method provides more accurate quantitative results, making the assessment process clearer and more comprehensive, and helping to identify potential performance bottlenecks and collaboration issues. A hybrid environment job execution assessment report is generated by combining each comprehensive assessment value, the overall assessment value, single-dimensional execution indicators, and cross-environment collaboration indicators. This report provides managers or operators with a detailed and systematic summary of job execution results, including a comprehensive analysis of performance across various dimensions, resource consumption, and synergistic effects. This provides a necessary basis for subsequent job optimization and adjustments. Compared to traditional methods of simply reporting or logging, the assessment report, by comprehensively considering multiple indicators, more comprehensively and accurately reflects the actual situation of job execution, helping managers to identify problems and make corresponding adjustments in a timely manner.

[0138] In one embodiment, step S6, which optimizes the parameter weights of the initial hybrid environment adaptation model based on the performance evaluation report to obtain the final hybrid environment adaptation model, includes:

[0139] S67. Calculate the parameter weight deviation value of the initial model based on the single-dimensional execution indicators and cross-environmental collaboration indicators in the execution evaluation report. The parameter weight deviation value is used to characterize the degree of deviation between the model's predicted value and the actual evaluation value.

[0140] S68. Construct an objective optimization function based on the parameter weight deviation value, wherein the objective optimization function is constrained by minimizing the deviation value and maximizing the model generalization ability.

[0141] S69. The adaptive particle swarm optimization algorithm is used to solve the objective optimization function to obtain the parameter weight adjustment coefficients, and the parameter weight matrix of the initial hybrid environment adaptation model is iteratively updated according to the parameter weight adjustment coefficients until the preset convergence condition is met.

[0142] S610. The converged model is determined as the final hybrid environment adaptation model, which is used to predict the execution adaptation degree of the sub-job unit in the hybrid environment.

[0143] As described in steps S67-S610 above, the step of calculating the parameter weight deviation value of the initial model includes extracting environmental feature parameters and operation attribute parameters from the initial hybrid environment adaptation model and obtaining the initial parameter weight matrix; calculating the predicted adaptability of the sub-operation unit based on the initial model; summing the results by multiplying each environmental feature parameter and operation attribute parameter by its corresponding weight; extracting the actual comprehensive evaluation value of the sub-operation unit from the evaluation report; calculating the single sample deviation value; dividing the absolute value of the difference between the predicted adaptability and the actual comprehensive evaluation value by the actual comprehensive evaluation value (the actual comprehensive evaluation value is not 0); obtaining the single sample deviation values ​​of all sub-operation units in the evaluation report; calculating the average deviation value; summing all the single sample deviation values ​​and dividing by the total number of sub-operation units; calculating the environmental adaptability index and data consistency index in the cross-environmental collaboration index; multiplying the environmental adaptability index by 0.6 and the data consistency index by 0.4 by calculating the collaboration correction coefficient; adding the two results together; and multiplying the average deviation value by (1 minus the collaboration correction coefficient) by calculating the parameter weight deviation value.

[0144] The steps for constructing the objective optimization function based on the parameter weight deviation value include: taking the parameter weight deviation value as the first optimization objective; setting the first objective function as the parameter weight deviation value; introducing a model complexity penalty term; calculating the second objective function by multiplying the sum of the absolute values ​​of all parameter weights by the first penalty coefficient; multiplying the difference between the maximum and minimum parameter weights by the second penalty coefficient; adding the two results; calculating the generalization ability index based on the overall evaluation value in the evaluation report by multiplying the overall evaluation value by (1 minus 0.5 times the standard deviation of the comprehensive evaluation value); setting the third objective function as (1 minus the generalization ability index); constructing a multi-objective optimization function by multiplying the first objective function by the first objective weight, the second objective function by the second objective weight, and the third objective function by the second objective weight, and then summing them; and setting constraints that all parameter weights are positive and less than 1; the sum of all parameter weights is 1; and the difference between any two parameter weights does not exceed 0.4.

[0145] The steps of using an adaptive particle swarm optimization algorithm to solve the objective optimization function to obtain parameter weight adjustment coefficients, and iteratively updating the parameter weight matrix of the initial hybrid environment adaptation model according to the parameter weight adjustment coefficients until a preset convergence condition is met include: using the parameter weight matrix as a particle position vector; randomly generating a preset number of initial particle positions; calculating the fitness value (i.e., the function value of the multi-objective optimization function) of each particle; and recording the individual optimal position of each particle and the global optimal position of all particles. For updating particle velocity, it can be calculated based on the current inertia weight, the current particle velocity, the learning factor, a random number, the difference between the individual optimal position and the current position, and the difference between the global optimal position and the current position. The inertia weight changes with the number of iterations. For updating particle position, the current particle position can be added to the updated velocity. Positions exceeding the constraint range are then... The algorithm performs truncation (adjusting to the constraint range). It checks if the iteration stopping condition is met. If so, it outputs the parameter weight adjustment coefficient corresponding to the global optimal position (i.e., the ratio of the global optimal position to the initial weight matrix). It calculates the weight matrix for each iteration by multiplying the previous iteration's weight matrix by the corresponding parameter weight adjustment coefficient. Based on the updated weight matrix, it recalculates the model's predicted value and compares it with the actual value in the evaluation report to obtain a new deviation value. It checks if the new deviation value is less than the convergence threshold. If it is, the iteration stops. If it is not less than the convergence threshold, the parameter weight adjustment coefficient is updated based on the ratio of the new deviation value to the previous deviation value (multiplying the current adjustment coefficient by this ratio). The iteration count is increased, and the update process is repeated. The final weight matrix obtained from the iterations is used as the optimized parameter weight matrix.

[0146] This invention calculates the parameter weight deviation value of the initial model by using single-dimensional execution indicators and cross-environmental collaborative indicators from the performance evaluation report. The parameter weight deviation value characterizes the degree of deviation between the model's predicted values ​​and the actual evaluation values. By introducing single-dimensional execution indicators and cross-environmental collaborative indicators, the performance of the model in different dimensions and environments can be comprehensively evaluated. Calculating the parameter weight deviation value of the initial model accurately reflects the model's adaptability in the current environment and indicates the deviation between the model's predicted values ​​and the actual evaluation values, ensuring the accuracy of the evaluation results. Existing technologies typically focus only on optimization in a single environment (e.g., physical machine or cloud environment), ignoring the impact of different environments on model adaptability in mixed environments. This invention, by introducing cross-environmental collaborative indicators, can more comprehensively reflect adaptability in mixed environments, effectively overcoming the limitations of existing technologies. The objective optimization function is constructed using the parameter weight deviation value, with the constraints of minimizing the deviation value and maximizing the model's generalization ability. The parameter weight deviation value is used as the basis for constructing the objective optimization function. This approach simultaneously considers minimizing prediction bias and maximizing model generalization ability in the same optimization process, avoiding the shortcomings of existing technologies that can only optimize one objective at a time. In mixed environments, models face complex and variable operating conditions, and optimizing the bias value alone may not effectively improve the model's adaptability. However, by constructing an optimization function, the model's adaptability to different environments can be enhanced while minimizing the bias, improving its stability and robustness in multiple environments. By simultaneously adding constraints to minimize bias and maximize generalization ability to the objective optimization function, the final model can be guaranteed to have not only high accuracy but also strong generalization ability. This optimization method solves the overfitting problem that traditional models may encounter in complex and variable environments, ensuring the model's universality and scalability. Existing technologies often neglect the adaptation of models in multiple environments. By maximizing the constraint of generalization ability, the model can maintain good performance when migrating between different environments, solving the problem of difficulty in sharing data and state between different environments.

[0147] By employing an adaptive particle swarm optimization (PSO) algorithm to solve the objective optimization function and obtain parameter weight adjustment coefficients, the parameter weight matrix of the initial hybrid environment adaptation model is iteratively updated based on these coefficients until a preset convergence condition is met. Compared to traditional PSO algorithms, the adaptive PSO algorithm can automatically adjust the optimization strategy according to the actual situation of the problem, thus avoiding the limitations that may result from fixed parameter settings in traditional methods. In hybrid environments, due to the complexity and variability of the environment, traditional optimization algorithms may not be able to effectively handle diverse adaptation problems, while the adaptive PSO algorithm can adjust the optimization strategy based on real-time feedback in a dynamic environment, improving the optimization effect. This method effectively accelerates the convergence process through an adaptive mechanism while ensuring the accuracy of parameter weight adjustment. It can efficiently find the global optimum and avoid the trap of local optima. By updating the parameter weight matrix through multiple iterations, the adaptability of the model can be gradually adjusted in each iteration, so that the final model can better adapt to complex tasks in mixed environments. Compared with the coarse method of modifying model parameters all at once, this invention can provide more accurate optimization. Setting convergence conditions can ensure that the optimization process stops after reaching the expected goal, avoiding overfitting problems that may be caused by over-optimization. At the same time, the introduction of convergence conditions can ensure that the final output of the model has stability and can adapt to different working environments and task requirements in the long term.

[0148] By determining the converged model as the final hybrid environment adaptation model, which is used to predict the execution adaptability of sub-job units in a hybrid environment, after multiple rounds of optimization and updates, the final hybrid environment adaptation model can exhibit the best adaptability in various environments (including physical machine and cloud environments). Existing technologies usually only focus on optimization in a single environment and cannot effectively support cross-platform adaptation in hybrid environments. However, the optimization method of this invention successfully solves this problem through cross-environment collaborative design. By ensuring the maximization of model adaptability, it can effectively improve the system's working efficiency and reduce resource waste caused by model incompatibility. At the same time, this invention can realize data and state sharing between different environments, improve resource utilization, and significantly reduce operating costs, especially when working collaboratively in multiple environments.

[0149] like Figure 2 As shown, this application also provides a cross-platform, cross-cluster hybrid operating system, including:

[0150] The first construction module is used to obtain the attribute parameters of the job to be executed, the first resource characteristic parameters of the physical machine cluster and the second resource characteristic parameters of the cloud environment cluster, and to construct an initial hybrid environment adaptation model based on the first resource characteristic parameters, the second resource characteristic parameters and the attribute parameters.

[0151] The first adjustment module is used to pre-adjust the job scheduling strategy according to the initial hybrid environment adaptation model, and to decompose the job into multiple sub-job units based on the pre-adjusted job scheduling strategy.

[0152] The second construction module is used to obtain the heterogeneity characteristics, job data interaction requirements and hybrid environment collaborative scheduling objectives of physical machine clusters and cloud environment clusters, and to construct a cross-environment data sharing space based on the heterogeneity characteristics, job data interaction requirements and hybrid environment collaborative scheduling objectives.

[0153] The data acquisition and generation module is used to assign a corresponding context-aware module to each sub-job unit, collect the running status parameters of the corresponding sub-job unit and the dynamic resource change data of the environment in real time according to each context-aware module, and generate a corresponding job scheduling optimization factor based on each running status parameter and dynamic resource change data.

[0154] The second adjustment module is used to establish a dependency graph between multiple sub-job units, and dynamically adjust the execution order and resource allocation of the corresponding sub-job units according to the dependency graph and each job scheduling optimization factor, and update the context information in the cross-environment data sharing space in real time to obtain the corresponding execution results.

[0155] The optimization module is used to generate a hybrid environment job execution evaluation report based on multiple execution results and historical interaction data in the cross-environment data sharing space, and optimize the parameter weights of the initial hybrid environment adaptation model based on the execution evaluation report to obtain the final hybrid environment adaptation model, so as to achieve seamless collaborative execution of jobs between physical machines and cloud environments.

[0156] In one embodiment, the first adjustment module includes:

[0157] The generation unit is used to obtain the basic allocation ratio of jobs in the physical machine and cloud environments based on the physical machine adaptation degree and cloud environment adaptation degree output by the initial hybrid environment adaptation model, and to generate a job scheduling strategy based on the basic allocation ratio.

[0158] The first acquisition unit is used to acquire the first computing power, first average computing power, unit time operation and maintenance cost, and first execution time of the physical machine, as well as the second computing power, second average computing power, unit time rental cost, and second execution time of the cloud environment.

[0159] The second acquisition unit is used to acquire cross-environment data transmission time and resource load constraints, and to acquire the estimated total execution time of the job based on the cross-environment data transmission time, the first computational load, the first average computing power, the second computational load, and the second average computing power.

[0160] The verification and adjustment unit is used to obtain the total scheduling cost based on the unit time operation and maintenance cost, the first execution duration, the unit time rental cost, and the second execution duration, and to verify and pre-adjust the job scheduling strategy based on the total scheduling cost, resource load constraints, and the estimated total job execution time.

[0161] The decomposition unit is used to obtain the parallelism parameters of the pre-adjusted job scheduling strategy, and decompose the job into multiple sub-job units according to the parallelism parameters through a hierarchical clustering algorithm.

[0162] It should be noted that each module and unit in the cross-platform and cross-cluster hybrid operation system corresponds one-to-one with the steps in the cross-platform and cross-cluster hybrid operation method.

[0163] like Figure 3 As shown, this application also provides a computer device, which can be a server, and its internal structure can be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores all data required for the process of a cross-platform, cross-cluster hybrid operation method. The network interface is used to communicate with external terminals via a network connection. The computer program is executed by the processor to implement the cross-platform, cross-cluster hybrid operation method.

[0164] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.

[0165] An embodiment of this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the above-described cross-platform, cross-cluster hybrid operation methods.

[0166] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in this application and in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0167] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0168] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A cross-platform, cross-cluster hybrid operation method, characterized in that, include: Obtain the attribute parameters of the job to be executed, the first resource characteristic parameters of the physical machine cluster, and the second resource characteristic parameters of the cloud environment cluster, and construct an initial hybrid environment adaptation model based on the first resource characteristic parameters, the second resource characteristic parameters, and the attribute parameters; The job scheduling strategy is pre-adjusted based on the initial hybrid environment adaptation model, and the job is decomposed into multiple sub-job units based on the pre-adjusted job scheduling strategy. Obtain the heterogeneity characteristics, job data interaction requirements, and hybrid environment collaborative scheduling objectives of physical machine clusters and cloud environment clusters, and construct a cross-environment data sharing space based on the aforementioned heterogeneity characteristics, job data interaction requirements, and hybrid environment collaborative scheduling objectives; Each sub-job unit is assigned a corresponding context-aware module. The running status parameters of the corresponding sub-job unit and the dynamic resource change data of the environment are collected in real time according to each context-aware module. Based on each running status parameter and dynamic resource change data, a corresponding job scheduling optimization factor is generated. Establish a dependency graph among multiple sub-job units, and dynamically adjust the execution order and resource allocation of the corresponding sub-job units according to the dependency graph and each job scheduling optimization factor, and update the context information in the cross-environment data sharing space in real time to obtain the corresponding execution results; A hybrid environment job execution evaluation report is generated based on multiple execution results and historical interaction data in the cross-environment data sharing space. The parameter weights of the initial hybrid environment adaptation model are optimized based on the execution evaluation report to obtain the final hybrid environment adaptation model, so as to achieve seamless collaborative execution of jobs between physical machines and cloud environments.

2. The cross-platform, cross-cluster hybrid operation method according to claim 1, characterized in that, The steps of pre-adjusting the job scheduling strategy based on the initial hybrid environment adaptation model, and decomposing the job into multiple sub-job units based on the pre-adjusted job scheduling strategy, include: The basic allocation ratio of jobs in physical machines and cloud environments is obtained based on the physical machine adaptation degree and cloud environment adaptation degree output by the initial hybrid environment adaptation model, and a job scheduling strategy is generated based on the basic allocation ratio. Obtain the first computing load, first average computing power, unit time operation and maintenance cost, and first execution time of the physical machine, as well as the second computing load, second average computing power, unit time rental cost, and second execution time of the cloud environment; Obtain cross-environment data transmission time and resource load constraints, and obtain the estimated total execution time of the job based on the cross-environment data transmission time, the first computational load, the first average computational capacity, the second computational load, and the second average computational capacity; The total scheduling cost is obtained based on the unit time operation and maintenance cost, the first execution duration, the unit time rental cost, and the second execution duration. The job scheduling strategy is then verified and pre-adjusted based on the total scheduling cost, resource load constraints, and the estimated total job execution time. Obtain the parallelism parameters of the pre-adjusted job scheduling strategy, and decompose the job into multiple sub-job units according to the parallelism parameters using a hierarchical clustering algorithm.

3. The cross-platform, cross-cluster hybrid operation method according to claim 1, characterized in that, The steps for constructing a cross-environment data sharing space based on the aforementioned heterogeneous characteristics, job data interaction requirements, and hybrid environment collaborative scheduling objectives include: Based on the heterogeneity characteristics, obtain the hardware architecture difference value, network latency coefficient, storage protocol adaptability and computing resource elasticity coefficient, and construct an environment difference matrix based on the hardware architecture difference value, network latency coefficient, storage protocol adaptability and computing resource elasticity coefficient; Based on the data interaction requirements of the operation, obtain the data interaction frequency, data transmission volume and data timeliness requirements, and construct a data transmission weight vector based on the data interaction frequency, data transmission volume and data timeliness requirements; Based on the hybrid environment collaborative scheduling objectives, resource utilization objectives, job response time objectives, and cost consumption objectives are obtained, and a scheduling priority matrix is ​​constructed based on the resource utilization objectives, job response time objectives, and cost consumption objectives. The environmental difference matrix, data transmission weight vector, and scheduling priority matrix are input into the preset shared space basic model to generate an initial cross-environment data sharing space. Obtain the real-time data transmission rate, data consistency deviation value, and resource utilization rate of the initial cross-environment data sharing space, and obtain the sharing space optimization coefficient based on the real-time data transmission rate, data consistency deviation value, and resource utilization rate; The initial cross-environment data sharing space is dynamically adjusted based on the shared space optimization coefficient to obtain the target cross-environment data sharing space.

4. The cross-platform, cross-cluster hybrid operation method according to claim 1, characterized in that, The step of generating a corresponding job scheduling optimization factor based on each of the aforementioned operating status parameters and dynamic resource change data includes: Obtain the sub-job real-time response latency, task dependency, and calculated load fluctuation coefficient of the running status parameters, and construct a sub-job status evaluation matrix based on the sub-job real-time response latency, task dependency, and calculated load fluctuation coefficient; The environmental resource utilization rate, resource elastic scaling rate, cross-environment data transmission bandwidth, and resource competition coefficient of the dynamic resource change data are obtained, and an environmental resource adaptability vector is constructed based on the environmental resource utilization rate, resource elastic scaling rate, cross-environment data transmission bandwidth, and resource competition coefficient. Initial scheduling optimization factors are generated based on the environmental resource adaptability vector and the sub-job status evaluation matrix using a multi-dimensional weighted fusion algorithm. Obtain the sub-job completion time deviation, resource waste rate, and cross-environment data synchronization error corresponding to the initial scheduling optimization factor, and obtain the factor correction coefficient based on the sub-job completion time deviation, resource waste rate, and cross-environment data synchronization error; The target job scheduling optimization factor is obtained by iteratively optimizing the initial scheduling optimization factor based on the factor correction coefficient.

5. The cross-platform, cross-cluster hybrid operation method according to claim 1, characterized in that, The steps of dynamically adjusting the execution order and resource allocation of corresponding sub-job units based on the dependency graph and each job scheduling optimization factor include: Extract all predecessor job sets and successor job sets for each sub-job unit from the dependency graph, and obtain the dependency strength of the corresponding sub-job unit based on each predecessor job set; The execution priority of each sub-job unit is obtained based on each set of subsequent jobs and the dependency strength, and all sub-job units are sorted from high to low according to the execution priority to obtain the initial execution order; Determine whether the initial execution order satisfies the execution timing constraints in the dependency graph; If the initial execution order does not satisfy the execution timing constraints in the dependency graph, the sub-job units that violate the execution timing constraints will be adjusted until all their predecessor jobs have been executed to obtain the final execution order; The target execution environment for the corresponding sub-job unit is determined based on each job scheduling optimization factor, and the total available computing resources, total available storage resources, and total available network bandwidth of the target execution environment are obtained. Obtain the resource allocation ratio for each sub-job unit, and obtain the initial computing resource allocation based on the resource allocation ratio and the total available computing resources; The initial storage resource allocation is obtained based on the resource allocation ratio and the total available storage resources, and the initial network bandwidth allocation is obtained based on the resource allocation ratio and the total available network bandwidth. Based on the data interaction weights in the dependency graph, the computational resource allocation, storage resource allocation, and network bandwidth allocation for sub-job units with strong dependencies are balanced and adjusted, and the adjusted computational resource allocation, storage resource allocation, and network bandwidth allocation are used as the final resource allocation for the sub-job units.

6. The cross-platform, cross-cluster hybrid operation method according to claim 1, characterized in that, The steps for generating a hybrid environment job execution evaluation report based on multiple execution results and historical interaction data in a cross-environment data sharing space include: Obtain the job completion time, total resource consumption, and data transmission success rate for each execution result, and obtain the single-dimensional execution indicators of the corresponding sub-job unit based on the job completion time, total resource consumption, and data transmission success rate for each job. The cross-environment data transmission delay, environment switching frequency, and state synchronization error of the historical interaction data are obtained, and the cross-environment collaboration index of the hybrid environment is obtained based on the cross-environment data transmission delay, environment switching frequency, and state synchronization error. A comprehensive evaluation matrix is ​​constructed based on multiple single-dimensional performance indicators and cross-environmental collaborative indicators, and the weight coefficient of each indicator is determined by the analytic hierarchy process. The comprehensive evaluation value of each sub-work unit and the overall evaluation value of the mixed environment are obtained based on the comprehensive evaluation matrix and weighting coefficients. A hybrid environment operation performance evaluation report is generated based on each comprehensive evaluation value, overall evaluation value, single-dimensional performance indicator, and cross-environment collaborative indicator.

7. A cross-platform, cross-cluster hybrid operating system for implementing the method according to any one of claims 1 to 6, characterized in that, include: The first construction module is used to obtain the attribute parameters of the job to be executed, the first resource characteristic parameters of the physical machine cluster and the second resource characteristic parameters of the cloud environment cluster, and to construct an initial hybrid environment adaptation model based on the first resource characteristic parameters, the second resource characteristic parameters and the attribute parameters. The first adjustment module is used to pre-adjust the job scheduling strategy according to the initial hybrid environment adaptation model, and to decompose the job into multiple sub-job units based on the pre-adjusted job scheduling strategy. The second construction module is used to obtain the heterogeneity characteristics, job data interaction requirements and hybrid environment collaborative scheduling objectives of physical machine clusters and cloud environment clusters, and to construct a cross-environment data sharing space based on the heterogeneity characteristics, job data interaction requirements and hybrid environment collaborative scheduling objectives. The data acquisition and generation module is used to assign a corresponding context-aware module to each sub-job unit, collect the running status parameters of the corresponding sub-job unit and the dynamic resource change data of the environment in real time according to each context-aware module, and generate a corresponding job scheduling optimization factor based on each running status parameter and dynamic resource change data. The second adjustment module is used to establish a dependency graph between multiple sub-job units, and dynamically adjust the execution order and resource allocation of the corresponding sub-job units according to the dependency graph and each job scheduling optimization factor, and update the context information in the cross-environment data sharing space in real time to obtain the corresponding execution results. The optimization module is used to generate a hybrid environment job execution evaluation report based on multiple execution results and historical interaction data in the cross-environment data sharing space, and optimize the parameter weights of the initial hybrid environment adaptation model based on the execution evaluation report to obtain the final hybrid environment adaptation model, so as to achieve seamless collaborative execution of jobs between physical machines and cloud environments.

8. The cross-platform, cross-cluster hybrid operation system according to claim 7, characterized in that, The first adjustment module includes: The generation unit is used to obtain the basic allocation ratio of jobs in the physical machine and cloud environments based on the physical machine adaptation degree and cloud environment adaptation degree output by the initial hybrid environment adaptation model, and to generate a job scheduling strategy based on the basic allocation ratio. The first acquisition unit is used to acquire the first computing power, first average computing power, unit time operation and maintenance cost, and first execution time of the physical machine, as well as the second computing power, second average computing power, unit time rental cost, and second execution time of the cloud environment. The second acquisition unit is used to acquire cross-environment data transmission time and resource load constraints, and to acquire the estimated total execution time of the job based on the cross-environment data transmission time, the first computational load, the first average computing power, the second computational load, and the second average computing power. The verification and adjustment unit is used to obtain the total scheduling cost based on the unit time operation and maintenance cost, the first execution duration, the unit time rental cost, and the second execution duration, and to verify and pre-adjust the job scheduling strategy based on the total scheduling cost, resource load constraints, and the estimated total job execution time. The decomposition unit is used to obtain the parallelism parameters of the pre-adjusted job scheduling strategy, and decompose the job into multiple sub-job units according to the parallelism parameters through a hierarchical clustering algorithm.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images