A scheduling method, device and equipment for a natural resource multi-mode computing analysis model

Abstract

The present application discloses a scheduling method, device and equipment for a multi-mode computing and analysis model of natural resources. By virtualizing the computing and analysis model, model integration across multiple modes is achieved; the data in the computing process is scheduled in computing mode and computing resources, and subdivided into multiple computing subtasks to support the calculation of large-scale data; scheduling strategies at different scales are executed according to the scale characteristics of data computing and analysis. This method aims to solve the problems in existing methods that computing and analysis models of different modes have similar functions and inconsistent parameter settings, while the scheduling of computing resources is not refined enough, and the timeliness and accuracy of computing and analysis are insufficient when resources are scarce. This method uses a unified virtualization model to integrate multi-modal analysis models, and finely schedules computing resources to improve timeliness and accuracy. At the same time, it optimizes resource usage, supports large-scale data computing and analysis, and can quickly use new resources to improve efficiency.

Description

Technical Field

[0001] The present application relates to multi-mode spatiotemporal computing and analysis in the field of natural resources, and in particular to a scheduling method, device and equipment for a multi-mode computing and analysis model of natural resources. Background Art

[0002] The current multi-mode spatiotemporal computing and analysis technologies and methods for natural resources have the problem that the spatiotemporal analysis models in different modes have similar functions but inconsistent parameter settings. This problem not only limits the functional versatility between computing modes, affects the reuse of analysis models, but also hinders the lack of effective collaborative scheduling of analyses between different computing modes. Real-time, parallel, and distributed computing modes have different characteristics, so the analysis model needs to be restructured and designed for different computing modes, making the parameters of the same analysis model inconsistent in different computing modes, and they cannot be directly reused with each other. At the same time, due to the inconsistency of parameters of the same analysis model, it is also impossible to effectively achieve collaborative scheduling between different computing modes.

[0003] At the same time, in the scheduling of computing resources for spatiotemporal analysis of natural resources, the current method is to select or manually specify based on the amount of information in the data resources. This method fails to fully consider the characteristics of the computing process and the needs of different computing and analysis functions, resulting in insufficient refinement of resource scheduling. In the case of limited computing resources, there is a lack of effective task scheduling strategies to support the complex and stable computational analysis of large-scale natural resource spatiotemporal data under limited computing resources, which not only makes the computing efficiency low, but also makes it impossible to complete the analysis task. In addition, when computing resources are dynamically expanded, it is impossible to expand computing resources for the executed tasks in a timely manner, which affects the timeliness and accuracy of the computational analysis. Summary of the invention

[0004] The main purpose of this application is to provide a scheduling method for a multi-mode computing analysis model of natural resources, aiming to solve the problem of multi-mode collaborative computing caused by the non-uniform setting of parameters of the same analysis model in different modes in the existing method, and the problem that the computing mode and computing resources cannot be reasonably scheduled and allocated according to the computing data environment and computing resource environment, resulting in insufficient timeliness and accuracy of computing analysis. This method proposes to realize the integration of analysis models in multiple modes by building a unified virtualized computing analysis model, thereby promoting the reuse of analysis models to support effective scheduling between different computing modes. On the basis of multi-mode collaboration, it is proposed to perform fine scheduling according to the amount of information of the computing analysis data required at each stage, and reasonably allocate computing resources to ensure that each link can make full use of the existing computing resources, realize the coordination and cooperation between different computing tasks, and achieve more efficient and economical computing effects. At the same time, in the case of limited computing resources, by splitting the computing analysis process into multiple sub-computing task packages, under the scheduling strategy of "trading time for resources", it supports complex and stable computing analysis of large-scale data; in addition, after the computing resources are dynamically expanded, the newly added computing resources can be scheduled to analyze the queued sub-tasks, thereby improving the efficiency of computing analysis.

[0005] To achieve the above objectives, the present application provides a scheduling method for a natural resource multi-mode computing analysis model, the method comprising the following steps:

[0006] By establishing a unified virtualized computing analysis model, scheduling analysis between different computing modes can be performed;

[0007] Based on the scheduling analysis between different computing modes, the computing modes and computing resources required for each stage of the computing process are scheduled;

[0008] Splitting the computing process into different computing subtask packages;

[0009] When limited computing resources are dynamically expanded, the computing subtask packages are dispatched to the newly added computing resources for computing and analysis.

[0010] In this embodiment, the step of performing scheduling analysis between different computing modes by establishing a unified virtualized computing analysis model further includes:

[0011] Setting the common variables of the virtualized computing analysis model;

[0012] Configure the calculation framework variable parameters and service variable parameters under different calculation modes;

[0013] According to different computing modes, the computing framework type variable parameters and the service type variable parameters are added to the virtualized computing analysis model to obtain the virtualized computing analysis model;

[0014] According to the virtualized computing analysis model, scheduling analysis between different computing modes is performed.

[0015] In this embodiment, the step of scheduling the computing modes and computing resources required for each stage in the computing process according to the scheduling analysis between different computing modes also includes:

[0016] Calculate the amount of information on the input data;

[0017] Different computing modes and computing resources are scheduled based on the amount of data information.

[0018] When the input is an intermediate value of other calculation results, the information amount of the intermediate value calculated in the previous step is used as input, and the information amount of the intermediate value is pre-calculated according to the different calculation modes;

[0019] After the amount of information calculated at each stage in the entire process is completed, the calculation mode and the scheduling of the calculation resources are performed.

[0020] In this embodiment, the step of scheduling different computing modes and computing resources based on the amount of data information as a scheduling basis also includes:

[0021] When the calculation and analysis type is a spatial calculation type, the minimum value of the amount of spatial calculation information in the input data is used as a scheduling basis;

[0022] When the calculation and analysis type is the attribute calculation type, the maximum value of the attribute calculation information in the input data is used as the scheduling basis.

[0023] In this embodiment, when the calculation and analysis type is a spatial calculation type, the step of using the minimum value of the spatial calculation information amount in the input data as the scheduling basis further includes:

[0024] When it is determined that the computing analysis type is a spatial computing type and the minimum value of the spatial computing information amount is in the first interval, scheduling instant computing and corresponding computing resources;

[0025] When it is determined that the computing analysis type is a spatial computing type and the minimum value of the spatial computing information amount is in the second interval, scheduling parallel computing and corresponding computing resources;

[0026] When it is determined that the computing analysis type is a spatial computing type and the minimum value of the spatial computing information amount is in a third interval, distributed computing and corresponding computing resources are scheduled.

[0027] In this embodiment, when the calculation and analysis type is the attribute calculation type, the step of using the maximum value of the attribute calculation information amount in the input data as the scheduling basis further includes:

[0028] When it is determined that the computing analysis type is an attribute computing type and the maximum value of the attribute computing information amount is in the first interval, scheduling instant computing and corresponding computing resources;

[0029] When it is determined that the computing analysis type is an attribute computing type and the maximum value of the attribute computing information amount is in the second interval, scheduling parallel computing and corresponding computing resources;

[0030] When it is determined that the computing analysis type is the attribute computing type and the maximum value of the attribute computing information amount is in the third interval, the distributed computing and the corresponding computing resources are scheduled.

[0031] In this embodiment, the step of splitting the computing process into different computing subtask packages further includes:

[0032] Splitting the computing process into multiple computing subtasks;

[0033] Combining the computing subtasks into task packages for sequential execution;

[0034] Scheduling the task package according to the current occupancy of the computing resources;

[0035] The scheduling of the task packages is dynamically adjusted according to the increase or decrease of the current computing resources.

[0036] In this embodiment, the step of scheduling newly added computing resources to be allocated to the computing subtask package for computing analysis when limited computing resources are dynamically expanded also includes:

[0037] Divide computing subtasks at different scales according to the data characteristics of natural resources;

[0038] A strategy for executing computing subtasks at different scales without dependencies to be prioritized into the scheduling queue;

[0039] According to the computing states of the computing subtasks at different scales without dependencies after the update is completed, a strategy is implemented to delay the computing subtasks with dependencies from entering the scheduling queue.

[0040] In addition, to achieve the above-mentioned purpose, the present application also provides a scheduling device for a natural resource multi-mode computing analysis model, the device comprising: a model startup module, a data scheduling module, a process splitting module and a strategy output module;

[0041] The model startup module conducts scheduling analysis between different computing modes by establishing a unified virtualized computing analysis model;

[0042] The data scheduling module schedules the computing modes and computing resources required at each stage of the computing process based on the scheduling analysis between different computing modes;

[0043] A process splitting module, which splits the computing process into different computing subtask packages;

[0044] The strategy execution module dispatches the computing subtask package to the newly added computing resources for computing and analysis when the limited computing resources are dynamically expanded.

[0045] In addition, to achieve the above-mentioned purpose, the present application also provides a scheduling device for a natural resource multi-mode computing analysis model, wherein the scheduling device for the natural resource multi-mode computing analysis model comprises: a memory, a processor, and a scheduling processing program for the natural resource multi-mode computing analysis model stored in the memory and executable on the processor, wherein the scheduling processing program for the natural resource multi-mode computing analysis model implements the steps of the scheduling method for the above-mentioned natural resource multi-mode computing analysis model when executed by the processor.

[0046] The above one or more technical solutions provided by this application may have the following advantages or at least achieve the following technical effects:

[0047] The present application discloses a scheduling method, device and equipment for a multi-mode computing analysis model of natural resources, which relates to the field of multi-mode computing analysis. The technical method includes the following steps: first, by establishing a unified virtualized computing analysis model, computing analysis of multiple modes is started to realize model integration across multiple modes; second, according to the computing analysis of multiple modes, the computing mode and computing resources are scheduled for the data required for computing analysis at each stage in the computing process; then, the computing process is split into different computing subtasks to support the calculation of large-scale data; finally, according to the different computing subtasks, the scheduling strategy of computing subtasks at different scales is executed. The present application aims to solve the problems that the computing analysis models of different modes in the existing methods have similar functions and inconsistent parameter settings, and the scheduling of computing resources is not refined enough, and the timeliness and accuracy of computing analysis are insufficient when computing resources are limited. In order to overcome these limitations, the method realizes the integration of analysis models across multiple modes by establishing a unified virtualized model, thereby promoting the reuse of analysis models to support effective scheduling between different computing modes. According to the amount of information of the computing analysis data, the computing resources required at each stage in the computing process are reasonably scheduled and allocated. This ensures that each link can make full use of existing computing resources and achieve coordination and cooperation between different computing tasks. It can also optimize resource use, avoid resource waste, and achieve more efficient and economical computing results. At the same time, it can support complex and stable computing analysis of large-scale data. Once computing resources are expanded, the newly added computing resources can be quickly used to analyze the queued subtasks, thereby improving computing and analysis efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0049] Figure 1 It is a flow chart of a first embodiment of a scheduling method of a multi-mode computing and analysis model for natural resources proposed in an embodiment of the present application;

[0050] Figure 2 A flowchart of a second embodiment of the scheduling method of the multi-mode computing and analysis model for natural resources proposed in the embodiment of the present application;

[0051] Figure 3 A flowchart of a third embodiment of the scheduling method of the multi-mode computing and analysis model for natural resources proposed in the embodiment of the present application;

[0052] Figure 4 A flowchart of a fourth embodiment of a scheduling method for a multi-mode computing and analysis model for natural resources proposed in an embodiment of the present application;

[0053] Figure 5 This is a flow chart of Example 5 of the scheduling method of the natural resource multi-mode computing analysis model proposed in the embodiment of the present application.

[0054] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture. If the specific posture changes, the directional indications will also change accordingly.

[0057] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features limited to "first" and "second" may explicitly or implicitly include at least one of the features. In addition, if "and / or" or "and / or" appears in the full text, its meaning includes three parallel schemes. Taking "A and / or B" as an example, it includes scheme A, or scheme B, or a scheme that satisfies both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0058] In order to achieve the integration and reuse of analysis models across multiple modes and to support effective scheduling between different computing modes, the present application aims to provide a scheduling method for a multi-mode computing analysis model of natural resources. The following technical scheme is adopted: a unified virtualized computing analysis model is established, and computing analysis models of multiple modes are integrated to form a unified virtualized computing analysis model, which can support model integration across multiple modes and realize data sharing and computing collaboration between different modes. Using the established unified virtualized computing analysis model, computing analysis of multiple modes is started. According to the needs of computing analysis, the data and computing modes required for each stage in the computing process are determined. According to the needs of computing mode and computing tasks, tasks and computing resources are dynamically scheduled to ensure efficient execution of computing tasks. The complex computing process is divided into multiple relatively independent computing subtasks. Each computing subtask can be executed independently and supports the processing and analysis of large-scale data. For each computing subtask, scheduling strategies at different scales are formulated according to its characteristics and needs. The scheduling strategy includes the allocation of computing resources, the priority sorting of computing tasks, etc., to ensure the efficient and accurate execution of computing tasks.

[0059] In order to break down the barriers between different modal computational analysis models, we first need to establish a unified virtualized computational analysis model. This model can integrate data from multiple modalities and provide a unified interface and parameter settings so that data from different modalities can be computed and analyzed in the same framework. By abstracting the features of data from different modalities, a universal computational analysis framework is constructed. A unified interface is provided so that data from different modalities can be easily input and output. A unified parameter setting standard is established to ensure consistency and comparability of computational analysis of different modalities.

[0060] In the computing process, the computing and analysis tasks at different stages have different data requirements and computing resource consumption. Therefore, we need to fine-tune the scheduling of the computing process to ensure that each stage can make full use of the existing computing resources. Split the computing process into multiple subtasks, each of which corresponds to a specific computing and analysis stage. Evaluate the computing resources required for each subtask, including CPU, memory, storage, etc. According to the resource evaluation results, dynamically allocate computing resources to each subtask to ensure efficient use of resources. For the computing and analysis of large-scale data, we need to adopt an effective strategy to support complex and stable calculations. This goal can be achieved by splitting the computing and analysis process into multiple sub-computing task packages and using the method of "trading time for resources". Split the computing and analysis process of large-scale data into multiple sub-computing task packages, each of which contains a part of data and a specific computing and analysis task. In the case of limited resources, more computing resources can be obtained by increasing the computing time. For example, the task packages can be assigned to different computing nodes and executed in parallel in different time periods. The stability and accuracy of the calculation results are ensured through redundant calculation and verification mechanisms.

[0061] When computing resources are expanded, we need to be able to quickly utilize these new resources to improve computing and analysis efficiency. We can achieve this goal by dynamically adjusting task allocation and scheduling strategies. Monitor the status of computing resources in real time, including CPU usage, memory occupancy, etc. When new resources are available, dynamically reallocate tasks to these resources based on the status of the current task queue. Through the load balancing algorithm, ensure that the task distribution between each computing node is balanced to avoid some nodes being overloaded while other nodes are idle.

[0062] The natural resource computing and analysis scenarios have spatiotemporal characteristics. The computing and analysis tasks at different scales have different resource requirements and scheduling strategies. Therefore, we need to define scheduling methods at different scales. According to the spatiotemporal characteristics of natural resources, the computing and analysis tasks are divided into different scales, such as local scale, regional scale, and global scale. For each scale, formulate corresponding scheduling strategies. For example, at the local scale, instant computing can be used first; at the regional scale, distributed computing can be used; at the global scale, cross-regional data transmission and computing resource scheduling need to be considered. According to the actual application scenarios and changes in computing resources, the scheduling strategy is continuously optimized to improve the pertinence and effectiveness of scheduling.

[0063] The multimodal computational analysis of natural resources is a complex and data-intensive process involving different types of models and a large amount of data processing. In order to improve the efficiency and accuracy of computational analysis, this application proposes a scheduling method that uses virtualization technology to achieve model integration and refined scheduling of computing resources. This application proposes a scheduling method for the multimodal computational analysis model of natural resources in the first embodiment, please refer to Figure 1 The scheduling method of the natural resource multi-mode computing analysis model includes steps S10 to S40:

[0064] Step S10, performing scheduling analysis between different computing modes by establishing a unified virtualized computing analysis model;

[0065] Step S20, scheduling the computing modes and computing resources required for each stage in the computing process according to the scheduling analysis between different computing modes;

[0066] Step S30, splitting the calculation process into different calculation subtask packages;

[0067] Step S40, when limited computing resources are dynamically expanded, scheduling newly added computing resources to be allocated to the computing subtask package for computing and analysis.

[0068] It should be noted that in this embodiment, the method is based on a virtualized natural resource multi-mode computing analysis model, and through a unified computing framework, multiple modes of computing analysis are performed on natural resource data. Through the dynamic scheduling of computing modes and computing resources, as well as the allocation and scheduling of computing tasks, efficient processing of large-scale data is achieved. Finally, according to different computing subtasks, targeted scheduling strategies are executed to support scientific management and decision-making of natural resources.

[0069] Step S10, by integrating different modes of computing and analysis models into a unified computing framework, seamless connection and efficient processing of multiple types of data are achieved. Through virtualization technology, physical computing resources are abstracted into virtual resources to achieve flexible configuration and efficient use of computing resources. Virtualization technology can shield the complexity of the underlying hardware, provide a unified computing environment, and support the operation of multiple computing and analysis models. Use the virtualization platform to build a computing environment that supports multiple computing and analysis models. According to the characteristics and analysis requirements of natural resource data, select appropriate computing and analysis models, and integrate and optimize them. Natural resource data is diverse, including geography, meteorology, ecology, hydrology and other forms. By building a unified computing and analysis model, integrated processing and analysis of data in multiple forms can be achieved, and the comprehensiveness and accuracy of data processing can be improved. Collect and analyze data features of different forms, including remote sensing data, meteorological data, geographic data, etc. According to data features, a general computing and analysis framework is constructed, including data processing, computing and analysis, and result output. Formulate unified interface and parameter setting standards to ensure that data in different forms can be computed and analyzed under the same framework.

[0070] First, the type, scale, and processing requirements of natural resource data are analyzed in detail to clarify the goals and requirements of computational analysis. The virtualization model in natural resource multi-mode computing is expressed as:

[0071] ;

[0072] Among them, the model It is a model variable, such as cache distance in buffer analysis; These are variables of different computing frameworks, such as data partitioning parameters and sampling parameters for parallel computing; These are service variables, such as the computing resources used by computing models under different frameworks. It is a variable shared by multiple computing modes, ensuring compatibility and consistency between different computing modes; and It is a collection of parameters in each computing mode, and is mainly configured as a common variable before starting the computing service. Virtualization model before scheduling the calculation Will only have parameter set , when calculating after scheduling, the parameter set will be set according to the calculation mode selected by the scheduling and Complete as .

[0073] According to the selected calculation mode, the corresponding calculation frame type variable and service variables Completion to virtualized model In this step, you need to ensure that the completed parameters match the computing mode and computing task requirements. After the parameter configuration is completed, start the computing task. The computing task will be executed according to the configured parameter set. , and Perform calculations and output results. The model should support the input and processing of multiple modal data and have flexible configuration and expansion capabilities. Use virtualization technology to build and deploy computational analysis models. Ensure that the model can run stably and has good performance and scalability. Verify and test the computational analysis model to ensure that it can correctly process and analyze natural resource data and meet the requirements of computational analysis.

[0074] Step S20, according to the calculation analysis results of multiple modes, determine the data processing volume and calculation complexity required for each stage in the calculation process, and then select the appropriate calculation mode and allocate the corresponding calculation resources. According to the data processing requirements of each stage in the calculation process, dynamically select the appropriate calculation mode and allocate calculation resources. Dynamic scheduling can monitor the execution of calculation tasks in real time, dynamically adjust calculation resources and scheduling according to factors such as task priority and resource utilization, and ensure the smooth completion of calculation tasks. Specifically subdivide the calculation process, and clarify the data input, processing method and output requirements required for each stage. According to these requirements, select a suitable calculation mode (such as parallel computing, distributed computing, etc.), and dynamically allocate computing resources (such as CPU, memory, storage, etc.) to ensure the smooth completion of calculation tasks.

[0075] Select the appropriate computing mode according to the data processing requirements at each stage of the computing process. For example, for large-scale data processing, you can choose computing modes such as parallel computing or distributed computing; for complex data analysis tasks, you can choose computing modes such as machine learning or deep learning. Evaluate the performance and availability of computing resources, including hardware resources such as CPU, memory, storage, and communication resources such as network bandwidth. Determine the allocation plan for computing resources based on the evaluation results. Adjust the allocation of computing resources in real time based on the execution of computing tasks. Strategies can include resource reservation, resource release, task migration, etc. to ensure the smooth completion of computing tasks. Monitor the execution of computing tasks and the utilization of computing resources in real time, and provide feedback and adjustments based on the monitoring results. Improve computing efficiency and resource utilization by continuously optimizing scheduling strategies.

[0076] Step S30, split the complex computing process into multiple relatively independent computing subtask packages to reduce the computing difficulty and improve processing efficiency. Analyze the computing process in detail and identify the computing links that can be split. Split the computing process into multiple computing subtask packages based on the data scale, computing complexity and dependencies between tasks. Each subtask should have clear data input, processing method and output requirements.

[0077] Analyze the computational process in detail and identify the computational links that can be split and the computational steps that can be optimized. Split the computational process into multiple relatively independent computational subtasks. Each subtask should have clear data input, processing methods, and output requirements. By splitting the tasks, the computational difficulty can be reduced and the processing efficiency can be improved. Parallelize the computational subtasks and use multi-threading, multi-processing and other technologies to improve the computational efficiency. At the same time, optimize the bottleneck links in the computational process to improve the overall computational performance.

[0078] Step S40, formulate targeted scheduling strategies according to different computing subtasks to achieve optimal configuration of computing resources and maximize computing efficiency. Perform detailed evaluation and analysis on each computing subtask to determine the required computing resources, computing mode and execution order. Based on this information, formulate specific scheduling strategies, including the allocation of computing resources, the selection of computing mode, the priority relationship between tasks, etc. Finally, output these scheduling strategies in a visual way for reference and execution by relevant personnel.

[0079] According to the size and complexity of the computing subtasks, they are divided into different scales. The scales can include task level, subtask level, data block level, etc. Analyze the dependencies between computing subtasks and determine the order and parallelism between tasks. By optimizing the dependencies between tasks, the computing efficiency can be further improved. Formulate specific scheduling strategies for computing subtasks of different scales. The strategies can include task priority sorting, resource allocation schemes, task migration strategies, etc. By formulating appropriate scheduling strategies, efficient execution of computing subtasks and optimal utilization of resources can be ensured. Implement the scheduling strategy into the computing system and monitor the execution of computing subtasks and the utilization of computing resources in real time. Adjust and optimize the scheduling strategy based on the monitoring results. Output the scheduling results of computing subtasks and evaluation indicators such as the utilization of computing resources. Analyze and evaluate the scheduling results to verify the effectiveness and feasibility of the scheduling strategy.

[0080] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 In this embodiment, step S10, the step of performing scheduling analysis between different computing modes by establishing a unified virtualized computing analysis model, further includes:

[0081] Step S11, setting the common variables of the virtualized computing analysis model;

[0082] Step S12, configuring the calculation framework type variable parameters and service type variable parameters under different calculation modes;

[0083] Step S13, according to different computing modes, the computing framework type variable parameters and the service type variable parameters are added to the virtualized computing analysis model to obtain the virtualized computing analysis model;

[0084] Step S14: performing scheduling analysis between different computing modes according to the virtualized computing analysis model.

[0085] Specifically, in this embodiment, the establishment of a unified virtualized computing analysis model to perform computing analysis of multiple modes is the basis of the entire scheduling process. This step not only requires the model to support data input and processing of multiple modes, but also requires a high degree of flexibility and scalability to adapt to changes in different computing modes and requirements. The following is a detailed principle process and step description of this step and its sub-steps S11 to S15:

[0086] Step S11: The core of the virtualized computing analysis model is to abstract physical computing resources into virtual resources to achieve flexible configuration and efficient use of computing resources. In the scenario of multi-modal computing analysis of natural resources, the model needs to support data input of multiple modes, such as geography, meteorology, ecology, etc., and be able to dynamically adjust computing resources and computing modes according to the characteristics and needs of these data. In multi-modal computing of natural resources, usually represents the i-th virtualization model. are the common variables of the model, which are common in multiple computing modes, ensuring the compatibility and consistency between different computing modes. For example, in buffer analysis, cache distance is a typical common variable. Through detailed analysis of the type, scale, and processing requirements of natural resource data, the goals and requirements of computing analysis are clarified. This includes the source, format, storage method, and specific requirements of computing tasks. According to the results of demand analysis, a unified virtualized computing analysis model architecture is designed. The architecture should include multiple layers such as data input layer, computing processing layer, and result output layer, and support the selection of multiple computing modes and algorithms. Select appropriate virtualization technology, computing framework, and service-oriented components to build a virtualized computing analysis model. Virtualization technology can shield the complexity of the underlying hardware and provide a unified computing environment; the computing framework is responsible for processing specific computing tasks; and the service-oriented components provide necessary support and services, such as data storage and network communication. Use the selected technology to realize the construction and deployment of virtualized computing analysis models. This includes model coding, testing, optimization, and deployment and configuration in the actual environment.

[0087] Step S12, the computing framework type variable parameters refer to parameters related to the computing framework itself, which determine the behavior and performance of the computing framework. These are different computing framework variables. These variables are related to a specific computing framework and are used to control the behavior of the computing process. For example, in parallel computing, data partitioning parameters and sampling parameters are typical computing framework variables. Related to a specific computing framework, it is necessary to configure before starting the computing service. The values ​​of these variables should be determined according to the characteristics of the computing framework and the requirements of the computing task. Different computing frameworks may require different computing framework variables, so they need to be carefully distinguished during configuration.

[0088] These are service variables. These variables are related to computing services and are used to configure computing resources and services. For example, computing models under different frameworks may use different computing resources, such as CPU, memory, or storage, which can be configured through service variables. Service variables sij are related to computing services and also need to be configured before starting computing services. The values ​​of these variables should be determined based on the requirements of computing resources and services. The configuration of service variables can ensure that computing services can correctly obtain and use the required resources and services at runtime.

[0089] In particular, it should be noted that these parameters may need to be configured differently in different computing modes, including parallel mode, real-time processing mode, distributed computing mode, etc. According to the characteristics of the computing mode, the computing framework type variable parameters that need to be configured are identified. These parameters may include the number of computing nodes, memory size, priority of computing tasks, etc. For each computing mode, the computing framework type variable parameters are configured according to actual needs. This includes setting the parameter value range, default value, and adjusting parameters to optimize computing performance.

[0090] Step S13: Before scheduling the calculation, virtualize the model Contains only parameter sets During the scheduling process, the parameter set needs to be set according to the selected calculation mode. and Complete as The scheduling process usually selects the appropriate computing mode based on the computing task requirements and computing resource conditions. The selection of computing mode should take into account factors such as computing efficiency, resource utilization, and computing cost. and service variables Completion to virtualized model In this step, you need to ensure that the completed parameters match the computing mode and computing task requirements. After the parameter configuration is completed, start the computing task. The computing task will be executed according to the configured parameter set. , and Calculate and output the results. After configuring the variable parameters in different calculation modes, you need to complete these parameters in the virtualized calculation analysis model. This step is the key to model construction. It ensures that the model can automatically adjust parameters according to different calculation modes, thereby realizing the calculation and analysis of multiple modal data. When completing parameters, you need to pay attention to the accuracy and consistency of the parameters to avoid errors or deviations in the model during the calculation process.

[0091] Step S14, finally, according to the virtualized computing analysis model that has been established and completed with parameters, different computing modes are analyzed. This step involves multiple links such as data input, model calculation, and result output. In the data input stage, it is necessary to ensure the accuracy and completeness of the input data; in the model calculation stage, it is necessary to select a suitable computing mode and algorithm for calculation; in the result output stage, it is necessary to present the calculation results in a form that is easy to understand and analyze. In summary, by establishing a unified virtualized computing analysis model, setting common variables, configuring variable parameters under different computing modes, and completing and analyzing multiple modal calculations, a comprehensive, accurate and efficient computing analysis of the data can be achieved. This helps to improve the efficiency and accuracy of data analysis and provide strong support for decision making.

[0092] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those of the above-mentioned first and second embodiments can refer to the above introduction, and will not be repeated later. Figure 3 In this embodiment, step S20, the step of scheduling the computing modes and computing resources required for each stage in the computing process according to the scheduling analysis between different computing modes, also includes:

[0093] Step S21, calculating and analyzing the amount of information of the input data;

[0094] Step S22, scheduling different computing modes and computing resources based on the amount of data information as a scheduling basis;

[0095] Step S23, when the input is an intermediate value of other calculation results, the information amount of the intermediate value calculated in the previous step is used as input, and the information amount of the intermediate value is pre-calculated according to the different calculation modes;

[0096] Step S24, after completing the information amount calculated in each stage of the whole process, the calculation mode and the calculation resources are scheduled.

[0097] It should be noted that in this embodiment, in a complex computing and analysis process, how to efficiently schedule computing modes and computing resources is the key to ensuring the successful completion of the task. Step S20 aims to intelligently schedule the data required for each stage in the computing process based on the computing and analysis results of multiple modes to optimize computing efficiency and resource utilization. This process will be described in detail below.

[0098] Step S21, evaluate the complexity and information content of the input data to provide a basis for subsequent computing mode selection and resource scheduling. By using methods in information theory, such as entropy and mutual information, the input data is quantitatively analyzed. These indicators can reflect the characteristics of data redundancy, correlation, and uncertainty. The information content index of each input data is obtained to provide a data basis for subsequent steps. In natural resource multi-mode computing, data labels are an effective expression of the information content of the data to be calculated. By using The triplet model allows us to accurately describe the parameter information contained in the region, where Represents the source data size, Indicates that by extracting the data involved in the calculation, It specifically reflects the amount of information in the data. Based on this amount of information, we can perform intelligent scheduling of computing modes and computing resources, thereby optimizing the computing process and improving computing efficiency.

[0099] Step S22, selecting the most suitable data for scheduling according to the characteristics and requirements of the computing mode.

[0100] Analyze the characteristics of different computing modes such as sensitivity to data volume, computational complexity, and resource consumption, and combine the information volume index obtained in step S21 to select data with moderate information volume that can meet the computing requirements as the scheduling basis. Determine the data set required for each computing mode to provide guidance for subsequent resource scheduling. First, analyze different computing modes to understand their data requirements, processing methods, and performance characteristics. According to the characteristics of the computing mode, set different information volume thresholds. These thresholds will be used to screen and filter data to ensure that the selected data can meet the requirements of the computing mode. According to the set information volume threshold, filter out data that meets the requirements from the input data. These data will be used as input for subsequent computing tasks. Verify the selected data to ensure that they can meet the requirements of the computing mode and have good processing effects. When setting the information volume threshold, it is necessary to fully consider the characteristics and requirements of the computing mode to ensure that the selected threshold is reasonable and effective. The data selection process may involve a large number of data screening and filtering operations, so it is necessary to optimize the algorithm to improve processing efficiency.

[0101] Step S23, when the input data is the intermediate value of other calculation results, this step aims to use the amount of information calculated in the previous step of the intermediate value as input to pre-calculate the amount of information of the intermediate value. This helps us better understand the complexity and processing difficulty of the intermediate value, and provides a basis for the subsequent selection of calculation mode and resource allocation. First, the intermediate value is obtained from the previous calculation task. These intermediate values ​​may be the results of the previous round of calculations, or they may be the outputs of other related calculation tasks. Using the amount of information calculated in the previous step of the intermediate value as input, the amount of information of the intermediate value is pre-calculated in a specific way (such as information theory method, machine learning model, etc.). The calculated amount of information of the intermediate value is analyzed to understand the complexity and redundancy of the intermediate value. This information will serve as the basis for the subsequent selection of calculation mode and resource allocation. According to the analysis results, the calculation mode or resource allocation is optimized and adjusted to ensure that the intermediate value can be effectively processed and analyzed. When transmitting the amount of information, it is necessary to ensure the accuracy and completeness of the information to avoid information loss or miscommunication. Pre-calculating the amount of information of the intermediate value may require additional computing resources and time, so it is necessary to weigh the pros and cons and optimize the algorithm to improve efficiency.

[0102] Step S24, after completing the information amount calculated at each stage in the whole process, this step aims to schedule the computing mode and computing resources according to this information amount. Through a reasonable scheduling strategy, it can be ensured that the computing task can be completed efficiently and accurately. First, the information amount of each computing stage is comprehensively evaluated to understand the complexity and resource requirements of the entire computing process. According to the evaluation results, a suitable computing mode is selected. These computing modes include parallel computing, distributed computing, real-time computing, etc., and the specific selection depends on the scale, complexity and processing requirements of the data. According to the selection of the computing mode and the requirements of the computing task, the computing resources are reasonably allocated. These resources include CPU, memory, storage, network bandwidth, etc. According to the results of the computing mode and resource allocation, a detailed scheduling strategy is formulated. These strategies include task priority sorting, resource reservation and release, task migration and load balancing, etc. The computing task is executed according to the formulated scheduling strategy, and the execution of the task and the utilization of resources are monitored in real time. According to the monitoring results, the scheduling strategy is adjusted and optimized. After the computing task is completed, the calculation results are output and evaluated. The evaluation content includes the accuracy, efficiency and resource utilization of the calculation results. According to the evaluation results, the scheduling strategy is further optimized and improved.

[0103] Furthermore, in this embodiment, the step S22, the step of selecting data with different amounts of information as the scheduling basis according to the different calculation and analysis types, further includes:

[0104] Step S221, when the calculation and analysis type is a spatial calculation type, the minimum value of the spatial calculation information amount in the input data is used as a scheduling basis;

[0105] Step S222, when the calculation and analysis type is attribute calculation type, taking the maximum value of attribute calculation information in the input data as the scheduling basis;

[0106] Specifically, in this embodiment, it is necessary to select data with an appropriate amount of information from the input data as the basis for scheduling according to the different characteristics of the computing mode. This process is intended to ensure that the computing task can be completed efficiently and accurately while making full use of computing resources. The following is a detailed description of step S22, which includes two sub-steps:

[0107] Step S221, when the calculation and analysis type is spatial calculation type, the spatial information in the data (such as geographic location, spatial relationship, etc.) is crucial to the accuracy of the calculation results. Therefore, we need to select data with sufficient spatial information as the basis for scheduling to ensure that the computing task can correctly process the spatial information. However, in order to avoid too much redundant data and computing burden, we select the minimum value of spatial computing information as the scheduling basis, that is, select data that can meet the computing requirements but has the lowest information redundancy. First, the spatial information in the input data is quantitatively calculated to obtain the spatial computing information of each data item. This can be achieved through spatial analysis algorithms (such as spatial autocorrelation, spatial clustering, etc.). Sort the calculated spatial information and arrange it from small to large. From the sorted data, select the data with the minimum information value in the data as the basis for scheduling. This means that the data we selected contains sufficient spatial information and avoids excessive redundancy. According to the selected data, a scheduling strategy suitable for the spatial computing mode is formulated to allocate computing resources, sort tasks, and so on.

[0108] Step S222, when the calculation and analysis type is attribute calculation type, the attribute information in the data (such as attribute value, attribute relationship, etc.) is crucial to the accuracy of the calculation result. In order to obtain accurate calculation results, we need to select data with rich attribute information as the basis for scheduling. Therefore, we select the maximum amount of information in the data as the basis for scheduling, that is, select the data containing the most useful attribute information. First, the attribute information in the input data is quantitatively calculated to obtain the attribute calculation information of each data item. This can be achieved through attribute analysis algorithms (such as correlation analysis, principal component analysis, etc.). Sort the calculated attribute information from large to small. From the sorted data, select the data with the maximum amount of information as the basis for scheduling. This means that the data we selected contains the most useful attribute information. According to the selected data, formulate a scheduling strategy suitable for the attribute calculation mode. This may include data preprocessing, computing resource allocation, task priority sorting, etc. At the same time, it is necessary to consider how to effectively use this rich attribute information to improve the accuracy of the calculation results.

[0109] Further, in this embodiment, step S221, when the calculation and analysis type is the spatial calculation type, the step of using the minimum value of the spatial calculation information amount in the input data as the scheduling basis, further includes:

[0110] When it is determined that the computing analysis type is a spatial computing type and the minimum value of the spatial computing information amount is in the first interval, scheduling instant computing and corresponding computing resources;

[0111] When it is determined that the computing analysis type is a spatial computing type and the minimum value of the spatial computing information amount is in the second interval, scheduling parallel computing and corresponding computing resources;

[0112] When it is determined that the computing analysis type is a spatial computing type and the minimum value of the spatial computing information amount is in a third interval, distributed computing and corresponding computing resources are scheduled.

[0113] Specifically, in this embodiment, when the calculation analysis type is determined to be spatial calculation type, we need to formulate a scheduling strategy based on the minimum value of the spatial calculation information in the input data and allocate corresponding computing resources. In order to manage resources more finely and improve computing efficiency, we divide the minimum value of the spatial calculation information into different intervals and schedule different computing modes and resources according to these intervals. The following is a detailed description of step S221:

[0114] Determine the intervals with the minimum value of the amount of spatial computing information, where the first interval indicates that the amount of spatial computing information is low, the computing task is relatively simple, and does not require too many computing resources. The second interval indicates that the amount of spatial computing information is moderate, the computing task has a certain degree of complexity, and appropriate parallel computing capabilities are required to accelerate processing. The third interval indicates that the amount of spatial computing information is high, the computing task is very complex, and powerful distributed computing capabilities are required to cope with it. Set the thresholds for each interval based on historical data and experience. These thresholds can be adjusted according to different application scenarios and computing requirements.

[0115] First interval (instant calculation): When the minimum value of spatial calculation information is in the first interval, the calculation task is relatively simple, and the instant calculation mode can be used directly. A small amount of computing resources, such as a single CPU core and an appropriate amount of memory, are allocated to meet the calculation requirements. The response speed is fast and the resource consumption is low.

[0116] Second interval (parallel computing): When the minimum value of spatial computing information is in the second interval, the computing task has a certain complexity and requires parallel computing to accelerate processing. Schedule parallel computing mode and use parallel computing resources such as multi-core CPU or GPU. Dynamically allocate appropriate computing resources according to the scale of the computing task to balance the computing speed and resource consumption. Fast computing speed, suitable for computing tasks of medium complexity.

[0117] The third interval (distributed computing): When the minimum value of spatial computing information is in the third interval, the computing task is very complex and requires distributed computing to deal with. Distributed computing mode is scheduled to utilize resources on multiple computers or cloud computing platforms. According to the scale and complexity of the computing task, a large amount of computing resources, including CPU, memory, storage, and network bandwidth, are dynamically allocated. It has strong computing power and is suitable for large-scale and highly complex computing tasks.

[0118] In particular, it should be noted that during the execution of computing tasks, the utilization of computing resources and the progress of computing tasks are dynamically monitored. Based on the monitoring results, the computing mode and resource allocation are adjusted in a timely manner to optimize computing performance. During the computing process, various errors and abnormal situations may be encountered. A complete error handling mechanism is established to capture and handle errors in a timely manner to ensure the smooth progress of computing tasks. When necessary, computing tasks can be resumed or resources can be reallocated.

[0119] Further, in this embodiment, step S222, when the calculation and analysis type is the attribute calculation type, the step of using the maximum value of the attribute calculation information amount in the input data as the scheduling basis, further includes:

[0120] When it is determined that the computing analysis type is an attribute computing type and the maximum value of the attribute computing information amount is in the first interval, scheduling instant computing and corresponding computing resources;

[0121] When it is determined that the computing analysis type is an attribute computing type and the maximum value of the attribute computing information amount is in the second interval, scheduling parallel computing and corresponding computing resources;

[0122] When it is determined that the computing analysis type is the attribute computing type and the maximum value of the attribute computing information amount is in the third interval, the distributed computing and the corresponding computing resources are scheduled.

[0123] Specifically, in this embodiment, when the calculation analysis type is determined to be attribute calculation type, we formulate a scheduling strategy based on the maximum value of the attribute calculation information in the input data and allocate corresponding computing resources. In order to optimize resource utilization and improve computing efficiency, we divide the maximum value of the attribute calculation information into different intervals and schedule different computing modes and resources according to these intervals. The following is a detailed description of step S222:

[0124] Determine the interval of the maximum value of the attribute calculation information, where the first interval indicates that the attribute calculation information is relatively low, the calculation task is relatively simple, and does not require too many computing resources. The second interval indicates that the attribute calculation information is moderate, the calculation task has a certain complexity, and appropriate parallel computing capabilities are required to accelerate processing. The third interval indicates that the attribute calculation information is very high, the calculation task is very complex, and powerful distributed computing capabilities are required to cope with it. Set the threshold for each interval based on historical data and experience. These thresholds can be adjusted according to different application scenarios and computing needs.

[0125] First interval (instant calculation): When the maximum amount of attribute calculation information is in the first interval, the calculation task is relatively simple, and the instant calculation mode can be used directly. A small amount of computing resources, such as a single CPU core and an appropriate amount of memory, are allocated to meet the calculation requirements. It has fast response speed and low resource consumption, and is suitable for simple attribute calculation tasks.

[0126] Second interval (parallel computing): When the maximum amount of attribute computing information is in the second interval, the computing task has a certain complexity and requires parallel computing to speed up the processing. Schedule parallel computing mode and use multi-core CPU, GPU or dedicated parallel computing equipment. According to the scale of the computing task, dynamically allocate the appropriate amount of computing resources to balance the computing speed and resource consumption. Advantages: fast computing speed, suitable for medium-complexity attribute computing tasks.

[0127] The third interval (distributed computing): When the maximum amount of attribute computing information is in the third interval, the computing task is very complex and requires distributed computing to deal with. Distributed computing mode is scheduled to use the resources of a cluster of multiple computers or a cloud computing platform. According to the scale and complexity of the computing task, a large amount of computing resources, including CPU, memory, storage, and network bandwidth, are dynamically allocated. It has strong computing power and is suitable for large-scale and highly complex attribute computing tasks.

[0128] During the execution of computing tasks, the utilization of computing resources and the progress of computing tasks are dynamically monitored.

[0129] According to the monitoring results, timely adjust the computing mode and resource allocation to optimize computing performance. If it is found that the resources are insufficient or the computing speed is too slow, you can dynamically increase computing resources or adjust the computing mode.

[0130] Based on the first embodiment and / or the second embodiment and / or the third embodiment of the present application, in the fourth embodiment of the present application, the same or similar contents as those of the above-mentioned first embodiment, second embodiment and third embodiment can refer to the above introduction and will not be repeated later. Figure 4 In this embodiment, step S30, the step of splitting the computing process into different computing subtask packages, further includes:

[0131] Step S31, splitting the calculation process into multiple calculation subtasks;

[0132] Step S32, combining the computing subtasks into task packages for sequential execution;

[0133] Step S33, scheduling the task package according to the current occupancy of the computing resources;

[0134] Step S34, dynamically adjusting the scheduling of the task package according to the increase or decrease of the current computing resources.

[0135] Specifically, in this embodiment, in order to efficiently utilize computing resources and optimize computing performance, it is usually necessary to split a complex computing process into multiple independent computing subtasks and manage and execute them in the form of task packages. The following is a detailed description of step S30:

[0136] Step S31: Split the computational process into multiple computational subtasks. Perform a detailed analysis of the entire computational process to identify computational steps or modules that can be executed independently. Ensure that each computational subtask is complete, executable, and has clear inputs and outputs. Based on the results of the process analysis, split the computational process into multiple computational subtasks. Each computational subtask should be as small as possible to make it easier to process in parallel, while maintaining a certain granularity to ensure the independence and integrity of the task. Describe each computational subtask in detail, including its input data, output data, computational logic, required resources, etc. This helps with subsequent task management and resource allocation.

[0137] Step S32: Combine the computing subtasks into task packages for sequential execution. Combine computing subtasks with dependencies into task packages. The computing subtasks in each task package should be arranged in their logical order to ensure the correctness and efficiency of the calculation. In the task package, the computing subtasks are executed in sequence. Ensure that each computing subtask is executed only after the predecessor task on which it depends is completed. Monitor the execution status of each computing subtask in the task package. If a computing subtask fails or times out, take corresponding error handling measures, such as retrying, rolling back, or notifying the administrator.

[0138] Step S33: Schedule the task package according to the current occupancy of computing resources. Monitor the occupancy of computing resources in real time, including CPU, memory, disk I / O, etc. Evaluate the availability and performance bottleneck of current resources. According to the results of resource monitoring, assign the task package to the appropriate computing node or computing cluster. Prioritize the assignment of task packages to computing nodes with sufficient resources and good performance to improve computing efficiency. In the process of task assignment, consider the factor of load balancing. Avoid scheduling too many task packages to the same computing node to prevent resource overload and performance degradation.

[0139] Step S34: dynamically adjust the scheduling of task packages according to the increase or decrease of current computing resources. Monitor the increase or decrease of computing resources in real time, including newly added computing nodes, exited computing nodes, resource expansion or reduction, etc. Dynamically adjust the allocation of task packages according to resource changes. If new computing nodes are added or resources are expanded, more task packages can be allocated to these resources. If computing nodes exit or resources are reduced, the corresponding task packages need to be migrated to other available computing nodes. In the dynamic adjustment process, consider using optimization strategies to improve computing performance. For example, according to the type and characteristics of computing tasks, select appropriate parallel computing modes (such as data parallelism, task parallelism, etc.); according to the type and performance of computing resources, adjust the priority and scheduling strategy of computing tasks. By splitting the computing process into multiple computing subtasks and managing and executing them in the form of task packages, and dynamically adjusting the allocation of task packages according to the current occupancy and increase or decrease of computing resources, efficient management and optimization of large-scale data computing can be achieved. This can not only improve computing efficiency, but also reduce resource consumption, providing a more flexible and efficient solution for the execution of multi-mode computing analysis models of natural resources.

[0140] Further, on this basis, please refer to Figure 5 In this embodiment, step S40, the step of allocating newly added computing resources to the computing subtask package for computing analysis when limited computing resources are dynamically expanded, further includes:

[0141] Step S41, dividing computing subtasks at different scales according to data characteristics of natural resources;

[0142] Step S42, executing a strategy of giving priority to the computing subtasks at different scales that have no dependency to enter the scheduling queue;

[0143] Step S43, according to the computing states of the computing subtasks at different scales without dependencies after the update is completed, a strategy of delaying the computing subtasks with dependencies from entering the scheduling queue is executed.

[0144] Specifically, in this embodiment, in the natural resource multi-mode computing analysis model, it is crucial to execute subtask scheduling strategies at different scales for different types of computing subtasks. This can not only improve computing efficiency, but also ensure the rational use of computing resources. The following is a detailed description of step S40:

[0145] Step S41: Divide the computing subtasks at different scales according to the data characteristics of natural resources. In-depth analysis of the characteristics of natural resource data, including data type, scale, complexity, spatiotemporal distribution, etc.

[0146] According to the characteristics of the data, the computing process is divided into computing subtasks at different scales. These scales can include different levels such as macro, meso and micro. At each scale, the computing task is further refined into independently executable subtasks. Ensure that each subtask has clear input, output and computing logic.

[0147] Describe each subtask in detail, including its computational purpose, required resources, expected results, etc. Set the corresponding priority according to the importance and urgency of the subtask.

[0148] Step S42: Implement a strategy that prioritizes computing subtasks of different scales without dependencies into the scheduling queue. Analyze the dependencies between computing subtasks of different scales. Identify subtasks without dependencies, which can be executed independently without waiting for the results of other tasks. Prioritize computing subtasks without dependencies into the scheduling queue. Schedule these tasks for calculation in sequence according to the priorities of the subtasks and the availability of computing resources. During the scheduling process, consider the optimal use of computing resources. Prioritize allocating resources to subtasks with large computational workloads and high priorities to improve overall computing efficiency.

[0149] Step S43: According to the computing status of the computing subtasks at different scales without dependencies after the update is completed, the strategy of delaying the computing subtasks with dependencies from entering the scheduling queue is implemented. The computing status of the computing subtasks without dependencies is monitored in real time. When the calculation of these tasks is completed, their status information is updated and the scheduling system is notified. For the computing subtasks with dependencies, check whether their predecessor tasks have been completed. If the predecessor tasks have been completed, these tasks are placed in the scheduling queue waiting for scheduling; if the predecessor tasks are not completed, continue to wait. According to the dependency check results, the computing subtasks with dependencies are delayed from entering the scheduling queue. Ensure that these tasks are executed only after their predecessor tasks are completed to maintain the correctness and integrity of the computing process. In the process of delayed scheduling, consider the dynamic adjustment of computing resources. If resources are sufficient and the predecessor tasks have been completed, the subtasks with dependencies can be scheduled in advance; if resources are tight, the scheduling is appropriately delayed to balance computing demand and resource supply. By dividing computing subtasks at different scales according to the data characteristics of natural resources, and implementing the strategy of prioritizing the scheduling of subtasks without dependencies and delaying the scheduling of subtasks with dependencies, the computing process can be finely managed and optimized. This can not only improve computing efficiency, but also ensure the rational use of computing resources and the accuracy of computing results.

[0150] The above are only preferred embodiments of the present application, and the patent scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made by using the contents of the specification and drawings of this application, or directly or indirectly applied in other related technical fields, are similarly included in the patent processing scope of the present application. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the instructions in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

Claims

1. A scheduling method for a natural resource multi-mode computing analysis model, characterized in that: The method comprises the following steps: By establishing a unified virtualization computing analysis model, scheduling analysis between different computing modes is performed; the virtualization computing analysis model is expressed as: Among them, the model It is a model variable, representing the cache distance in buffer analysis; It is a variable of different computing frameworks, indicating the data partitioning parameters and sampling parameters of parallel computing; It is a service variable, which indicates the computing resources used by models under different frameworks; Based on the scheduling analysis between different computing modes, the computing modes and computing resources required for each stage of the computing process are scheduled; Splitting the computing process into different computing subtask packages; When limited computing resources are dynamically expanded, newly added computing resources are dispatched and allocated to the computing subtask packages for computing and analysis; The step of establishing a unified virtualized computing analysis model to perform scheduling analysis between different computing modes also includes: Setting the common variables of the virtualized computing analysis model; Configure computing framework type variable parameters and service type variable parameters under different computing modes; the computing framework type variable parameters and service type variable parameters are parameters corresponding to the computing framework type variables and service type variables in the virtualized computing analysis model; According to different computing modes, the computing framework type variable parameters and the service type variable parameters are added to the virtualized computing analysis model to obtain the virtualized computing analysis model; According to the virtualized computing analysis model, scheduling analysis between different computing modes is performed.

2. The method according to claim 1, characterized in that The step of scheduling the computing modes and computing resources required for each stage in the computing process according to the scheduling analysis between different computing modes also includes: Calculate the amount of information on the input data; Different computing modes and computing resources are scheduled based on the amount of data information. When the input is an intermediate value of other calculation results, the information amount of the intermediate value calculated in the previous step is used as input, and the information amount of the intermediate value is pre-calculated according to the different calculation modes; After the amount of information calculated at each stage in the entire process is completed, the calculation mode and the scheduling of the calculation resources are performed.

3. The method according to claim 2, characterized in that The step of scheduling different computing modes and computing resources based on the amount of data information as a scheduling basis also includes: When the computing and analysis type is spatial computing type, the minimum value of the amount of spatial computing information in the input data is used as the scheduling basis; when the data to be input is data related to spatial information, the computing and analysis type of the corresponding virtualized computing and analysis model is spatial computing type; When the calculation and analysis type is attribute calculation type, the maximum value of the attribute calculation information in the input data is used as the scheduling basis; when the data to be input is data related to attribute information, the calculation and analysis type of the corresponding virtualization calculation and analysis model is attribute calculation type.

4. The method according to claim 3, characterized in that When the calculation and analysis type is a spatial calculation type, the step of using the minimum value of the spatial calculation information amount in the input data as the scheduling basis also includes: When it is determined that the computing analysis type is a spatial computing type and the minimum value of the spatial computing information amount is in the first interval, scheduling instant computing and corresponding computing resources; When it is determined that the computing analysis type is a spatial computing type and the minimum value of the spatial computing information amount is in the second interval, scheduling parallel computing and corresponding computing resources; When it is determined that the computing analysis type is a spatial computing type and the minimum value of the spatial computing information amount is in a third interval, distributed computing and corresponding computing resources are scheduled.

5. The method according to claim 4, characterized in that When the calculation and analysis type is attribute calculation type, the step of using the maximum value of attribute calculation information in the input data as the scheduling basis also includes: When it is determined that the computing analysis type is an attribute computing type and the maximum value of the attribute computing information amount is in the first interval, scheduling instant computing and corresponding computing resources; When it is determined that the computing analysis type is an attribute computing type and the maximum value of the attribute computing information amount is in the second interval, scheduling parallel computing and corresponding computing resources; When it is determined that the computing analysis type is the attribute computing type and the maximum value of the attribute computing information amount is in the third interval, the distributed computing and the corresponding computing resources are scheduled.

6. The method according to claim 5, characterized in that The step of splitting the computing process into different computing subtask packages also includes: Splitting the computing process into multiple computing subtasks; Combining the computing subtasks into task packages for sequential execution; Scheduling the task package according to the current occupancy of the computing resources; The scheduling of the task packages is dynamically adjusted according to the increase or decrease of the current computing resources.

7. The method according to claim 6, characterized in that The step of scheduling newly added computing resources to be allocated to the computing subtask package for computing and analyzing when the limited computing resources are dynamically expanded also includes: Divide computing subtasks at different scales according to the data characteristics of natural resources; A strategy for executing computing subtasks at different scales without dependencies to be prioritized into the scheduling queue; According to the computing states of the computing subtasks at different scales without dependencies after the update is completed, a strategy is implemented to delay the computing subtasks with dependencies from entering the scheduling queue.

8. A scheduling device for a natural resource multi-mode computing and analysis model, characterized in that: The device comprises: a model startup module, a data scheduling module, a process splitting module and a strategy execution module; The model startup module performs scheduling analysis between different computing modes by establishing a unified virtualized computing analysis model; the virtualized computing analysis model is represented as: Among them, the model It is a model variable, representing the cache distance in buffer analysis; It is a variable of different computing frameworks, indicating the data partitioning parameters and sampling parameters of parallel computing; It is a service variable, which indicates the computing resources used by models under different frameworks; The data scheduling module schedules the computing modes and computing resources required at each stage of the computing process based on the scheduling analysis between different computing modes; A process splitting module, which splits the computing process into different computing subtask packages; A strategy execution module allocates newly added computing resources to the computing subtask packages for computing analysis when limited computing resources are dynamically expanded; The model startup module is also used to set the common variables of the virtualized computing analysis model; The model startup module is also used to configure the computing framework type variable parameters and the service type variable parameters under different computing modes; the computing framework type variable parameters and the service type variable parameters are respectively the parameters used by the computing framework type variables and the service type variables in the virtualized computing analysis model; The model startup module is further used to complete the computing framework type variable parameters and the service type variable parameters into the virtualized computing analysis model according to different computing modes to obtain the virtualized computing analysis model; The model startup module is also used to perform scheduling analysis between different computing modes according to the virtualized computing analysis model.

9. A scheduling device for a natural resource multi-mode computing and analysis model, characterized in that: The scheduling device of the natural resource multi-mode computing analysis model comprises: a memory, a processor, and a scheduling processing program of the natural resource multi-mode computing analysis model stored in the memory and executable on the processor. When the scheduling processing program of the natural resource multi-mode computing analysis model is executed by the processor, the steps of the scheduling method of the natural resource multi-mode computing analysis model as described in any one of claims 1 to 7 are implemented.

Images