An open-innovation service system of natural-social coupling mode

Abstract

The application discloses a kind of natural-social coupling mode's crowd-creation service system, including one-stop user end, running control service end and high-performance computing end;The one-stop user end is used to provide the code collaboration of natural-social coupling mode research, test construction, test submission, running monitoring and result evaluation one-stop interactive operation interface;The running control service end is used for the link of mode test each link and the interaction with relevant platform;Test link includes coupling test deployment, workflow generation, workflow engine interaction, coupling test scheduling operation and inspection calculation execution;Relevant platform includes high-performance computer system, code management platform, workflow engine and inspection tool;The high-performance computing end is used for scheduling high-performance heterogeneous computing resources, operating environment and shared data environment.Utilize the system for natural-social coupling mode's crowd-creation service, can improve the mode coupling collaborative research efficiency of natural-social coupling mode.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cross-domain computing application integration. In particular, it is a crowd-sourcing service system based on natural-social coupling mode. BACKGROUND

[0002] Climate change has a profound impact on land, freshwater, and marine ecosystems, affecting not only water security and natural ecosystems, but also human culture and well-being. Since the 1950s, human activities have had a greater impact on global warming than natural influences.

[0003] The natural-social coupling mode system couples earth system models, multi-scale energy economic models, and other social and economic models. It is a cross-disciplinary study of the socio-economic impacts of climate change and the feedback patterns of human-earth systems. The earth system model is a numerical prediction system that includes complex data preprocessing, data assimilation, prediction integration, and post-processing processes. It is mainly implemented in Fortran language. The multi-scale energy economic model and other social and economic models are mainly implemented in Python, C / C++, and AI computing. The coupling process involves complex heterogeneous collaborative computing and multi-scale, multi-type data coupling and exchange processes.

[0004] Therefore, the operation of the natural-social coupling mode system formed by coupling the earth system model and the social and economic model is a complex computing process and heterogeneous high-performance computing process involving cross-disciplinary research. How to collaboratively develop the natural-social coupling mode system by scientists from different fields, conduct heterogeneous computing, evaluate the effectiveness, and provide one-stop research support tools for the whole chain of fine-grained monitoring, solve the pain points of cross-disciplinary collaboration, and improve the efficiency of collaborative research on model coupling has become a problem to be solved.

[0005] The end-to-end software development and management tools and DevOps development and operation integrated platforms in the industry mainly face the management of general software project engineering needs, iteration follow-up, software testing, and continuous integration and deployment, helping enterprises to respond efficiently, improve the level of research and development automation, agile development, and delivery quality. The crowd-sourcing platform in the industry mainly builds an open space and platform for Internet collaborative innovation, technology exchange, creative communication, capital and technology docking, and innovation and entrepreneurship activities, providing functions such as scientific research project collaborative research, technology problem solving, and wisdom exchange.

[0006] The above tool and platform support technology does not involve complex scientific calculation, multi-scale coupling data exchange, and supercomputer heterogeneous computing and complex computing process monitoring technology, and does not have the one-stop research support capability of natural-society coupling mode system collaborative development, heterogeneous computing, effect evaluation and running full-chain fine-grained monitoring for natural-society coupling mode scientists in different fields. Therefore, it is necessary to develop a one-stop natural-society coupling mode full-process integrated application system to solve the problem of low cross-disciplinary collaboration efficiency. SUMMARY

[0007] Therefore, the technical problem to be solved by the present application is to provide a natural-society coupling mode crowdsourcing service system integrating collaborative research and development, high-performance computing, and running process scheduling and monitoring, to solve the cross-disciplinary collaboration pain points of natural-society coupling mode and improve the efficiency of natural-society coupling mode collaborative research.

[0008] To solve the above technical problems, the present application provides the following technical solutions:

[0009] A natural-society coupling mode crowdsourcing service system, comprising a one-stop user end, a running control service end and a high-performance computing end; the running control service end controls the interactive communication of the one-stop user end and the high-performance computing end; specifically:

[0010] The one-stop user end is used to provide a one-stop interactive operation interface of code collaboration, test construction, test submission, running monitoring and result evaluation;

[0011] The running control service end is used to realize the linking of each link of natural-society coupling mode test and the interaction with related platforms; the links of natural-society coupling mode test include test deployment, workflow generation, workflow engine interaction and verification calculation execution; the related platforms include high-performance computer systems, code management platforms, workflow engines and verification tools;

[0012] The high-performance computing end is used to schedule high-performance heterogeneous computing resources, running environments and shared data environments, and provide natural-society coupling mode computing instances and verification calculation instances.

[0013] The above natural-society coupling mode crowdsourcing service system, the one-stop user end includes a code collaboration module, a test construction module, a test submission module, a running monitoring module and a result evaluation module; the one-stop integration of the code collaboration module, the test construction module, the test submission module, the running monitoring module and the result evaluation module is realized through WEB integration and RestFul interface technology.

[0014] The crowdsourcing service system of the natural-social coupling mode, the code collaboration module calls the integrated Git code warehouse of the running control server through the one-stop user end using interface technology, provides code hosting, version management and integrated sharing of the natural-social coupling mode for researchers in different fields; researchers can directly access the code warehouse, perform code addition, deletion, modification, inquiry, integration, merging, version acquisition and sharing between different researchers;

[0015] The test construction module is used to provide a test construction process based on a test template and components, and realize crowdsourcing sharing of the natural-social coupling mode collaborative research;

[0016] The test submission module is used to initiate a test submission request on the one-stop user end, cache test configuration information according to key-value to a data dictionary, receive instructions from the running control server to schedule the workflow construction module, generate a workflow definition and a configuration file, then structurally organize the workflow definition and the configuration file and the mode running component module, complete workflow assembly, generate an executable automatic workflow for the test, and generate a test instance, trigger workflow scheduling job running;

[0017] The running monitoring module is used to realize full-process visual monitoring and alarm of the test running process, view error output, view progress logs, and debug and re-submit jobs;

[0018] The result evaluation module is used to configure test calculation parameters, submit test calculation tasks, receive test index drawing results generated by the high-performance computing end, and provide test results and comprehensive analysis for users.

[0019] The crowdsourcing service system of the natural-social coupling mode, the test construction module has three ways of providing test rapid construction for users, namely test copy, test template and interactive drag-and-drop running component; the test copy quickly generates a new test scheme by copying a specified historical test, improving test efficiency;

[0020] The test template faces the system test of the natural-social coupling mode, is composed of common operation processes, operation components and operation configurations, constructs a test template for a certain field research and reusability, and is used for rapid construction of a specific solidification test or a common test; the operation component encapsulates the operation modules and default configurations of each component mode and model of the natural-social coupling mode system by using a workflow component technology, and includes an earth system mode preprocessing module, an assimilation module, a prediction module, a post-processing module, and operation components of global and regional energy and economic models; then, a running process of the natural-social coupling mode system is visually constructed by using a workflow arrangement drag-and-drop; the interactive drag-and-drop operation component supports different field scientists to construct a complex coupling test by dragging and dropping components, adding component dependencies and selecting configuration parameters, automatically converts a test description constructed by the user into a specific task instance that can be executed in a high-performance computing environment, and automatically submits the specific task instance to a bottom workflow engine and a job scheduling system for running.

[0021] The specific process of the test rapid construction of the natural-social coupling mode crowdsourcing service system is as follows: after a field scientist constructs a natural-social coupling mode test by using a test template or constructs the test based on a drag-and-drop operation component through the one-stop user end, the interactive configured test parameters are written into a database, and test information is generated.

[0022] The running monitoring module of the natural-social coupling mode crowdsourcing service system faces the running whole-process visualization of a complex coupling test of the natural-social coupling mode, schedules and executes according to a cycle and a dependency relationship of a test task, and tracks a running state of the complex coupling test of multiple models, multiple steps and possibly running on different computing nodes in real time, identifies a state of a task by capturing a normal or abnormal signal returned by a Linux operating system level task, and

[0023] The running process of the earth system mode can be tracked and monitored line by line, the test running process and the real-time monitoring running state are displayed in a tree structure on the one-stop user end, a test failure module is alarmed, error output can be directly viewed, progress logs can be viewed, integrated debugging and rework are supported, the user does not need to jump to a bottom command line or a complex job management system, error troubleshooting efficiency is improved, and the start and end times of a computing task can be monitored, so that a computing task time consumption can be calculated.

[0024] The crowdsourcing service system of the natural-social coupling mode provides a pre-defined or user-configurable standardized test index calculation process and a drawing template for the natural-social coupling mode research. A user only needs to select a test index and configure a test parameter on a one-stop user end. A running control server builds a test calculation task, a high-performance calculation end builds a test calculation instance, and the high-performance calculation end executes the test. The platform of the crowdsourcing service system automatically manages the submission, queuing, and execution of tasks, and returns the test results to the one-stop user end for display. The crowdsourcing service system can provide comprehensive display and comparison of test results, support the display of a single test result, and support the automatic comparison and visualization of results of different test schemes and different parameter configurations, thereby facilitating the user to evaluate the differences between different scenarios.

[0025] The crowdsourcing service system of the natural-social coupling mode comprises a running control server.

[0026] The code management module is configured to integrate a Git code repository, host natural-social coupling mode codes, and provide a version management mechanism.

[0027] The mode running component module is configured to implement a test process arrangement and scheduling based on a workflow. The running modules of various component modes and models of the natural-social coupling mode system are encapsulated as running components recognized by a workflow engine. For example, the running components of a pre-processing module, an assimilation module, a prediction module, and a post-processing module included in an earth system mode component, and components such as a global energy and economic model are core technical components supporting complex system research. The pre-processing component is configured to shield technical details of a high-performance computing heterogeneous environment, such as model compilation, dependency management, and data format linkage. The pre-processing component mainly includes a compilation environment pre-processing component, a job card pre-processing component, and a shared data environment pre-processing component. The compilation environment pre-processing component automatically detects a platform of a high-performance computer to select an appropriate compiler. The job card pre-processing component encapsulates different job cards according to characteristics of serial and parallel tasks to schedule different resource queues and resource quantities. In the pre-processing component, parameter values interactively constructed by a one-stop user end can be passed from a workflow definition through a variable, thereby flexibly adapting to individualized configurations of parameter adjustment of different tests and solving problems such as poor model compatibility and complex parameter configuration in multidisciplinary research.

[0028] The test template module is configured to provide and store a standardized test template, including a running process of a standard test, test parameters, calculation resources, and a default code version. The test template module indexes and retrieves related configuration information corresponding to a test template through a test template name, receives a call from the one-stop user end, and automatically configures a running process, test parameters, calculation resources, a default code version, and data according to a template constraint, calls a workflow construction module, and constructs a test instance.

[0029] The workflow construction module is configured to receive the test flow information, test configuration information and test configuration information of the one-stop user terminal interaction, and obtain and parse all test parameters from a database and the one-stop user terminal to generate a workflow definition and a workflow configuration file; the workflow definition file, a preprocessing component and a running component are structured and organized, so that the assembly of the workflow is completed, an executable automatic workflow for the test is generated, and a test instance is generated to trigger the workflow scheduling job running;

[0030] The database management module is configured to store and manage system data by using an RDBMS, including user information, test template information, test configuration information, test management information, permission information, log information and alarm information; based on the stored test information data, the test management sharing function is provided for research users to facilitate the callback, reproduction and sharing of tests;

[0031] The communication control module adopts a standardized message definition format, and realizes the communication of system interaction of the one-stop user terminal, the running control server and the high-performance computing terminal based on a message middleware technology. The communication control module completes the full-process automatic control of the user terminal to the bottom layer through control logic.

[0032] In the above crowd-creation service system of the natural-social coupling mode, the method for generating the workflow definition is that: a trigger is added to build a test running flow according to a predefined execution order for a mode module selected by a user, and resources, loops and running control parameters configured by the user are obtained; the method for generating the workflow configuration file is that: a parameter file is generated after data and test parameter schemes are parsed, a preprocessing component is used in a running component, and the parameter file is introduced to use the parameters therein;

[0033] In the database management module, the test configuration information is mainly metadata management of the test, including a source code version, parameterized scheme information, resource information, running data list and archiving space information of the test;

[0034] The test management sharing function includes test comparison, test data management, test query statistics, test deletion, test copying, test sharing and test template management, supports test comparison, provides one-key comparison of test scheme information, combines test result verification and comparison, and supports users to comprehensively compare test schemes and improvement effects;

[0035] The test query statistics provide test query and statistics of related information of the number of tests in a specified time period; and the test template management stores common or commonly used test types as test templates, and quickly constructs a new test flow and parameterized scheme based on the test templates;

[0036] The test sharing function supports users to share tests with others for comparison, copying and reproduction, supports the like and comment functions of the test, improves the test management sharing level, and solves the problems of test difficulty in reproduction and sharing;

[0037] The communication control module adopts a standardized queue management design, adopts a multi-queue mode, includes a user queue, a mode queue and a receipt queue, and based on the above standardized rules, a Python automatic communication expansion service is developed, and a modular calling mode is added and deleted, and a message communication expansion function is added and deleted by the user.

[0038] The crowd service system of the natural-social coupling mode has the following advantages:

[0039] The multi-mode shared data management module is used for sharing and managing massive earth system modes and input data and test data of multi-scale energy economic models, including mode background field data, observation data, assimilation data and static data required by the earth system mode, and greenhouse gas, ozone, solar radiation and aerosol data required by the social economic model, to support the calling of test instances and test instances during operation;

[0040] The mode coupling running instance module is used for completing test code deployment, compilation and installation, data linking, and test running process scheduling; and according to the cycle and dependence relationship of the test task, the running state is tracked in real time, and the test running process is returned and displayed to the one-stop user end through the running control service end.

[0041] The test calculation instance module is used for completing test data and comparison data linking of test calculation, deployment and execution of test algorithms, generation of index results, index drawing and graphic pushing.

[0042] The technical scheme of the present application has the following beneficial technical effects:

[0043] 1、The crowd service system of the natural-social coupling mode has the following advantages:

[0044] 2. The crowd-innovation service system of the nature-society coupling model of the present invention creatively integrates the scattered technologies in the background technology into a seamless end-to-end research platform (from code collaboration -> experiment construction -> submission and operation -> monitoring -> result evaluation), solving the problems of broken tool chains, high learning costs and low efficiency in traditional research. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Architecture diagram of the crowd innovation service system based on the nature-society coupling model of the present invention. DETAILED DESCRIPTION

[0046] like Figure 1 As shown, the crowd innovation service system of the nature-society coupling mode of this embodiment includes a one-stop user terminal, an operation control service terminal and a high-performance computing terminal.

[0047] Among them, the one-stop user terminal includes a code collaboration module, an experiment construction module, an experiment submission module, an operation monitoring module and a result evaluation module; the one-stop user terminal provides research users with an interactive interface for the development, testing, monitoring and evaluation of the entire chain of nature-society coupling models.

[0048] The operation control server includes a code management module, a mode operation component module, a test template module, a workflow construction module, a database management module, and a communication control module; the operation control server is responsible for receiving and processing interaction requests from the one-stop user side, performing multi-module collaborative control, and building computing instances on the high-performance computing side.

[0049] The high-performance computing end includes a multi-mode shared data management module, a mode coupling operation instance module and a verification computing instance module; the high-performance computing end is responsible for receiving collaborative control instructions from the operation control server, executing code compilation, data interaction, computing operation and result output of the computing instance.

[0050] In this embodiment, the method for implementing the crowd innovation service system in the nature-society coupling mode is as follows:

[0051] 1. One-stop user terminal

[0052] The one-stop user terminal provides a multifunctional, integrated interactive interface for users of crowdsourced research on nature-society coupling models. It offers one-stop operations for code collaboration, experiment construction, experiment submission, operation monitoring, and results evaluation. This integrated presentation and interaction connects the entire process of model research, experimentation, and evaluation, eliminating the need for users to interact with separate IT systems through command lines or scripts. This shields the underlying details and provides a one-stop user experience. The one-stop integration of the code collaboration module, experiment construction module, experiment submission module, operation monitoring module, and results evaluation module is achieved through web integration and RestFul interface technology.

[0053] (1)Code collaboration module: The integration Git code repository of the code management module of the running control service side is called through the interface technology of the one-stop user side, which provides code hosting, version management, integration and sharing for researchers in different fields of natural-social coupling mode, supports cross-field and distributed collaborative research, and improves the efficiency of collaborative research.

[0054] (2) Test construction module: In view of the complexity and inefficiency of manual batch script programming test, a test construction process based on templates and components is provided for researchers in natural-social coupling mode, realizing the crowd sharing of natural-social coupling mode collaborative research; the test template is used for standard test, providing solidified standardized test configuration, including test process, test parameter and computing resource; the code version to be run can be selected, and the component-based test construction is used for personalized test debugging, providing default parameter configuration, which reduces the interaction complexity of coupling research by experts in different fields; the test process is built by dragging the coupling mode module components, and the test parameters and computing resources are configured, and the test code version is selected.

[0055] The WEB process arrangement framework is realized by using LogicFlow technology, and the execution order of each module of natural-social coupling mode and the dependency between them are arranged by dragging, which shields the differences in the running process of different model development languages.

[0056] The test template is used for the system test of natural-social coupling mode, which is composed of common running process, running component and running configuration, and the test template for a certain field research and reusable is constructed, which is used for the rapid construction of specific solidified test or common test; the running component encapsulates the running module and default configuration of each component mode and model of natural-social coupling mode system by using workflow component technology, including the running component of earth system mode preprocessing, assimilation, prediction and post-processing module, and global and regional energy and economic model components; then the running process of natural-social coupling mode system is built by visual drag-and-drop workflow arrangement; the interactive drag-and-drop running component supports different field scientists to build complex coupling test by dragging components, adding component dependencies and selecting configuration parameters, automatically converts the test description built by users into specific task instances that can be executed in high-performance computing environment, and automatically submits to the underlying workflow engine and job scheduling system for instance running.

[0057] The specific process of rapid test construction by using test template is as follows: after the natural-social coupling mode test is built by the test template or based on the running component drag-and-drop by the field scientist on the one-stop user side, the interactive test parameters are written into the database, and the test information is generated.

[0058] The above interactive interfaces provide default parameters and support customization, and provide test quick construction based on test copies, test sharing to support crowd research, and improve reusability, speed and flexibility.

[0059] (3) Test submission module: triggers the running control server to store the test information formed by the test construction module and generates test instances, and triggers workflow scheduling job running. The test submission module is used to initiate a test submission request on the one-stop client, cache test configuration information according to key-value to a data dictionary, receive instructions by the running control server to schedule the workflow construction module, generate workflow definition and configuration file; then structure the workflow definition and configuration file and the mode running component module, complete the assembly of the workflow, generate an executable automatic workflow for the test, and generate test instances, trigger workflow scheduling job running.

[0060] (4) Running monitoring module: In order to solve the problem that the original command line batch processing mode of natural-social coupling mode lacks visual monitoring and alarm, and cannot timely find problems in the complex calculation chain in the coupling process, the workflow monitoring technology is adopted to provide task component level full process, fine-grained real-time running visual monitoring display and alarm for the complex coupling calculation process of natural-social coupling mode, view error output, view progress log, and debug and re-submit jobs, which can significantly reduce the delay in discovering coupling abnormalities and improve research test efficiency.

[0061] The running monitoring module is directed to the visual running of the whole process of the complex coupling test of the natural-social coupling mode, schedules and executes according to the cycle and dependency relationship of the test task, and tracks the running state of the complex coupling test of multiple models, multiple steps and possibly running on different computing nodes in real time. By capturing the normal or abnormal signal returned by the Linux operating system level task, the state of the task is identified; for the earth system mode, the running process can be tracked and monitored line by line, and the test running process and real-time monitoring state are displayed in a tree structure on the one-stop client, the test failure module is alarmed, the error output is directly viewed, the progress log is viewed, and integrated debugging and re-submit jobs are supported.

[0062] (5) Result evaluation module: configures test calculation parameters, submits test calculation tasks, receives test index drawing results generated by high-performance computing end test calculation instances, provides test results for users, and integrates test evaluation index display to support fast evaluation of mode collaborative research.

[0063] The result evaluation module integrates standardized and configurable test processes, provides pre-defined or user-configurable standardized test index calculation processes and drawing templates for natural-social coupling mode research, and after the user selects test indexes and configures test parameters on the one-stop user terminal, the running control server builds a test calculation task, builds a test calculation instance on the high-performance computing terminal, and executes the test calculation instance by the high-performance computing terminal; the platform of the crowdsourcing service system automatically manages the submission, queuing and execution of tasks, and returns the test results to the one-stop user terminal for display, and can provide comprehensive display and comparison of test results; it supports not only the result graph of a single test, but also the automatic comparison and visualization of results of different test schemes and different parameter configurations.

[0064] 2. The running control server

[0065] The natural-social coupling mode crowdsourcing service running control server is responsible for linking each link of the mode test, including test deployment, workflow generation, workflow engine interaction, test calculation execution, and interaction with related platforms, including high-performance computer systems, code management platforms, workflow engines, test tools, etc. Among them, the interaction between the server and the workflow engine is the core of automatic deployment and operation of the test.

[0066] (1) Code management module: integrate Git code repository, host natural-social coupling mode code, and provide version management mechanism.

[0067] (2) Mode running component module: encapsulate the running modules of each component mode and model of the natural-social coupling mode system as standardized running components recognized by the workflow engine, for workflow-based test process arrangement and scheduling; including loop control components, earth system mode components, global and regional energy and economic model components, compilation and construction components, test calculation components, etc. The components include code configuration, data configuration, parameter configuration, event configuration, resource configuration, test analysis configuration, etc.

[0068] (3) Test template module: test templates store standardized tests, including standard test running processes, test parameters, computing resources, and default code versions, and retrieve the related configuration information corresponding to the test template through the test template name index, and receive the call of the one-stop user terminal, and automatically construct test instances according to the template-constrained running processes, test parameters, computing resources, default code versions and data configurations, and call the workflow construction module; the one-stop user terminal can quickly construct tests and modify default parameters by calling test templates, providing a quick construction method for regular solidified tests, and can customize common test templates, share templates for reuse, and improve collaborative crowdsourcing efficiency.

[0069] (4) Workflow construction module: receiving the test flow, test configuration and test configuration information of one-stop user terminal interaction, and obtaining all test parameters from the database and the one-stop user terminal and parsing, generating workflow definition and workflow configuration file; structuring the workflow definition file, preprocessing component and running component, which can complete the assembly of the workflow, generate an executable automatic workflow for the test, and generate a test instance, trigger workflow scheduling job running; the embodiment uses XML data encapsulation technology for encapsulation, constructs a running workflow instance, generates a workflow definition and an instantiation of a calculation task component.

[0070] The method for generating a workflow definition is to add a trigger to build a test running flow according to a predefined execution order, and obtain user-configured resources, loops and running control parameters; the method for generating a workflow configuration file is to generate a parameter file after parsing data, test parameter schemes and the like, use a preprocessing component in a running component, and introduce the parameter file to use the parameters therein.

[0071] (5) Database management module: used for storage and management of system data, using RDBMS management, including user information, test template information, test configuration information, test management information, permission information, log information and alarm information. Based on the stored test information data, the test management sharing function is provided for research users.

[0072] The test configuration information is mainly metadata management of the test, including the source code version used by the test, parameterization scheme information, resource information, running data list and archiving space information;

[0073] The test management sharing function includes test comparison, test data management, test query statistics, test deletion, test copy, test sharing and test template management, supports test comparison, provides one-key comparison of test scheme information, combines test result verification and comparison, and supports users to comprehensively compare test schemes and improve effects;

[0074] The test query statistics provide test query and test quantity-related information statistics in a specified time period; the test template management stores common or commonly used test types as test templates, and quickly constructs new test flows and parameterization schemes based on the test templates;

[0075] The test sharing function supports users to share tests with others for comparison, copying and reproduction, and supports the functions of liking and commenting on the test.

[0076] (6) Communication control module: using standardized message definition format, based on message middleware technology to realize the communication of system interaction of system user end, service end and high-performance computing end, and through control logic to complete the whole process automation control from user end to bottom.

[0077] The communication control module adopts standardized queue management design, adopts multi-queue mode, including user queue, mode queue and reply queue, and based on the above standardized rules, develops Python automatic communication expansion service, modularly calls mode adding and deleting, and user adding and deleting message communication expansion function.

[0078] 3, High-performance computing end

[0079] The high-performance computing end provides high-performance heterogeneous computing resources, running environment and shared data environment for the natural-society coupling mode, supports the calculation running of the natural-society coupling mode and the verification calculation and drawing of the output result.

[0080] (1) Multi-mode shared data management module, used for shared management of massive earth system mode and multi-scale energy economic model input data, verification data, etc., including mode background field data, observation data, assimilation data and static data required by the earth system mode, and greenhouse gas, ozone, solar radiation and aerosol data required by the social economic model, to support the calling of test instances and verification instances running.

[0081] (2) Mode coupling running instance module, which completes test code deployment, compilation and installation, data linkage, test running process scheduling, executes according to the cycle and dependence relationship of test task, and tracks the running state in real time, and returns and displays the test running process to the user end through the service end.

[0082] (3) Verification calculation instance module, which completes the linkage of verification data and comparison data of verification calculation, the deployment and execution of verification algorithm, the generation of index result, the drawing of index and the pushing of graphics.

[0083] Next, taking the collaborative research of different researchers using the natural-society coupling mode of the invention through test tasks as an example, the running process of the natural-society coupling mode of the invention is further described.

[0084] The running process of the natural-society coupling mode of the invention is as follows:

[0085] (1) Different researchers in different fields use the code collaboration module on the interface of the one-stop user end to develop, save and integrate the code part of the natural-society coupling mode in the field, and need to further verify the improvement effect of the mode, and call the test construction module to perform coupling test construction operation;

[0086] (2) The code management module stores and records the code version;

[0087] (3) The test construction module provides three ways of test construction, including test template construction, test copy construction, and drag-and-drop running component assembly construction, which calls the test template module and the mode running component module to construct the test, and calls the database management module to record the test information;

[0088] (4) The workflow construction module receives and processes the test flow information, test configuration information and test configuration information of the test construction module, generates workflow definition and configuration file, and generates executable automatic workflow for test;

[0089] (5) The test submission module receives and stores the workflow generated by the workflow construction module, triggers the workflow job submission to the high-performance computing end, executes mode compilation through the mode coupling running instance module, links the multi-mode shared data management module to build the data environment required by instance running, and executes the workflow;

[0090] (6) The running monitoring module visually monitors and displays the whole process of the submitted test running process and alarms, and locates and analyzes the error alarm part and re-submits it;

[0091] (7) The result evaluation module configures test indicators, calls the test calculation instance module through the communication control module, links the multi-mode shared data management module to build the data required for test calculation, executes test calculation, and transmits the test result to the result evaluation module for display, providing test result analysis for users and evaluating R&D effect;

[0092] (8) The database management module stores and manages user information, test template information, test configuration information, test management information, permission information, log information and alarm information;

[0093] (9) The communication control module automatically controls the communication of system interaction of the one-stop user end, the running control service end and the high-performance computing end;

[0094] (10) The multi-mode shared data management module provides the calculation required data for the mode coupling running instance module and the test calculation instance module;

[0095] The crowd-creation service system of the natural-social coupling mode of the embodiment involves mode templates, mode running components, test calculation algorithms and test indexes, which support self-definition and rapid integration, sharing and distribution; the flexible scalability of the operation improves the efficiency of collaborative crowd creation.

[0096] Obviously, the above embodiments are only examples for clearly illustrating, but not limitation of the embodiments. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, all the embodiments need not and cannot be exhausted. The obvious changes or variations derived therefrom are still within the protection scope of the patent application claims.

Claims

1. A crowd innovation service system based on nature-society coupling mode, characterized by: Includes a one-stop user terminal, operation control server terminal and high-performance computing terminal; The one-stop user terminal is used to provide a one-stop interactive operation interface for code collaboration, experiment construction, experiment submission, operation monitoring and result evaluation; The operation control server is used to realize the connection of each link of the nature-society coupling model experiment and the interaction with related platforms; the nature-society coupling model experiment link includes coupling experiment deployment, workflow generation, workflow engine interaction, coupling experiment scheduling operation and verification calculation execution; the related platforms include high-performance computer systems, code management platforms, workflow engines and verification tools; The high-performance computing end is used to schedule high-performance heterogeneous computing resources, operating environments, and shared data environments, provide nature-society coupling model computing instances, and verify the operation of computing instances; The one-stop user terminal includes a code collaboration module, a test construction module, a test submission module, an operation monitoring module and a result evaluation module; the one-stop integration of the code collaboration module, the test construction module, the test submission module, the operation monitoring module and the result evaluation module is achieved through WEB integration and RestFul interface technology; The operation control server includes a code management module, a mode operation component module, a test template module, a workflow construction module, a database management module and a communication control module; The model operation component module includes a cycle control component, an earth system model component, a global and regional energy economic model component, a preprocessing component, a compilation and construction component, and a verification and calculation component; the attributes of each component include code configuration, data configuration, parameter configuration, event configuration, resource configuration, and verification and analysis configuration; The high-performance computing end includes a multi-mode shared data management module, a mode coupling operation instance module and a verification computing instance module; The experiment construction module is used to provide an experiment construction process based on experiment templates, running components or experiment copies, so as to realize the public innovation and sharing of collaborative research on nature-society coupling models; The test submission module is used to initiate a test submission request at the one-stop user terminal, cache the test configuration information in the data dictionary according to key-value, and the operation control server receives the instruction to schedule the workflow construction module to generate a workflow definition and configuration file; Then the workflow definition and configuration file and the mode operation component module are structured and organized to complete the workflow assembly, generate an automatic workflow that can be interpreted and executed for the experiment, generate an experimental instance, and trigger the workflow scheduling job to run.

2. The crowd innovation service system of the nature-society coupling model according to claim 1 is characterized in that: In the one-stop user terminal: The code collaboration module uses the interface technology to call the integrated Git code repository of the code management module through the one-stop user terminal, providing code hosting, version management and integrated sharing in a nature-society coupling model for researchers in different fields; The operation monitoring module is used to realize the visual monitoring display and alarm of the whole process of the test operation, view error output, view progress logs, and debug and resubmit operations; The result evaluation module is used to configure inspection calculation parameters, submit inspection calculation tasks, receive inspection indicator drawing results generated by the inspection calculation instance of the high-performance computing end, and provide users with test results and comprehensive analysis.

3. The crowd innovation service system of the nature-society coupling model according to claim 2 is characterized in that: The test construction module has three ways to provide users with rapid test construction: test copy, test template, and interactive drag-and-drop operation components; test copy quickly generates a new test plan by copying the specified historical test; The test template is designed for system experiments in nature-society coupling models. It consists of common operating processes, operating components, and operating configurations. It builds a reusable test template for research in a certain field and is used for the rapid construction of specific solidified experiments or common experiments. The operational component uses workflow component technology to encapsulate the operational modules and default configurations of each component mode and model of the nature-society coupling model system, including the operational components of the Earth system model pre-processing, assimilation, prediction, and post-processing modules, as well as the global and regional energy economic model components. Then, the operational process of the nature-society coupling model system is constructed in a visual drag-and-drop manner through workflow orchestration. The interactive drag-and-drop running component supports scientists from different fields to build complex coupled experiments by dragging components, adding component dependencies and selecting configuration parameters. It automatically converts the user-built experimental description into specific task instances that can be executed in a high-performance computing environment, and automatically submits them to the underlying workflow engine and job scheduling system running instance.

4. The crowd innovation service system of the nature-society coupling model according to claim 3 is characterized in that: The specific process of using experimental templates to quickly build experiments is as follows: after scientists in a certain field use experimental templates to build a natural-social coupling model experiment on the one-stop user terminal or drag and drop running components, they write the interactively configured experimental parameters into the database to generate experimental information.

5. The crowd innovation service system of the nature-society coupling model according to claim 2 is characterized in that: The operation monitoring module visualizes the entire operation process of complex coupling experiments in nature-society coupling models, schedules execution according to the cycles and dependencies of experimental tasks, and tracks the operation status of complex coupling experiments with multiple models, multiple steps, and running on different computing nodes in real time. It identifies the status of tasks by capturing the normal or abnormal return status signals of Linux operating system-level tasks; For the Earth system model, the operation process can be tracked and monitored line by line, and the test operation process and real-time monitoring operation status can be displayed in a tree structure on the one-stop user terminal, and an alarm can be issued for the test fault module. It supports direct viewing of error output, viewing of progress logs, integrated debugging and resubmission of jobs.

6. The crowd innovation service system of the nature-society coupling model according to claim 2 is characterized in that: The result evaluation module integrates standardized and configurable inspection processes, and provides predefined or user-configurable standardized inspection indicator calculation processes and drawing templates for nature-society coupling model research; After the user selects the inspection indicators and configures the inspection parameters on the one-stop user terminal, the operation control server will build the inspection calculation task, and the high-performance computing terminal will build the inspection calculation instance, which will be executed by the high-performance computing terminal. The platform of the crowd innovation service system automatically manages the submission, queuing and execution of tasks, and transmits the inspection results back to the one-stop user terminal for display, and can also provide a comprehensive display and comparison of the inspection results. It supports not only the result graph of a single test, but also the automated comparative analysis and visual display of the results of different test schemes and different parameter configurations.

7. The crowd innovation service system of the nature-society coupling model according to claim 1 is characterized in that: On the operation control server: The code management module is used to integrate the Git code repository, host the nature-society coupling model code, and provide a version management mechanism; The pattern operation component module is used to implement the experimental process arrangement and scheduling based on the workflow, and encapsulate the operation modules of each component pattern and model of the natural-social coupling pattern system into operation components recognized by the workflow engine; The test template module is used to provide and store standardized test templates, including the operating process, test parameters, computing resources and default code version of the standard test, retrieve the relevant configuration information corresponding to the test template through the test template name index, and receive the call of the one-stop user terminal. The automation part calls the workflow construction module to construct the test instance according to the operating process, test parameters, computing resources, default code version and data configuration constrained by the template; The workflow construction module is used to receive the test process information, test configuration information and inspection configuration information interacted by the one-stop user terminal, and obtain and parse all test parameters from the database and the one-stop user terminal to generate a workflow definition and a workflow configuration file; By structuring the workflow definition file, preprocessing components, and running components, the workflow can be assembled, and an automatic workflow that can be interpreted and executed can be generated for the experiment. The experiment instance is generated and the workflow scheduling job is triggered. The database management module is used to store and manage system data, using RDBMS management, including user information, test template information, test configuration information, test management information, permission information, log information and alarm information; based on the stored test information data, it provides test management sharing functions for research users; The communication control module adopts a standardized message definition format and realizes the system interactive communication between the one-stop user end, the operation control service end and the high-performance computing end based on the message middleware technology, and completes the full-process automated control from the user end to the underlying layer through control logic.

8. The crowd innovation service system of the nature-society coupling model according to claim 7 is characterized in that: In the workflow construction module, the method for generating a workflow definition is as follows: the user-selected mode module is added with triggers according to a predefined execution order to build the test operation process, and the user-configured resources, loops, and operation control parameters are obtained; the method for generating a workflow configuration file is as follows: the data and test parameter scheme are parsed to generate a parameter file, the preprocessing component is used in the operation component, and the parameters in the parameter file are imported to use them; In the database management module, the test configuration information is metadata management of the test, including the source code version, parameterization scheme information, resource information, operation data list and archiving space information adopted by the test; The test management and sharing functions include test comparison, test data management, test query statistics, test deletion, test copying, test sharing, and test template management. They support test comparison and provide one-click comparison of test plan information. Combined with test results, they enable users to comprehensively compare test plans and improve their results. The test query statistics provide statistics on test queries and test quantity within a specified time period; The test template management stores common or frequently used test types as test templates, and quickly builds new test processes and parameterized schemes based on the test templates; The experiment sharing function allows users to share experiments with others for comparison, copying and reproduction, and supports likes and comments on experiments; The communication control module adopts a standardized queue management design and a multi-queue mode, including user queues, pattern queues and receipt queues. Based on standardized rules, it develops Python automated communication extension services and modularly calls message communication extension functions for pattern addition and deletion, and user addition and deletion.

9. The crowd innovation service system of the nature-society coupling model according to claim 1 is characterized in that: On the high performance computing side: The multi-model shared data management module is used to share and manage massive Earth system models and input data and verification data of multi-scale energy and economic models, including model background field data, observation data, assimilation data and static data required by Earth system models, and greenhouse gas, ozone, solar radiation and aerosol data required by socio-economic models, to support the call of test instances and verification instances during operation; The mode coupling running instance module is used to complete the deployment, compilation and installation of test code, data linking, and process scheduling of test operation; it also schedules execution according to the cycle and dependency relationship of test tasks, tracks the running status in real time, and returns and displays the test operation process to the one-stop user terminal through the operation control server; The inspection calculation instance module is used to complete the inspection data and comparison data linking of the inspection calculation, the deployment and execution of the inspection algorithm, the generation of indicator results, indicator drawing and graphic push.

Images