Space science experiment data online collaborative analysis method and system
By creating online research projects for space science data through an online collaborative analysis system, scheduling environmental mirroring and computing resources, the system solves the problems of low efficiency and limited resources in traditional scientific data processing methods, and realizes multi-person collaborative processing and efficient scientific research.
Patent Information
- Application Number
- CN202511237777.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-01
- Publication Date
- 2025-12-05
- Estimated Expiration
- 2045-09-01
AI Technical Summary
Traditional scientific data processing methods are time-consuming and labor-intensive, limited by local computing resources, and inefficient in multi-user collaborative work, making it difficult to meet the needs of modern scientific research. Existing online data processing platforms have not effectively solved the problems of computing resource allocation and collaborative task planning.
This paper provides a method and system for online collaborative analysis of space science experimental data. By creating online research projects for space science data, scheduling environmental mirroring and computing resources, configuring multiple functional components, it enables multi-person collaborative processing, supports fine-grained permission management and project version control, and optimizes the allocation of computing resources.
It enables rapid and accurate analysis of complex scientific tasks in multi-user collaborative processing scenarios, improves research efficiency, optimizes the utilization of computing resources, reduces research costs, and supports multi-user collaboration and results sharing.
Smart Images

Figure CN121073180A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of scientific experiment data processing and analysis, and particularly relates to a space scientific experiment data online collaborative analysis method and system. BACKGROUND
[0002] The space laboratory includes scientific experiment cabinets, out-of-cabin exposure experiment platforms and other scientific research facilities, and can support on-orbit implementation of space life science and biotechnology, microgravity fundamental physics, space material science, space astronomy and many other research fields. Space scientific experiment data has the characteristics of large volume, real-time batch generation, multiple fields, multiple types, multiple forms and wide sources, and has the characteristics of wide, open, cooperation and sharing.
[0003] In scientific research, data processing and analysis is a crucial link. With the rapid development of data science, scientific data processing and analysis is becoming more and more complex. The traditional scientific data processing method usually requires users to build a computing environment locally, which not only consumes time and effort, but also is limited by local computing resources. In addition, complex scientific research projects often require multiple researchers to work collaboratively, and the traditional collaboration method is inefficient and difficult to meet the needs of modern scientific research. The existing online data processing platform has not effectively solved the problems of how to efficiently allocate computing resources, how to manage project versions and how to support complex collaboration task planning when multiple users collaborate.
[0004] Therefore, there is an urgent need for a space scientific experiment data online collaborative analysis method and system that can conveniently complete online processing and analysis of complex scientific tasks, efficiently utilize space scientific experiment data and promote scientific and applied achievements. SUMMARY
[0005] The embodiments of the present application provide a space scientific experiment data online collaborative analysis method and system, which can quickly and accurately complete online processing and analysis of complex scientific tasks in a multi-person collaborative processing scenario, efficiently utilize space scientific experiment data and effectively improve data processing and analysis efficiency.
[0006] To achieve the above-mentioned purpose, the embodiments of the present application adopt the following technical solutions:
[0007] In a first aspect, a method for online collaborative analysis of space science experimental data is provided. The method is applied to an online collaborative analysis system, which is in communication with a first device and a plurality of second devices. The method comprises: creating a space science data online research project in response to a project creation request sent by the first device, the space science data online research project being configured to determine a target result, the target result of the space science data online research project comprising a plurality of sub-results, each of the sub-results being configured with different identification information; the project creation request carrying running environment information; scheduling an environment image and computing resources according to the running environment information to create a running environment in response to an analysis request of the space science data online research project sent by the second device, the running environment being configured with a plurality of functional components; determining a target sub-result based on the plurality of functional components of the running environment in response to an online analysis operation sent by the second device, the target sub-result being one of the plurality of sub-results; and updating the target result according to the identification information of the target sub-result in response to a result update request sent by the second device, the result update request carrying the identification information of the target sub-result.
[0008] In a possible implementation manner of the first aspect, the project creation request further carries permission information of each of the plurality of second devices; and the running environment is created according to the running environment information in response to the analysis request of the space science data online research project sent by the second device, comprising: determining whether the second device has online analysis permission of the space science data online research project according to the permission information in response to the analysis request of the space science data online research project sent by the second device; and creating the running environment according to the running environment information in a case where the second device has the online analysis permission of the space science data online research project.
[0009] In a possible implementation manner of the first aspect, the method further comprises: determining whether the second device has result reading permission of the space science data online research project according to the permission information in response to a result reading request of the space science data online research project sent by the second device; and sending the target result corresponding to the space science data online research project to the second device in a case where the second device has the result reading permission of the space science data online research project.
[0010] In a possible implementation manner of the first aspect, the online collaborative analysis system is further configured with a plurality of space science experimental data sources, the plurality of space science experimental data sources comprising a local data set, a database, object storage and NAS space; and the target sub-result is determined based on the plurality of functional components of the running environment in response to the online analysis operation sent by the second device, comprising: obtaining space science experimental data from the space science experimental data sources in response to the online analysis operation sent by the second device; and determining the target sub-result according to the space science experimental data based on the plurality of functional components of the running environment.
[0011] In a possible implementation manner of the first aspect, the item creation request further carries resource configuration information, and the method further includes: determining the computing resource corresponding to the running environment according to the resource configuration information; determining the computing resource occupation rate corresponding to the second device when the target sub-result is determined in response to the operation of the plurality of functional components configured by the running environment by the second device; applying an expanded computing node to a resource pool when the computing resource occupation rate is greater than or equal to a first threshold, the resource pool including a plurality of computing nodes, and the type of the computing node matching the hardware architecture of the running environment; and releasing an idle computing node to the resource pool when the computing resource occupation rate is less than a second threshold and for a preset time length.
[0012] In a possible implementation manner of the first aspect, the method further includes: in response to an instruction of a custom running environment input by the second device, the instruction carrying custom functional component information, incrementally installing the custom functional component on the running environment to generate the custom running environment; and in response to an online analysis operation sent by the second device, determining the target sub-result based on the plurality of functional components configured by the custom running environment.
[0013] In a possible implementation manner of the first aspect, the result update request further carries a version number of the target sub-result, and the target result is updated according to the identification information of the target sub-result, including: in a case where no data is stored in a storage location corresponding to the identification information of the target sub-result, storing the target sub-result and the version number of the target sub-result in the storage location corresponding to the identification information of the target sub-result; in a case where data is stored in the storage location corresponding to the identification information of the target sub-result, determining a version number of the stored data; and in a case where the version number of the stored data is less than the version number of the target sub-result, storing the target sub-result and the version number of the target sub-result in the storage location corresponding to the identification information of the target sub-result.
[0014] In a possible implementation manner of the first aspect, the method further includes: in a case where at least two second devices send result update requests for the same sub-result, sending update conflict information to the first device, the update conflict information carrying identification information of each of the at least two second devices and the sub-result; in response to a conflict resolution instruction sent by the first device, the conflict resolution instruction carrying identification information of a third device, the third device being one of the at least two second devices; and updating the target result according to the sub-result corresponding to the third device.
[0015] The method provided by the application can realize online analysis and collaborative processing of large-scale, multi-field and multi-type space scientific experimental data, thereby supporting research requirements of complex space scientific experimental tasks. On the one hand, the efficiency of space scientific experimental data processing is improved, and the scientific research collaboration capability is strengthened. Through the online collaborative analysis method, the scientific researchers can quickly and accurately decompose complex research tasks, and multiple teams can collaborate online, thereby greatly shortening the data processing period and improving the scientific research efficiency. The application supports multiple users to collaboratively participate in the same scientific research project, realizes fine-grained permission management and project version control, and enables the scientific research team to efficiently solve problems and share achievements. On the other hand, the application can optimize the allocation of computing resources and reduce the cost of scientific research. The online collaborative analysis system automatically schedules computing resources according to user requirements, constructs an online research environment, effectively solves the problem of limited computing resources in the traditional data processing mode, and improves the utilization rate and reusability of the computing resources. The application avoids the cumbersome process of locally building an analysis environment in the traditional scientific research process, and enables the scientific researchers to focus on the investment in data processing and analysis.
[0016] That is, the technical scheme provided by the application can realize online analysis and collaborative processing of large-scale, multi-field and multi-type space scientific experimental data, thereby supporting research requirements of complex space scientific experimental tasks. On the one hand, the efficiency of space scientific experimental data processing is improved, and the scientific research collaboration capability is strengthened. Through the online collaborative analysis method, the scientific researchers can quickly and accurately decompose complex research tasks, and multiple teams can collaborate online, thereby greatly shortening the data processing period and improving the scientific research efficiency. The application supports multiple users to collaboratively participate in the same scientific research project, realizes fine-grained permission management and project version control, and enables the scientific research team to efficiently solve problems and share achievements. On the other hand, the application can optimize the allocation of computing resources and reduce the cost of scientific research. The online collaborative analysis system automatically schedules computing resources according to user requirements, constructs an online research environment, effectively solves the problem of limited computing resources in the traditional data processing mode, and improves the utilization rate and reusability of the computing resources. The application avoids the cumbersome process of locally building an analysis environment in the traditional scientific research process, and enables the scientific researchers to focus on the investment in data processing and analysis.
[0017] In a second aspect, the present application provides an online collaborative analysis system, the online collaborative analysis system being in communication with a first device and a plurality of second devices respectively, the online collaborative analysis system comprising: a project creation module, configured to create a spatial scientific data online research project in response to a project creation request sent by the first device, the spatial scientific data online research project being configured to determine a target result, the target result of the spatial scientific data online research project comprising a plurality of sub results, each of the plurality of sub results being configured with different identification information; the project creation request carrying running environment information; a mirror creation module, configured to create a running environment in response to an analysis request of the spatial scientific data online research project sent by the second device, the running environment being configured with a plurality of functional components by scheduling an environment mirror and a computing resource according to the running environment information; a result iteration module, configured to determine a target sub result in response to an online analysis operation sent by the second device, the target sub result being one of the plurality of sub results based on the plurality of functional components of the running environment; and a result update module, configured to update the target result in response to a result update request sent by the second device, the result update request carrying the identification information of the target sub result.
[0018] In a third aspect, an electronic device is provided, the electronic device comprising a memory and one or more processors; the memory being coupled to the processors; and wherein the memory stores computer program codes comprising computer instructions that, when executed by the processors, cause the electronic device to perform the method according to any implementation manner of the first aspect.
[0019] In a fourth aspect, a computer readable storage medium is provided, comprising computer instructions that, when executed on an electronic device, cause the electronic device to perform the method according to any implementation manner of the first aspect.
[0020] In a fifth aspect, a computer program product is provided, which, when executed on a computer, causes the computer to perform the method according to any implementation manner of the first aspect.
[0021] It can be understood that the beneficial effects achieved by the spatial scientific experiment data online collaborative analysis system according to the second aspect, the electronic device according to the third aspect, the computer readable storage medium according to the fourth aspect, and the computer program product according to the fifth aspect can refer to the beneficial effects of the first aspect and any possible design manner thereof, which will not be described herein again. BRIEF DESCRIPTION OF DRAWINGS
[0022] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present application;
[0023] Figure 2 A structural schematic diagram of an electronic device provided by an embodiment of the present application;
[0024] Figure 3 A flow chart of a space science experiment data online collaborative analysis method provided by the embodiment of the present application is shown in the figure;
[0025] Figure 4a An interface schematic diagram of Notebook interactive programming shown by the embodiment of the present application is shown in the figure;
[0026] Figure 4b An interface schematic diagram of graphical drag-and-drop programming shown by the embodiment of the present application is shown in the figure;
[0027] Figure 4c An interface schematic diagram of Web IDE programming shown by the embodiment of the present application is shown in the figure;
[0028] Figure 5 A task decomposition schematic diagram of a space science experiment data online collaborative analysis method shown by the embodiment of the present application is shown in the figure;
[0029] Figure 6 An interface schematic diagram of selecting project running resources shown by the embodiment of the present application is shown in the figure;
[0030] Figure 7a An interface schematic diagram of a space science experiment data task planning shown by the embodiment of the present application is shown in the figure;
[0031] Figure 7b An interface schematic diagram of a space science experiment data analysis project Kanban shown by the embodiment of the present application is shown in the figure;
[0032] Figure 7c An interface schematic diagram of a space science experiment data analysis task details shown by the embodiment of the present application is shown in the figure;
[0033] Figure 8 A processing interface schematic diagram of collaborative analysis code merging shown by the embodiment of the present application is shown in the figure;
[0034] Figure 9 A work flow chart of a multi-person collaborative analysis scene shown by the embodiment of the present application is shown in the figure;
[0035] Figure 10 A structure schematic diagram of an online collaborative analysis system provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION
[0036] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the present application, unless otherwise specified, " / " represents an "or" relationship between the objects before and after it, for example, A / B can represent A or B; in the present application, "or" is only a description of the relationship between the associated objects, which means that there can be three relationships, for example, A or B, which can mean that A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. In the description of the present application, unless otherwise specified, "multiple" means two or more than two. "At least one of the following" or the like means any combination of the items, including any combination of single or multiple items.
[0037] In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, "first", "second" and the like are used to distinguish the same items or similar items with basically the same function and effect. Those skilled in the art can understand that "first", "second" and the like do not limit the quantity and execution order, and "first", "second" and the like do not necessarily mean different.
[0038] Meanwhile, in the embodiments of the present application, "exemplary" or "for example" and the like are used to represent as an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as more excellent or more advantageous than other embodiments or design schemes. Rather, "exemplary" or "for example" and the like are used to present the relevant concept in a specific manner, for the purpose of understanding.
[0039] The space laboratory includes scientific experiment cabinets, extravehicular exposure experiment platforms and other scientific research facilities, which can support on-orbit rolling implementation of space life science and biotechnology, microgravity fundamental physics, space material science, space astronomy and many other research fields. Space science experiment data has the characteristics of large volume, real-time batch generation, multi-field, multi-type, multi-form and wide source, and has the characteristics of wide, open, cooperation and sharing.
[0040] In scientific research, data processing and analysis is a crucial link. With the rapid development of data science, the processing and analysis of scientific data has become more and more complex. Traditional scientific data processing methods usually require users to build a computing environment locally, which not only consumes time and effort, but also is limited by local computing resources. In addition, complex scientific research projects often require multiple researchers to work collaboratively, and traditional collaboration methods are inefficient and difficult to meet the needs of modern scientific research. The existing online data processing platform has not effectively solved the problems of how to efficiently allocate computing resources, how to manage project versions, and how to support complex collaboration task planning when multiple users collaborate.
[0041] Therefore, there is an urgent need for a space science experiment data online collaborative analysis method and system, which can conveniently complete online processing and analysis of complex scientific tasks, efficiently utilize space science experiment data, and promote scientific and application achievements.
[0042] In view of this, the embodiments of the present application provide a space science experiment data online collaborative analysis method, applied to an online collaborative analysis system, the online collaborative analysis system is in communication with a first device and a plurality of second devices, the method comprising: in response to a project creation request sent by the first device, creating a space science data online research project, the space science data online research project being used to determine a target result, the target result of the space science data online research project comprising a plurality of sub-results, each sub-result being respectively configured with different identification information; the project creation request carrying running environment information; in response to an analysis request of the space science data online research project sent by the second device, scheduling an environment image and computing resources according to the running environment information, creating a running environment, the running environment being configured with a plurality of functional components; in response to an online analysis operation sent by the second device, determining a target sub-result based on the plurality of functional components configured by the running environment, the target sub-result being one of the plurality of sub-results; in response to a result update request sent by the second device, the result update request carrying identification information of the target sub-result, updating the target result according to the identification information of the target sub-result.
[0043] The method provided by the present application can quickly and accurately complete online processing and analysis of complex scientific tasks in a multi-person collaborative processing scenario, efficiently utilize space science experiment data, and effectively improve data processing and analysis efficiency.
[0044] That is, the technical scheme provided by the present application can realize online analysis and collaborative processing of large-scale, multi-field, multi-type space scientific experiment data, thereby supporting the research needs of complex space scientific experiment tasks. On the one hand, the efficiency of space scientific experiment data processing can be improved, and the scientific research collaboration capability is strengthened. Through the online collaborative analysis method, the scientific researchers can quickly and accurately decompose complex research tasks, and multiple teams can work online, which greatly shortens the data processing period and improves the scientific research efficiency. The present application supports multiple users to participate in the same scientific research project, realizes fine-grained permission management and project version control, and enables the scientific research team to efficiently solve problems and share achievements. On the other hand, the present application can optimize the allocation of computing resources and reduce the cost of scientific research. The online collaborative analysis system automatically schedules computing resources according to user needs, constructs an online research environment, effectively solves the problem of limited computing resources in the traditional data processing mode, and improves the utilization rate and reusability of computing resources. The present application avoids the cumbersome process of building an analysis environment locally in the traditional scientific research process, so that the scientific researchers can focus on data processing and analysis.
[0045] Referring to Figure 1 , Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present application is shown. The application scenario includes an online collaborative analysis system 100, a first device 11 and a plurality of second devices 12. The online collaborative analysis system 100 communicates with the first device 11 and the plurality of second devices 12 respectively. The user operating the first device 11 is a manager of a space scientific data online research project, and the users operating the second devices 12 are collaborators of the space scientific data online research project.
[0046] In some embodiments, the space scientific experiment data online collaborative analysis method provided by the present application can be executed by a space scientific experiment data online collaborative analysis system 100 (hereinafter referred to as the online collaborative analysis system 100).
[0047] As an example, the online collaborative analysis system 100 can be any electronic device 200 with data processing capability, such as a general-purpose computer, a personal computer, a notebook computer, a switch or a tablet computer, etc. The specific implementation of the online collaborative analysis system 100 is not limited here.
[0048] Figure 2 A hardware structure schematic diagram of an electronic device provided by an embodiment of the present application is shown. The electronic device 200 includes a processor 210, a memory 220 and a communication interface 230.
[0049] The processor 210 can include one or more processing cores. The processor 210 connects various parts within the electronic device 200 with various interfaces and lines, performs various functions of the electronic device 200 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 220, and calling data stored in the memory 220. Alternatively, the processor 210 can be implemented in at least one of a hardware form of a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA).
[0050] The memory 220 can include a random access memory (RAM) and can also include a read-only memory (ROM). Alternatively, the memory 220 includes a non-transitory computer-readable storage medium. The memory 220 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 220 can include a program storage area. The program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a project management function, a collaboration management function, and a shared publishing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.
[0051] The communication interface 230 is configured to communicate with other devices, equipment or communication networks, such as data storage devices, image processing equipment or Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.
[0052] In physical implementation, the above-mentioned devices (such as the processor 210, the memory 220 and the communication interface 230) can be devices in the same device (such as a notebook computer), respectively. Alternatively, at least two of them can be arranged in the same device, i.e. as different devices in a device, similar to the deployment of devices or equipment in a distributed system.
[0053] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 200. In other embodiments of the present invention, the electronic device 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
[0054] The following description, in conjunction with the accompanying drawings, illustrates an online collaborative analysis method for space science experimental data provided by an embodiment of the present invention.
[0055] Figure 3 This is a flowchart illustrating an online collaborative analysis method for space science experimental data provided in an embodiment of the present invention. Optionally, this method can be... Figure 1 The online collaborative analysis system 100 shown is executed by, that is, by Figure 2 The illustrated electronic device 200 performs this operation. The method may include the following steps:
[0056] S1. In response to the project creation request sent by the first device, create an online research project for space science data. The online research project for space science data is used to determine the target results.
[0057] Specifically, for complex research projects requiring multiple participants, teams can use project management tools to better conduct data science research. Before the first device sends a project creation request to the online collaborative analysis system, the user of the first device can create a new task plan, dividing the space science data research project into multiple tasks, each including multiple sub-tasks. The user of the first device can then distribute the sub-tasks, and each sub-task can be further specified as an online research project (space science data online research project) to be carried out, enabling the collaborative team to jointly complete complex research projects.
[0058] like Figure 5 As shown, the space science data research project A includes research task A1, research task A2, and research task A3. Research task A2 includes research sub-task A. 21 Research Subtask A 22 and research subtask A 23 Each research subtask corresponds to an online research project (Space Science Data Online Research Project). That is, Research Subtask A... 21 Corresponding online research project A 21 Research subtask A 22 Corresponding online research project A 22 Research subtask A 23 Corresponding online research project A 23 .
[0059] Exemplarily, the space science data online research project in S1 corresponds to a research subtask A. 21 , the research subtask A 22 , and the research subtask A 23 .
[0060] It should be noted that the space science data research project can include multiple tasks, each task can include multiple subtasks, and each subtask can further include a larger number of subtasks. The above division of the level of the space science data research project is only exemplary, and the embodiments of the present application do not particularly limit the specific number of task levels included in the space science data research project.
[0061] Specifically, the space science data online research project (hereinafter referred to as the project) is a user's space science data analysis content management unit. Each project collects code, data, required environment, and result output during analysis. The user can manage and share various types of information required for research and analysis and the output results of research and analysis through this function, which facilitates communication and cooperation.
[0062] The target result of the space science data online research project includes multiple sub-results, each of which is configured with different identification information. In addition, the project creation request carries running environment information.
[0063] Exemplarily, the project creation request further includes basic information of the project, and the basic information of the project includes: a project name for naming the created project, a project description for briefly describing the created project, a tag for the user to add a tag to the project when creating the project, which classifies the project and facilitates the screening of the project after the project is created. A project can only add one tag, and the tag supports a multi-level structure. A data source can be mounted by the user when creating the project.
[0064] In a possible implementation manner, the identification information of each sub-result includes ID information and attribute information of a space science experiment, wherein the attribute information includes an experiment ID, a sub-result type, a device number, and a timestamp.
[0065] In the case of obtaining sub-result A from data A, the sub-result type corresponds to different standard process stages of space science experiment data analysis, such as preprocessing, data correction, algorithm model training, and result verification. The device number is associated with the on-orbit experiment cabinet that generates data A, and the timestamp is used to represent the time when data A is generated.
[0066] In this way, the method provided by the application can solve the problem of multi-device parallel collection and multi-stage data cross-validation tracing in the process of space science experiments, for example, when the sub-result A is abnormal, the source of the data A can be quickly located through the identification information, the abnormal data can be quickly and accurately identified, and the processing efficiency is improved.
[0067] In some embodiments, the user can upload a local existing ".ipynb" format file for subsequent code work.
[0068] After the project is created, the user of the first device can send a request to the online collaborative analysis system to view the personally created project in the workbench, and after entering the project detail page, the user can view the project content, run the project, modify the project attribute, share the project, or select to delete the project.
[0069] In one example, the project detail page includes project content and project data.
[0070] The project content carries the user's modeling process, and the specific coding content can be viewed. Figure 4a Figure 4a The Notebook interactive programming interface shown in FIG. 1 includes a plurality of "programming units" (cells), and the cell is the smallest unit of content written in the Notebook, each cell can be run independently, and the input and output are one-to-one. Figure 4b Figure 4b The graphical drag-and-drop programming shown in FIG. 2 can directly use encapsulated commonly used machine learning algorithm components for zero-code modeling. Figure 4c Figure 4c The Web IDE shown in FIG. 3 provides a code-level programming tool in the form of an integrated development environment based on a web page, which supports online development of programming languages such as Python, Java, C, C++, Fortran, and R.
[0071] The project data displays a list of data mounted in the project. Clicking on an entry displays an overview of the data and related files. The preview function supports.txt,.csv,.jpg,.png,.mp4,.mkv,.json, and other types of files, helping users to have a preliminary understanding of the data before analysis.
[0072] It should be understood that the implementation of the project details page described above is only an example, and the embodiments of the present application do not particularly limit the specific implementation of the project.
[0073] S2, in response to the analysis request of the spatial science data online research project sent by the second device, scheduling the environment image and the computing resource according to the running environment information, and creating a running environment.
[0074] The running environment is configured with multiple functional components, and the running environment is a cloud running environment.
[0075] Specifically, the running environment is configured with multiple functional components, and the running environment is used to provide software environment support for project running. The online collaborative analysis system is configured with multiple data analysis and machine learning toolkits according to the preconfigured running environment. Among them, the running environment has the same operating system and the same version of functional components.
[0076] In one possible implementation, the method provided by the embodiments of the present application further includes:
[0077] In response to the instruction of the custom running environment input by the second device, the instruction carrying the custom functional component information, incrementally installing the custom functional component on the running environment to generate a custom running environment; in response to the online analysis operation sent by the second device, determining a target sub-result based on the multiple functional components configured by the custom running environment.
[0078] It can also be understood that the user of the second device can maintain his own running environment through custom image, and the new image can also be constructed based on the original running environment, so as to avoid repeated installation of some packages. The user can create an image by adding a toolkit, and currently supports installing Python 3, Python 2, R, Julia 1.0, and Julia 1.5.
[0079] In some embodiments, the project creation request also carries resource configuration information.
[0080] The computing resource included in the resource configuration information provides hardware environment support for project running. The user of the second device can view the computing resource available to the user, such as 8-core CPU 32 GB memory, 16-core CPU 64 GB memory of different specifications, whether a GPU card is included, and information such as a computing instance running under the resource, an offline task, and a single use duration.
[0081] In a possible implementation, if the existing computing resource is insufficient to meet the data analysis requirement, the user of the second device can apply for more resource use permissions from the user of the first device through the online collaborative analysis system, select the required computing resource and the corresponding duration, fill in the application reason, and submit an application form. The online collaborative analysis system expands the corresponding computing node for the second device in response to a resource expansion confirmation operation of the first device to meet the data analysis requirement.
[0082] For example, as shown in FIG. 8, after the user selects the environment image and the computing resource, the user sends a request to the online collaborative analysis system. The online collaborative analysis system maintains a resource pool (which can include CPU nodes and GPU nodes) composed of multiple nodes, schedules the nodes meeting the configuration from the resource pool to provide computing services for the user, and automatically runs the running environment selected by the user. Figure 6
[0083] Optionally, the method provided in the embodiment of the present application further includes:
[0084] According to the resource configuration information, the computing resource corresponding to the running environment is determined; when the target sub-result is determined in response to the operation of the second device on the plurality of functional components of the running environment, the computing resource occupation rate corresponding to the second device is determined; in the case where the computing resource occupation rate is greater than or equal to a first threshold value, a computing node is applied for expansion to a resource pool, the resource pool includes a plurality of computing nodes, and the type of the computing node matches the hardware architecture of the running environment; in the case where the computing resource occupation rate is less than a second threshold value and lasts for a preset duration, an idle computing node is released to the resource pool.
[0085] It can also be understood that the method provided by the application supports dynamically switching computing resources and environment images during project operation, so as to meet the project operation requirements in real time. During the project operation, the user can view the current available duration, CPU usage, memory usage, disk usage and other information in the monitoring bar. When the memory occupation or disk usage (storage resource occupation rate) exceeds 80% (the first threshold value), the online collaborative analysis system will trigger a reminder to avoid project operation failure caused by memory overflow and the like. The user can also view the historical use of the project, including the historical use of CPU, memory, disk IO and network IO. The user can accurately anchor the resource change at a specific moment, and more comprehensively understand the running state of the instance, so as to provide a basis for selecting a more suitable computing specification.
[0086] In order to ensure effective allocation and use of resources, the single use duration of the project is limited to X hours, and the resources will be automatically disconnected after X hours. The X is set by the user of the first device. The limitation is to prevent the user from causing consumption of computing resources due to forgetting to close the running interface. When the single duration is left for 10 minutes, the online collaborative analysis system will prompt the user of the second device that the remaining duration is insufficient, and if the user of the second device is performing a long-time work, the single available duration can be manually extended.
[0087] When the user of the second device closes the research project, the online collaborative analysis system automatically releases the computing resources, so that the resources can serve other users and businesses, and the utilization rate and reusability of the computing resources are improved. The online collaborative analysis system supports two running modes of fixed resource pool and dynamic resource pool. When multiple users use and the resource request quantity is greater than the resource quantity in the existing cluster: in the fixed mode, the online collaborative analysis system provides an automatic queuing mechanism to reduce the task waiting time in the case of fixed resource supply, and to improve the resource utilization rate as much as possible, and to efficiently support data analysis and processing work of various users. In the dynamic mode, the online collaborative analysis system sets threshold 1 and threshold 2. If the total resources occupied by the current running project / the total quantity of the resource pool>threshold 1, an automatic expansion mechanism is triggered to add computing nodes to the resource pool. If the total resources occupied by the current running project / the total quantity of the resource pool<threshold 2, an automatic contraction mechanism is triggered to remove the computing nodes from the resource pool.
[0088] In a possible implementation, the project creation request also carries permission information of each of the plurality of second devices, and the S2 includes:
[0089] In response to the analysis request of the online research project of the spatial scientific data sent by the second device, it is determined whether the second device has the online analysis permission of the online research project of the spatial scientific data according to the permission information.
[0090] In a case that the second device has online analysis permission of the online research project of the space science data, a running environment is created according to the running environment information.
[0091] In one example, the first device displays a task planning interface as shown in Figure 7a , enters a specific task, and can see a project board as shown in Figure 7b . In the project board, the user can see all the tasks in planning and their states. A new task can be set with a task name and a deadline, and after the new task is completed, it will be formed in the project board. As shown in Figure 7c , in the task detail page, the user can edit the task details, communicate in the discussion area, share attachments as supplementary materials, and finally upload and summarize the task results.
[0092] The user of the first device can invite members to jointly maintain the task planning by adding collaborators. The user of the first device can add collaborators individually (that is, configure corresponding permissions for different second devices, such as task modification permission and task viewing permission), or add them in batches according to groups, and if the members (users of different second devices) of the selected group change (members leave or new members are added), the scope of the user of the first device will be updated synchronously.
[0093] The following explains and describes the permission information of the first device and the second device of the embodiments of the present application in combination with a specific example. The permissions of the task planning are divided into the following two kinds: Collaborators (users of the second device): can access the content in the task planning, create tasks, edit tasks, and knowledge base. Managers (users of the first device): in addition to having the “collaborator” permission, also have the permission to adjust member permissions and add / edit task delivery.
[0094] If a member (any second device) has exited the project group, the creator (the first device) can directly remove it. If the member (any second device) enters the collaborator list in a group synchronization manner, the creator can remove the user (any second device) from the synchronized group, thereby realizing the removal of the user.
[0095] S3, in response to the online analysis operation sent by the second device, based on the plurality of function components configured by the running environment, determining a target sub-result, the target sub-result being one of the plurality of sub-results.
[0096] In some embodiments, the online collaborative analysis system is also configured with a plurality of space science experiment data sources, including local data sets, databases, object storage, and NAS space; and the above S3 includes: in response to the online analysis operation sent by the second device, obtaining space science experiment data from the plurality of space science experiment data sources; and based on the plurality of function components configured by the running environment, determining a target sub-result according to the space science experiment data.
[0097] The following describes the process of determining the target sub-result by the user through the second device in the embodiment of the application with reference to an example.
[0098] Specifically, the specific research task is carried out through the online research project. The platform provides code-level collaboration functions, and multiple second devices can be added to the project to enable multiple users of multiple second devices to jointly solve the data science problem and obtain the target result.
[0099] Specifically, the second device can create a new project (sub-project) with the same content as the selected version of the Fork project (online research project of space science data) in the online collaborative analysis system, that is, create a copy of the online research project of space science data, and the user can edit and modify the target result or any sub-result in the target result of the online research project of space science data in the copy of the online research project of space science data. All attachment files (results generated) in the parent project (online research project of space science data) are automatically imported, and the user can view the attachment content (for example, view any sub-result included in the target result) under the corresponding directory of the file tree when running the project.
[0100] It should be noted that the project version is the basis for systematized management of the project, sharing of the project, and collaboration of the project. It can facilitate sharing and collaboration: when the project does not generate any version, no one except the creator can browse the project content. By generating a version, the project content can be changed into a state that can be previewed by others, and can be shared and publicly published. Record the achievements at different stages: when performing data analysis work, different stages of content can be saved by generating multiple versions. A same project can generate multiple versions, and the content between different versions can be compared and replaced. The solutions and actual progress of the user at each stage can be retained through the version function. At this time, the organization user who obtains the project viewing permission can see the project content of the corresponding version.
[0101] In some embodiments, the method provided by the embodiment of the application further includes:
[0102] In response to the result reading request of the online research project of space science data sent by the second device, determining whether the second device has the result reading permission of the online research project of space science data according to the permission information.
[0103] In the case where the second device has the result reading permission of the online research project of space science data, the target result corresponding to the online research project of space science data is sent to the second device.
[0104] Based on the online project and the version management of the project, multi-user project collaboration can be carried out.
[0105] Specifically, the first device can share the project to multiple members in the organization by adding multiple second devices, share project content, and co-write code. Each user of the second device has the following three types of permissions:
[0106] Only view: only can browse the project content, cannot copy the content by online Fork (generate content copy), download form. Can Fork: can view and Fork the project. Can collaborate: can view and Fork the project, and can submit a merge request to realize content collaboration.
[0107] Optionally, project collaboration is limited to members within the organization. If it is necessary to share project content with members outside the organization, the project sharing function can be used. Select the version of the project you want to share, and a sharing link can be generated. The sharing link can be set with a password. After the link is shared with others, the external user can view the content of the project.
[0108] S4, in response to the result update request sent by the second device, the result update request carries the identification information of the target sub-result, and the target result is updated according to the identification information of the target sub-result.
[0109] It should be noted that the online collaborative analysis system can also respond to the analysis request of the spatial scientific data online research project sent by the first device, and create a running environment according to the running environment information. The online collaborative analysis system can also respond to the online analysis operation sent by the first device, determine the target sub-result based on the multiple function components configured by the running environment, and the online collaborative analysis system can also respond to the result update request sent by the first device, and update the target result according to the identification information of the target sub-result. That is, the first device can also realize all the functions of the second device. The above distinction between the first device and the second device is only for explanation and description, that is, the user of the first device is the manager of the spatial scientific data online research project, and the user of the second device is the collaborator of the spatial scientific data online research project.
[0110] Specifically, the target result is the stage achievement of the current spatial scientific data online research project, including the processed data file, the trained model file, and other types of files. By publishing the target result, the first device or the second device can be used for viewing and reproduction in the future.
[0111] The target result can be integrated and managed by relying on the knowledge base and the algorithm base.
[0112] The knowledge base receives various contents in a tree structure, helping users to gather and organize personal and organizational resources. The knowledge base realizes the organization and sharing of projects, data and other contents. Each research task is equipped with a knowledge base, supporting the uploading of various formats of project materials, including projects, data, files, videos, external links, and assisting team research. The administrator can sort, add, and delete the contents, and classify them by using folders. Members can view and download the knowledge base contents, run project codes, and create projects using data.
[0113] Using the algorithm library, users can manage the algorithm models produced in research work, realize the organization, sharing and reuse of these algorithm models. Users can edit, delete, authorize, etc. The model file supports parsing and visualizing the structure, and the supported frameworks include ONNX (.onnx,.pb,.pbtxt), Keras (.h5,.keras), CoreML (.mlmodel), MXNet (.model), Caffe (.caffemodel,.prototxt), PyTorch (.pth), Torch (.t7), CNTK (.model,.cntk), Darknet (.cfg), scikit-learn (.pkl), TensorFlow Lite (.tflite), TensorFlow.js (.pb) and TensorFlow (.pb,.meta,.pbtxt).
[0114] In a possible implementation, the result update request further carries a version number of the target sub-result, and the target result is updated according to the identification information of the target sub-result, including:
[0115] In a case where the storage location corresponding to the identification information of the target sub-result does not store data, the storage location corresponding to the identification information of the target sub-result stores the target sub-result and the version number of the target sub-result; in a case where the storage location corresponding to the identification information of the target sub-result stores data, a version number of the stored data is determined; and in a case where the version number of the stored data is less than the version number of the target sub-result, the storage location corresponding to the identification information of the target sub-result stores the target sub-result and the version number of the target sub-result.
[0116] In some embodiments, the method provided by the embodiments of the present application further includes:
[0117] In a case where at least two second devices send result update requests for the same sub-result, update conflict information is sent to the first device, the update conflict information carrying identification information of each of the at least two second devices and the sub-result; in response to a conflict resolution instruction sent by the first device, the conflict resolution instruction carrying identification information of a third device, the third device being one of the at least two second devices; and the target result is updated according to the sub-result corresponding to the third device.
[0118] In some embodiments, the method further comprises:
[0119] In a case where at least two second devices send result update requests for different sub- results, update request information is sent to the first device, the update request information carrying identification information of each of the at least two second devices and the different sub- results; and in response to a result update instruction sent by the first device, the target result is updated according to the sub-result corresponding to each of the two second devices.
[0120] Specifically, a user can run a sub-project through a second device to perform code editing, submission merging, etc. The user performs a merge request through a first device, as shown in Figure 8
[0121] Specifically, a submission merging request: after the sub-project is updated, the update can be sent to the parent project through "submission merging". The merging request can be actively revoked. The collaborators can see the processing result of the request. After the merging request is passed or rejected, the collaborators will receive a notification prompt. Processing a merging request: the creator of the parent project can process the merging request and compare the modified content. The differences between the sub-project and the parent project can be visually displayed (the modified part is highlighted), and the modified content can be viewed piece by piece. The modified content can be accepted or rejected piece by piece or in batches. After completion, the new version of the sub-project will be merged into the parent project, and thereafter all the collaborators of the project can obtain the relevant modifications by checking the update. Checking the update: when the parent project generates a new version, the platform will trigger a "source project update reminder" to all the collaborators, so that the collaborators can pull the latest content. After acceptance, the current workspace content of the sub-project will be replaced with the latest version content of the parent project, and a new version record is generated for this modification. In the version list tab, the user can compare each version of the project and intuitively view the differences between each version. Through version comparison, the user can view the differences between each version compared to the current workspace. For each modification, the user can replace the content edited in the current runtime with the content in the selected version, thereby realizing code-level backtracking. The user can also completely replace the content edited in the current runtime with the content of a certain historical version.
[0122] The application scenario of the method provided by the embodiment of the application is described below in combination with a specific example. In an example, the working operation process of a project leader A (a first device) and a member B (a second device) in a two-person small team collaborative development scenario is as shown in FIG. 1. Figure 9
[0123] S41, the first device creates a project, performs online debugging, generates a version, and records the association of the project.
[0124] S42, in response to an operation input by the project leader A, the first device shares the project with other members (second devices) in the organization, shares the project content, and jointly writes the code. When sharing, the image and data used by the project can be shared together. After the second device obtains the project permission, the collaborative project (P) can be seen in the workspace of the second device, a project copy (sub-project) (P') is generated through Fork, and code running in the same analysis environment is supported.
[0125] Optionally, the first device can share the project through a project link. The project website link is copied and sent to the collaborators. At the same time, batch addition of collaborators in a group synchronization manner is supported, and batch distribution of content is realized. If the group members change (members leave or are added), the collaborator range is updated synchronously. If a member has exited the project group, the creator can directly remove the member.
[0126] Optionally, the second device completely reproduces the project, needs to use the data, computing power, and image dependent on the project running, and if the relevant permission has not been obtained, the Fork can be synchronized to apply.
[0127] S43, the second device runs the project and enters the programming interface to modify the content (P'). After the code writing is completed, a version of the current content is generated. At the same time, the first device (which can also be another second device) can continue to edit the project (P), and the mutual work is not affected.
[0128] S44, after the second device generates the version, the second device can submit the merge, that is, apply to merge the latest version of the project P' into the source project P. The first device processes the merge request (accepts the modification or rejects the modification) in the project P, batch acceptance / rejection is supported. The platform highlights the content difference, helps the first device to compare and integrate the updated content.
[0129] Optionally, the second device can pull the latest content of the project P through the check and update (S45) before submitting the merge, and submit the content after the latest adjustment.
[0130] S45, after the first device completes the content review, the platform will automatically generate a version for the operation, and send a project update reminder to the second device. Before the second device modifies the content (P') again, pull the latest content to avoid subsequent merging conflicts.
[0131] In a possible implementation, the method provided by the embodiment of the present application further includes: generating a traceability chain corresponding to each identification information based on the identification information of each sub-result after the update.
[0132] For example, after the second device updates the space radiation dose sub-result SRD-5, the system automatically generates a traceability chain based on the device number and calibration parameters associated with the identification information. The traceability chain is:
original data (sensor 02, 2024-05-10 12:00) → calibration processing (Toolkit 6.0) → sub-result SRD-005 (version V2) → target result (space radiation dose measurement module data update)
[0133] The method provided by the embodiment of the present application can perform one-key backtracking through the traceability chain. In the case of subsequent discovery of abnormal results, the traceability chain can be used to quickly locate whether the abnormality is caused by abnormal original data or calibration parameter error, thereby effectively improving the processing efficiency and realizing fast and accurate backtracking of abnormal data.
[0134] As can be seen from the above S1-S4, the method provided by the embodiment of the present application creates a space science data online research project for determining a target result, responds to analysis requests for the space science data online research project sent by a plurality of second devices, creates a running environment according to running environment information, and the running environment is configured with a plurality of functional components; in response to online analysis operations sent by the plurality of second devices, different sub-results are respectively determined based on the plurality of functional components configured by the running environment; and in response to a result update request sent by each second device, the target result is updated according to the identification information of the sub-result of each second device. The method provided by the present application can quickly and accurately complete online processing and analysis of complex scientific tasks in a multi-person collaborative processing scenario, efficiently utilize space science experimental data, and effectively improve the data processing and analysis efficiency.
[0135] That is, the technical solution provided by the present application can realize online analysis and collaborative processing of large-scale, multi-field, multi-type space scientific experiment data, thereby supporting the research needs of complex space scientific experiment tasks. On the one hand, the efficiency of space scientific experiment data processing can be improved, and the scientific research collaboration capability is strengthened. Through the online collaborative analysis method, the scientific researchers can quickly and accurately decompose the complex research tasks, and multiple teams can collaborate online, which greatly shortens the data processing period and improves the scientific research efficiency. The present application supports multiple users to participate in the same scientific research project, realizes fine-grained permission management and project version control, and enables the scientific research team to efficiently solve problems and share achievements. On the other hand, the present application can optimize the allocation of computing resources and reduce the cost of scientific research. The online collaborative analysis system automatically schedules computing resources according to user needs, constructs an online research environment, effectively solves the problem of limited computing resources in the traditional data processing mode, and improves the utilization rate and reusability of computing resources. The present application avoids the cumbersome process of building an analysis environment locally in the traditional scientific research process, so that the scientific researchers can focus on data processing and analysis.
[0136] In order to facilitate understanding of the present application, the method provided by the present application is explained and described below in combination with an example. The purpose of the method provided by the present application is to enable scientific research teams to more conveniently process and analyze complex space scientific data (such as space life science data and space material science data). The entire scheme is like an "online collaborative laboratory" specially designed for scientists, which provides a complete tool chain from project creation to final result sharing. First, project creation is performed. A user (manager) operating a first device sends a request to the online collaborative analysis system to create a dedicated online research project in the online collaborative analysis system, which is like establishing a virtual laboratory. This "laboratory" not only contains the conventional project name and description, but also allows the user to manage with a tag classification (such as marked as "space life science experiment data analysis"). The online collaborative analysis system provides three programming methods: interactive programming similar to Jupyter Notebook (suitable for data exploration), graphical drag-and-drop programming (suitable for visual process design), and web-based code editor (suitable for professional programmers). The project can be directly connected to various data sources, including local experimental data, scientific research databases, and even cloud storage.
[0137] Then, the online collaborative analysis system can manage the resources of the project, the user (collaborator) operating the second device can select a standardized "software package" (image) pre-installed with common tools (such as Python / R) or customize the environment, which can add toolkits through a simple interface or finely configure through a command line script, or even install software directly in the code and save it as a new image for later use. In terms of hardware resources, the online collaborative analysis system automatically allocates CPU / GPU computing nodes, and releases resources immediately after use. When multiple people use it at the same time, causing resource shortages, the online collaborative analysis system has two coping strategies: automatic queuing and waiting, and dynamically increasing and decreasing the online collaborative analysis system (automatic expansion and contraction of computing nodes). The user (manager) operating the first device can break down large projects into a tree-shaped task structure (task→subtask) and track progress on a shared board. Adding users (collaborators) operating the second device supports batch adding of entire research groups, and can finely control the permissions of each user (collaborator) operating the second device (such as read-only or editable). A version control function similar to GitHub is specially designed: users (collaborators) operating the second device can "fork" project copies for independent modification, synchronize modifications back to the main project through "merge requests", and the online collaborative analysis system will automatically mark code differences and help resolve conflicts. Any version can be traced back and compared at any time, even to the replacement of a single line of code. Finally, the target results (processed data, trained AI models, analysis reports, etc.) obtained by multiple users (collaborators) operating the second device can be stored in the "knowledge repository" in the online collaborative analysis system. Algorithmic models are managed separately as reusable "toolboxes" that can be directly invoked by other teams later. The online collaborative analysis system also has a special multi-format database that can store not only regular documents but also video materials and external reference links, forming a complete project knowledge graph.
[0138] The above mainly introduces the scheme of the embodiments of the application from the method aspect. It can be understood that the online collaborative analysis system 100 comprises at least one of the hardware structure and the software module for executing the respective functions in order to realize the above functions. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of the examples described in the embodiments disclosed herein, the embodiments of the application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the application.
[0139] The embodiment of the present application can divide the online collaborative analysis system 100 into functional units according to the above method examples. For example, the online collaborative analysis system 100 can be divided into functional units corresponding to each function, or two or more functions can be integrated into one processing unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit. It should be noted that the division of units in the embodiment of the present application is illustrative, and is only a logical functional division. In actual implementation, there can be another division method.
[0140] Exemplary, Figure 10 A hardware structure schematic diagram of an online collaborative analysis system provided by the embodiment of the present application is shown. The online collaborative analysis system 100 includes: a project creation module 110, configured to create a space science data online research project in response to a project creation request sent by a first device, the space science data online research project being used to determine a target result, the target result of the space science data online research project including a plurality of sub results, each of the sub results being respectively configured with different identification information; the project creation request carrying running environment information; a mirror creation module 120, configured to create a running environment in response to an analysis request of the space science data online research project sent by a second device, the running environment being configured with a plurality of functional components by scheduling an environment mirror and a computing resource according to the running environment information; a result iteration module 130, configured to determine a target sub result in response to an online analysis operation sent by the second device, the target sub result being one of the plurality of sub results based on the plurality of functional components configured by the running environment; and a result update module 140, configured to update the target result in response to a result update request sent by the second device, the result update request carrying identification information of the target sub result.
[0141] In some embodiments, the online collaborative analysis system 100 further includes a task management module 150, configured to perform a management operation on a space science data research task in response to a task management instruction sent by the first device, wherein the management operation includes adding or deleting a space science data research task or a sub task, and the management operation further includes creating, modifying or deleting an online research project included in the space science data research task or the sub task.
[0142] In other embodiments, the online collaborative analysis system 100 further includes an achievement management module 160, configured to perform an editing operation on a final target result generated by an online research project in response to an achievement management instruction sent by the first device, and to send the target result to any second device, and further configured to run a code or a program corresponding to the target result to realize a function corresponding to the target result.
[0143] It should be understood that the specific description of the above-mentioned optional modes can refer to the foregoing method embodiments, and will not be described here. In addition, the description of the explanation and benefits of any of the above-provided online collaborative analysis systems 100 can refer to the corresponding method embodiments described above, and will not be described here.
[0144] The embodiments of the present application also provide a computer readable storage medium, which stores at least one computer instruction. The at least one computer instruction is loaded and executed by a processor to implement the method of each of the above embodiments. The description of the explanation and benefits of any of the above-provided computer readable storage medium can refer to the corresponding embodiments described above, and will not be described here.
[0145] The embodiments of the present application also provide a chip. The chip integrates a control circuit and one or more ports for implementing the functions of the above online collaborative analysis system 100. Optionally, the functions supported by the chip can refer to the above, and will not be described here.
[0146] A person of ordinary skill in the art can understand that all or part of the steps of the above embodiments can be completed by a program instructing related hardware. The program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a random access memory, etc. The processing unit or the processor can be a central processing unit, a general-purpose processor, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA) or other programmable logic device, a transistor logic device, a hardware component or any combination thereof.
[0147] The embodiments of the present application also provide a computer program product containing instructions, which, when executed on a computer, cause the computer to perform any of the methods described in the embodiments above. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application are wholly or partially generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website site, computer, online collaborative analysis system or data center to another website site, computer, online collaborative analysis system or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as an online collaborative analysis system, a data center, etc. integrated with one or more available media sets. The available media can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as an SSD), etc.
[0148] It should be noted that the above devices for storing computer instructions or computer programs provided by the embodiments of the present application, such as but not limited to the above memory, computer readable storage medium and communication chip, etc., are non-volatile. Those skilled in the art should realize that the functions described in the embodiments of the present application can be realized by hardware, software, firmware or any combination thereof in one or more of the above examples. When implemented by software, these functions can be stored in a computer readable storage medium or transmitted as one or more instructions or codes on a computer readable storage medium. The computer readable storage medium includes computer storage medium and communication medium, wherein the communication medium includes any medium that facilitates the transfer of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general purpose or special purpose computer.
[0149] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.
Claims
1. A method for online collaborative analysis of space science experimental data, characterized in that, The method, applied to an online collaborative analysis system that communicates with a first device and multiple second devices, includes: In response to a project creation request sent by the first device, an online research project for space science data is created. The online research project for space science data is used to determine the target result. The target result of the online research project for space science data includes multiple sub-results, each of which is configured with different identification information. The project creation request carries operating environment information. In response to the analysis request of the online research project of space science data sent by the second device, an operating environment is created by scheduling the environment image and computing resources according to the operating environment information, and the operating environment is configured with multiple functional components; In response to the online analysis operation sent by the second device, a target sub-result is determined based on multiple functional components configured in the operating environment, wherein the target sub-result is one of the multiple sub-results; In response to a result update request sent by the second device, the result update request carrying the identification information of the target sub-result, the target result is updated according to the identification information of the target sub-result.
2. The method according to claim 1, characterized in that, The project creation request also carries permission information for each of the plurality of second devices; The step of responding to the analysis request of the online space science data research project sent by the second device, and creating an operating environment based on the operating environment information, includes: In response to the analysis request of the online research project of space science data sent by the second device, it is determined whether the second device has online analysis permission for the online research project of space science data based on the permission information; If the second device has online analysis permissions for the online research project of the space science data, an operating environment is created based on the operating environment information.
3. The method according to claim 2, characterized in that, The method further includes: In response to the result reading request of the online space science data research project sent by the second device, determine whether the second device has the permission to read the result of the online space science data research project according to the permission information; If the second device has permission to read the results of the online space science data research project, the target results corresponding to the online space science data research project are sent to the second device.
4. The method according to claim 3, characterized in that, The online collaborative analysis system is also equipped with a variety of space science experiment data sources, including local datasets, databases, object storage, and NAS space. In response to the online analysis operation sent by the second device, based on multiple functional components configured in the operating environment, the target sub-result is determined, including: In response to the online analysis operation sent by the second device, space science experiment data is obtained from the space science experiment data source; Based on the multiple functional components configured in the operating environment, the target sub-results are determined according to the space science experiment data.
5. The method according to claim 4, characterized in that, The project creation request also carries resource configuration information, and the method further includes: The computing resources corresponding to the operating environment are determined based on the resource configuration information. When the target sub-result is determined in response to the operation of multiple functional components configured by the second device on the operating environment, the computing resource utilization rate corresponding to the second device is determined; If the computing resource utilization rate is greater than or equal to a first threshold, an application is made to the resource pool to expand the computing nodes. The resource pool includes multiple computing nodes, and the type of the computing nodes matches the hardware architecture of the operating environment. If the computing resource utilization rate is less than the second threshold and continues for a preset duration, the idle computing nodes are released to the resource pool.
6. The method according to claim 5, characterized in that, The method further includes: In response to a custom runtime environment instruction input by a second device, the instruction carrying custom functional component information, the custom functional components are incrementally installed on the runtime environment to generate a custom runtime environment; In response to the online analysis operation sent by the second device, the target sub-result is determined based on multiple functional components configured in the custom operating environment.
7. The method according to claim 6, characterized in that, The result update request also carries the version number of the target sub-result, and updating the target result according to the identification information of the target sub-result includes: If no data is stored in the storage location corresponding to the identifier information of the target sub-result, the target sub-result and its version number are stored in the storage location corresponding to the identifier information of the target sub-result. If data has already been stored in the storage location corresponding to the identification information of the target sub-result, determine the version number of the stored data; If the version number of the stored data is less than the version number of the target sub-result, the target sub-result and its version number are stored in the storage location corresponding to the identifier information of the target sub-result.
8. The method according to claim 7, characterized in that, The method further includes: If at least two second devices are received to update the result of the same sub-result, update conflict information is sent to the first device, wherein the update conflict information carries the identification information of each of the at least two second devices and the sub-result; In response to a conflict resolution instruction sent by the first device, the conflict resolution instruction carries identification information of a third device, which is one of the at least two second devices; The target result is updated based on the sub-result corresponding to the third device.
9. An online collaborative analysis system for space science experimental data, characterized in that, The system communicates with a first device and multiple second devices respectively, and the system includes: The project creation module is used to create an online research project for space science data in response to a project creation request sent by the first device. The online research project for space science data is used to determine the target result. The target result of the online research project for space science data includes multiple sub-results, each of which is configured with different identification information. The project creation request carries operating environment information. The image creation module is used to respond to the analysis request of the online research project of space science data sent by the second device, schedule the environment image and computing resources according to the operating environment information, and create an operating environment, wherein the operating environment is configured with multiple functional components; The result iteration module is used to respond to the online analysis operation sent by the second device, and determine the target sub-result based on multiple functional components configured in the operating environment, wherein the target sub-result is one of the multiple sub-results; The result update module is used to respond to a result update request sent by the second device, wherein the result update request carries the identification information of the target sub-result, and update the target result according to the identification information of the target sub-result.
10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the online collaborative analysis method for space science experiment data as described in any one of claims 1-8.
Citation Information
Patent Citations
Online programming system, online programming method and related products
CN113094613A
Multi-source computing power data integration and intelligent scheduling system and method
CN118916147A
Visual space science experiment data analysis pipeline method
CN119398477A
Data-driven learning model combination optimization method based on edge cloud collaboration
CN120128577A
Integrated collaborative scientific research environment
US20140058782A1
