An experimental data processing method, device and system

Abstract

This paper relates to the field of automated data processing and provides an experimental data processing method, apparatus, and system. The method includes: sending a task preparation sequence instruction to a data computing system, causing the data computing system to create task-related computing processes according to the task preparation sequence instruction, create data objects within the computing processes, and assign initial values; sending a data acquisition instruction to a device control terminal, receiving data acquired by the device control terminal through operation of the experimental equipment; generating a data modification instruction based on the acquired data and sending it to the data computing system; sending a data calculation instruction to the data computing system, causing the data computing system to perform calculations on the data of the relevant task computing processes according to the data calculation instruction; and sending a data output instruction to the data computing system, causing the data of the relevant computing processes to be stored in a data object database according to the data output instruction. This paper can realize the entire business process and has the advantages of ease of use, ease of learning, and versatility.

Description

Technical Field

[0001] This article relates to the field of data processing, and in particular to an experimental data processing method, apparatus and system. Background Technology

[0002] Laboratory automation and intelligentization are important development directions for future life science research. Many domestic and foreign companies and research institutions have begun to develop integrated laboratory automation systems to achieve automated experiments by integrating robots and various experimental equipment. Standardizing and universalizing the operating parameters and collected data of various laboratory equipment is essential for achieving full automation and intelligentization of laboratories. However, currently, various experimental devices in laboratories lack unified data acquisition and on-site processing methods. Current laboratory integration solutions on the market heavily rely on copying and moving various data files and customized programming scripts, resulting in low integration and poor versatility of current automated experimental systems. This makes it difficult to analyze experimental data on-site and use it for subsequent experiments; furthermore, the complex flow of data and files in the experimental field is difficult to track and monitor, leading to poor security. Summary of the Invention

[0003] This paper addresses the problems of low integration, poor versatility, and complex data flow in existing experimental data processing methods.

[0004] To address the aforementioned problems, one embodiment of this paper provides an experimental data processing method applied to a task management system, comprising:

[0005] Send task preparation sequence instructions to the data computing system so that the data computing system can create a task-specific computing process according to the task preparation sequence instructions, create data objects in the computing process and assign initial values;

[0006] Send data acquisition instructions to the device control terminal, receive operation data from the device control terminal to operate the experimental equipment, and obtain operating data; generate data modification instructions based on the obtained operating data and send them to the data computing system, so that the data computing system can assign values ​​to data objects in the relevant task calculation process;

[0007] Send data calculation instructions to the data calculation system so that the data calculation system can calculate the data in the relevant task calculation process according to the data calculation instructions;

[0008] A data output instruction is sent to the data computing system, so that the data computing system stores the relevant computing process data into the data object database according to the data output instruction.

[0009] In a further embodiment, the task preparation sequence instructions include: an experimental task creation instruction, a data object creation instruction, and a data input instruction;

[0010] The experimental task creation command is used by the data computing system to establish a task-specific computing process;

[0011] The data object creation instruction is used by the data computing system to create data objects and assign initial values ​​during the computing process.

[0012] The data input instruction is used to create and assign data objects to the computing process based on user-stored data.

[0013] In a further embodiment, it also includes:

[0014] The system receives a data query request from a user for a target task within its authorized scope, sends the data query instruction to the data computing system, and then the target task's computing process queries data from the object database and returns it according to the data query instruction.

[0015] The second aspect of this paper provides an experimental data processing method, applied to a data computing system, including:

[0016] Receive task preparation sequence instructions sent by the task management system, start task-related computing processes according to the task preparation sequence instructions, create data objects in the computing processes and assign initial values;

[0017] Receive data modification instructions sent by the task management system, and assign values ​​to data objects in the relevant task calculation process using the first input script according to the data modification instructions;

[0018] Receive data calculation instructions sent by the task management system, push calculation scripts to the calculation process of relevant tasks according to the data calculation instructions, and realize data calculation through the calculation scripts;

[0019] The system receives data output instructions from the task management system, sends query scripts to the computation process of the relevant tasks according to the data output instructions, and stores the data retrieved by the query scripts into the data object database.

[0020] In a further embodiment, the task preparation sequence instructions include: an experimental task creation instruction, a data object creation instruction, and a data input instruction;

[0021] Receiving a task preparation sequence instruction sent by the task management system, starting a task-related computing process according to the task preparation sequence instruction, and creating data objects and assigning initial values ​​in the computing process include:

[0022] Receive the experimental task creation instruction sent by the task management system, and establish a task-specific computing process according to the experimental task creation instruction;

[0023] Receive a data object creation instruction sent by the task management system, and create a data object in the relevant computing process according to the data object creation instruction;

[0024] Receive data input instructions sent by the task management system, and assign initial values ​​to the data objects of the relevant computing processes according to the data input instructions.

[0025] In a further embodiment, after establishing a task-specific computing process according to the experimental task creation instruction, the method further includes:

[0026] Connect the task-specific computing process to the task's web application to enable access to the task-specific computing process's websocket address via the web application.

[0027] In a further embodiment, it also includes:

[0028] The system receives a data query instruction for a target task from the task management system; it then sends the data query instruction to the target task's computing process, so that the target task's computing process can query data from the object database and return it to the task management system.

[0029] In a further embodiment, it also includes:

[0030] The system receives data interaction instructions sent by a data interaction front-end, which displays an interactive interface. The interactive interface displays data objects within the user's permissions and interactive controls for the data objects. Data interaction instructions are issued through the interactive controls of the data objects.

[0031] The data interaction instructions are sent to the computing process of the relevant task so that the computing process can perform the interaction.

[0032] The third aspect of this paper provides an experimental data processing system, including: a task management system, a data computing system, and a data object database;

[0033] The task management system is used to send task preparation sequence instructions to the data computing system; send data acquisition instructions to the device control terminal; receive data acquired by the device control terminal from operating the experimental equipment; generate modification instructions based on the acquired data and send them to the data computing system; send data computing instructions to the data computing system; and send data output instructions to the data computing system.

[0034] The data computing system is used to receive task preparation sequence instructions sent by the task management system, start task-related computing processes according to the task preparation sequence instructions, create data objects in the computing processes and assign initial values; receive modification instructions sent by the task management system, assign values ​​to data objects in the relevant task computing processes according to the modification instructions; receive computing instructions sent by the task management system, push computing scripts to the computing processes of relevant tasks according to the computing instructions, and perform data computing through computing scripts; and receive data output instructions sent by the task management system, store the assigned data or computing results in the data object database according to the data output instructions.

[0035] In a further embodiment, the data object database also stores a data dictionary and a data display method;

[0036] The data dictionary is used to describe the storage and parsing methods for various types of data objects;

[0037] The data display method is used to describe how the data interaction front end displays the data to be displayed.

[0038] The fourth aspect of this paper provides an experimental data processing device for use in a task management system, including:

[0039] The task creation control unit is used to send task preparation sequence instructions to the data computing system, so that the data computing system can create task-related computing processes according to the task preparation sequence instructions, create data objects in the computing processes and assign initial values;

[0040] The data acquisition and control unit is used to send data acquisition commands to the device control terminal, receive data acquired by the device control terminal from operating the experimental equipment, and generate modification commands to the data calculation system based on the acquired data, so that the data calculation system can assign values ​​to data objects in the relevant task calculation process.

[0041] A data calculation control unit is used to send data calculation instructions to the data calculation system, so that the data calculation system can push calculation scripts to the calculation process of relevant tasks according to the data calculation instructions, and realize data calculation through the calculation scripts;

[0042] A data output control unit is used to send data output instructions to the data computing system, so that the data computing system can store the assigned data or calculation results into the data object database according to the data output instructions.

[0043] The fifth aspect of this paper provides an experimental data processing device, applied to a data computing system, including:

[0044] The task creation unit is used to receive task preparation sequence instructions sent by the task management system, start task-related computing processes according to the task preparation sequence instructions, and create data objects in the computing processes.

[0045] The data modification unit is used to receive modification instructions sent by the task management system and assign values ​​to data objects in the computing process according to the modification instructions.

[0046] The data calculation unit is used to receive calculation instructions sent by the task management system, push calculation scripts to the calculation process of relevant tasks according to the calculation instructions, and realize data calculation through the calculation scripts.

[0047] The data output unit is used to receive data output instructions sent by the task management system and store the assigned data or calculation results into the data object database according to the data output instructions.

[0048] A sixth aspect of this document provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in any of the foregoing embodiments.

[0049] A seventh aspect of this document provides a computer storage medium having a computer program stored thereon, which, when executed by a processor of a computer device, implements the method described in any of the foregoing embodiments.

[0050] This invention provides an experimental data processing method, apparatus, and system. By establishing a computational process based on a general-purpose scripting language, it realizes a complete business process from data acquisition, on-site processing, storage, and user interaction. All data storage and computation are based on a standard scripting language with a broad user base, offering advantages in ease of use, learning, and versatility. Furthermore, each experimental task establishes its own computational process, allowing each task to have its own dedicated computational client. The serial execution of each computational process ensures orderliness, and each computational variable is persistently updated to the data object database, guaranteeing data security.

[0051] To make the above and other objects, features and advantages of this document more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments or prior art described herein, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this article. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A structural diagram of the experimental data processing system in the embodiments of this paper is shown;

[0054] Figure 2 This diagram illustrates the storage of data in the data object database of the embodiments described herein;

[0055] Figure 3 The interactive flowchart of the experimental data processing system in the embodiments of this paper is shown;

[0056] Figure 4 Another structural diagram of the experimental data processing system in the embodiments of this paper is shown;

[0057] Figure 5 A flowchart of the experimental data processing method on the task management system side of the embodiments of this paper is shown;

[0058] Figure 6 A flowchart illustrating the experimental data processing method on the data computing system side of the embodiments in this paper is shown;

[0059] Figure 7 This paper shows a structural diagram of the experimental data processing device on the task management system side of an embodiment of the invention;

[0060] Figure 8 This paper shows a structural diagram of the experimental data processing device on the data computing system side of the embodiment.

[0061] Figure 9 A structural diagram of the computer device described in this embodiment is shown;

[0062] Figure 10 A flowchart of the task processing procedure in the embodiments of this article is shown.

[0063] Explanation of symbols in the attached drawings:

[0064] 101. Task Management System;

[0065] 102. Data computing system;

[0066] 103. Data Object Database;

[0067] 104. Data interaction front-end;

[0068] 701. Task Creation Control Unit;

[0069] 702. Data acquisition and control unit;

[0070] 703. Data calculation and control unit;

[0071] 704. Data output control unit;

[0072] 801. Task Creation Unit;

[0073] 802. Data Modification Unit;

[0074] 803. Data Calculation Unit;

[0075] 804. Data Output Unit;

[0076] 902. Computer equipment;

[0077] 904, Processor;

[0078] 906. Memory;

[0079] 908. Drive mechanism;

[0080] 910. Input / Output Module;

[0081] 912. Input devices;

[0082] 914. Output devices;

[0083] 916. Presentation equipment;

[0084] 918. Graphical User Interface;

[0085] 920. Network interface;

[0086] 922. Communication link;

[0087] 924. Communication bus. Detailed Implementation

[0088] The technical solutions in the embodiments described below will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments described herein, and not all of the embodiments. Based on the embodiments described herein, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this document.

[0089] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings herein are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product or device.

[0090] This specification provides the operational steps of the methods described in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or device products, the methods shown in the embodiments or drawings can be executed sequentially or in parallel.

[0091] It should be noted that the experimental data processing methods and apparatus described in this paper can be used in the field of biology, or in any field other than biology. The application fields of the experimental data processing methods and apparatus described in this paper are not limited.

[0092] In one embodiment of this paper, an experimental data processing system is provided to address the problems of low integration, poor versatility, and complex field data flow in existing experimental data processing methods. Specifically, as shown... Figure 1 The experimental data processing system includes: a task management system 101, a data computing system 102, and a data object database 103.

[0093] like Figure 2 As shown, the data object database 103 is used to persistently store experimental data objects. Specifically, the data objects are Python lists, arrays, etc., and are usually personalized JSON strings. The content includes experimental measurement settings parameters, user-stored data, and experimental measurement results. Figure 2 (Data objects of each task in the running data). Among them, user-stored data serves as the raw material for experiments and can be configured according to user needs.

[0094] Taking the optical absorption of an ELISA reader as an example, the parameters set for experimental measurements include wavelength, measurement range, etc. The experimental measurement results are an 8x12 matrix (two-dimensional array), including sampling name, task ID, user ID, data size, usage method, result output method, measurement direction, emission mode wavelength, device name, etc. In practice, users can create various data types according to the actual experimental task requirements, so that users can create data objects based on these data types later.

[0095] In practical implementation, the data object database 103 also stores data dictionaries and front-end display methods (data type library) for various data objects. The data dictionary describes the encoding used when various types of data objects are stored in the database in binary format or retrieved from the database. By setting the data dictionary, it can adapt to various data formats, including but not limited to XML, protoBuf, etc. The front-end display methods describe the graphical display methods used by the data interaction front-end to display the required data. The data interaction front-end queries the front-end display method library to obtain the data display method required by the data object and draws the graphics according to that method. Maintaining the data dictionary and front-end display methods can increase the data type support of the entire system. The data object database is implemented using database technologies, such as relational databases like MySQL or non-relational databases like Redis and MongoDB.

[0096] Task Management System 101 refers to the subsystem that controls the initiation, execution, and termination of experimental tasks. It is used to implement at least the following business functions, which are triggered by the user:

[0097] Service 1 corresponds to the experimental task creation command, which is used to create a new experimental task and start the task's dedicated computing process.

[0098] Service 2: Corresponding data object creation command, used to create data objects and assign initial values ​​during experimental operation (the source of the assigned values ​​can be: experimental instrument measurement data, data calculation results, user input, etc.).

[0099] Service 3: Corresponding data input instructions, used to input user-stored data into the computing process (user-stored data can be regarded as raw materials for experimental preparation).

[0100] Service 4 corresponds to the data modification command, which is used to assign values ​​to data objects during experimental operation.

[0101] Service 5 corresponds to a calculation instruction, used to query data and perform calculations according to the expression in the instruction during experimentation. In one implementation, the expression is used to determine whether a condition is met, for example:

[0102] if(expression){

[0103] Satisfies (true)

[0104] }else{

[0105] Not satisfied (false)

[0106] }

[0107] For example, if a>10, then perform aluminum film sealing; if a<=10, then perform light absorption measurement.

[0108] Service 6 corresponds to the data output instruction, which is used to output experimental data objects from the computing process to user-stored data.

[0109] Service 7: Corresponding task completion instruction, used to complete the task and terminate the task's dedicated computing process.

[0110] The above services 1 to 3 are sent when the experimental task is created, at which point the experiment has not yet started running.

[0111] The data computing system 102 is the core module for managing data object storage and computation, and directly manages the data object database 103. Specifically, it receives business information from the task management system 101, encapsulates it into objects according to the business type, and stores them in a queue to await a response.

[0112] When the instruction for Service 1 (creating a new experimental task and starting a dedicated computing process for the task) is received, a dedicated computing process will be created for that experimental task. In some implementations, such as... Figure 10 As shown, each user has a Jupyter Notebook server process, and each experimental task has its own dedicated computing process, the IPython Kernel process. These IPython Kernel processes connect to the Jupyter Notebook server process of the user to which the experimental task belongs. Within the Jupyter Notebook server process, these dedicated processes for each experimental task access each other via specific WebSocket addresses. After the computing process is established, the data computing system records the WebSocket address of the computing process for instruction distribution to that experimental task. Simultaneously, it establishes an instruction queue for that experimental task to receive, store, and send subsequent instructions.

[0113] In some implementations, the connection relationship between the user's Jupyter Notebook server process and the computing process's WebSocket addresses is stored in a ConcurrentHashMap (WebSocket connection manager). This method allows for retrieval without locking while ensuring security.

[0114] Upon receiving instructions from business functions 2 through 6, these instructions are encapsulated according to their respective business functions and translated into different Python statements. Based on the task ID, these business functions are distributed into the instruction queues of their respective tasks. During implementation, a PriorityBlockingQueue is used to process the Python script tasks sequentially. Each experimental task's dedicated computing process consumes the instructions in the queue and executes the operations accordingly.

[0115] Service 2: Corresponding data object creation command, used to create data objects and assign values ​​during experimental operation.

[0116] Specifically, the instructions for creating a new data object include: variable name, type, size, and initial value.

[0117] Based on the information in the data object creation instruction, the corresponding generated Python statements (i.e., the creation script) are used to create the data object and assign it initial values ​​in the IPython process. Then, based on the data dictionary of that data type recorded in the data type library, the data entry for that data object is created in the runtime data. A success status is returned upon completion. The initial values ​​are optional; any data object creation method will assign a default value.

[0118] Service 3: Corresponding to data input instructions, experimental data objects are input from user-stored data into the calculation process.

[0119] Data input instructions include: variable name, type, size, and object name / ID in user-stored data.

[0120] Based on the data input instructions, the corresponding generated Python statements (input script) are used to create data objects in the IPython process. The value input to the data object is then retrieved from the user-stored data in the data object database by object name / ID and assigned to the created data object. Afterwards, the data entry for that object is created or modified in the runtime data based on the data dictionary of that data type recorded in the data type library. Finally, a success status is returned.

[0121] Service 4: Corresponds to data modification instructions, used to assign values ​​to data objects during experimental operation.

[0122] Data modification instructions include: variable name, type, and value.

[0123] Based on the data modification instructions, the corresponding generated Python statements (input script) are used to modify the value of the data object in the IPython process. The data entry of the object is created or modified in the running data according to the data dictionary of the data type recorded in the data type library. After completion, the success status is returned.

[0124] Service 5: Corresponds to data calculation instructions, used to query data and calculate data based on expressions during experimental operation.

[0125] Data calculation instructions include: type and expression.

[0126] The expression is calculated in the IPython process according to the data calculation instructions and the corresponding generated Python statements (calculation scripts). The data is then encoded according to the data dictionary of the data type, and the encoded expression value is returned after completion.

[0127] Service 6: Corresponds to data output instructions, used to output experimental data objects from the computing process to user-stored data.

[0128] Data output instructions include: variable name, type, size, and object name / ID in user-stored data.

[0129] Based on the data output instructions, the corresponding generated Python statements (query scripts) are used to query the value of the data object in the IPython process. The data object is then created in the user-stored data of the data object database based on its name / ID. Afterwards, the retrieved value is encoded according to the data dictionary of that data type recorded in the data type library and assigned to the data object to complete the data output. A success status is then returned.

[0130] When the instruction to end the Business 7 experiment task is received, the IPython Kernel process will be shut down after the instruction queue of the experiment task is cleared. At that time, the process will no longer be visible in Jupyter notebooks (used to create computing processes on which Python is installed). At the same time, the instruction queue for the experiment task will be deleted and related resources will be released, which means that the experiment task is completely ended.

[0131] In this paper, the device control terminal is an independent process. It responds to the data acquisition commands of the task management system, executes the data acquisition process, and returns the acquired data to the task management system according to the data encoding of a specific data type. Then, the task management system sends the data encoding and the data object to be assigned a value to the data computing system through business function 4, completing the entire process from data acquisition to data assignment in the data computing process.

[0132] The scripts in this paper can be implemented using standard Python statements. That is, device acquisition, user storage and download are all implemented using Python data objects, thereby ensuring the ease of use and learning of the system. Through the standardization of the scripts, the following effects can be achieved: (1) Data can automatically enter and exit the experimental task (input and output); (2) Data can change during the experiment; (3) Data can be used as a parameter for a step in the experiment. By changing the value of the data, the value of the parameter can be changed, thereby affecting the experimental operation.

[0133] Specifically, such as Figure 3 As shown, the interaction process between the task management system 101, the data computing system 102, and the data object database 103 includes:

[0134] Step 1: The task management system 101 sends the task preparation sequence instruction to the data computing system 102.

[0135] The task preparation sequence instructions include: experimental task creation instructions, data object creation instructions, and data input instructions;

[0136] The experimental task creation command is used by the data computing system to establish a task-specific computing process;

[0137] The data object creation instruction is used by the data computing system to create data objects and assign initial values ​​during the computing process.

[0138] The data input instruction is used to create and assign data objects to the computing process based on user-stored data.

[0139] Step 2: The data computing system 102 creates task-related computing processes according to the task preparation sequence instructions, creates data objects in the computing processes, and assigns initial values.

[0140] Step 3: The task management system 101 sends a data acquisition command to the device control terminal, receives the operating data obtained by the device control terminal from operating the experimental equipment, generates a data modification command based on the acquired operating data, and sends it to the data calculation system 102.

[0141] The device control terminal is used to respond to the data acquisition instructions of the task management system for a certain experimental task, operate the experimental equipment to complete the operation and obtain data, and then feed it back to the task management system in a formatted manner.

[0142] Step 4: The data computing system 102 receives a data modification instruction and assigns values ​​to the data objects in the relevant task computing process using the first input script according to the data modification instruction.

[0143] Step 5: Task management system 101 sends data calculation instructions to data calculation system 102.

[0144] Step 7: The data computing system 102 receives the data computing instruction and pushes the computing script to the computing process of the relevant task according to the computing instruction, so as to realize the data computing through the computing script.

[0145] Step 8: The task management system 101 sends a data output instruction to the data computing system 102.

[0146] Step 9: The data computing system 102 receives the data output instruction sent by the task management system, sends the query script to the computing process of the relevant task according to the data output instruction, and stores the data queried by the query script in the data object database 103.

[0147] The experimental data processing system presented in this paper enables a complete business process from data acquisition, on-site processing, storage, and user interaction by creating computation processes based on a general-purpose scripting language within the data computing system according to task requirements and distributing computation scripts to these processes during data processing. All data storage and computation are based on a standard scripting language with a broad user base, offering advantages in ease of use, learning, and versatility.

[0148] In further embodiments of this article, such as Figure 4 As shown, it also includes a data interaction front-end 104, which is operated by users, such as issuing commands to view, modify, add, and delete data. It can also issue commands such as task preparation sequences, data acquisition commands, calculation commands, and output commands. It can also display data objects within the user's authorized scope, including data objects stored in the user's data and data objects during the experiment. It can also respond to the experiment completion command from the task management system and terminate the calculation process. Because the data calculation system is asynchronous, the data interaction front-end can serve multiple experimental tasks simultaneously, but the calculation process of each experimental task is synchronous, and the data from all experimental tasks is executed and fed back sequentially in the form of a queue.

[0149] In practice, a series of thread concurrency tools provided in java.util.concurrent in JDK8 can be used:

[0150] A) Containerization technology uses a thread-safe ConcurrentHashMap to store each user's each task and creates a separate WebSocket connection with Jupyter Notebooks.

[0151] B) Use PriorityBlockingQueue to process Python script tasks sequentially.

[0152] Data persistence is achieved using Redis (a NoSQL database) and MySQL (a relational database). Jupyter Notebooks are deployed elastically using a Kubernetes-based containerized architecture, with the cluster ensuring reliability.

[0153] In addition to enabling automatic acquisition and processing of experimental data, this embodiment also allows users to monitor and intervene in all data operations, thereby improving the user experience.

[0154] In one embodiment of this paper, an experimental data processing method applied to the task management system is provided, such as... Figure 5 As shown, it includes:

[0155] Step 501: Send the task preparation sequence instruction to the data computing system so that the data computing system can create a task-specific computing process according to the task preparation sequence instruction, create data objects in the computing process and assign initial values.

[0156] The task preparation sequence instructions in this step include: experimental task creation instructions, data object creation instructions, and data input instructions.

[0157] The experimental task creation command is used by the data computing system to establish a task-specific computing process. The experimental task creation command includes user information and task information. User information is used to uniquely identify the user, thus associating the experimental task data with the user's process. Task information includes the task name, identification information, etc., used to uniquely identify the experimental task. In specific implementations, in addition to including this information, the Python library is also imported.

[0158] The data object creation instruction is used by the data computing system to create data objects and assign initial values ​​during the computing process. The data object creation instruction includes: variable name, variable type, size, and variable value. The variable type can be a Python list, array, etc., but is typically a custom JSON string. The variable value is specified in the data object creation instruction. The size is used to constrain the array to be a two-dimensional array, for example, 3x4.

[0159] Data input instructions are used to create and assign data objects to the computing process based on user-stored data. These instructions include: variable name, type, size, and the name / ID of the object in the user-stored data.

[0160] Step 502: Send a data acquisition command to the device control terminal, receive the device control terminal's operation of the experimental equipment and acquire operational data; generate a data modification command based on the acquired operational data and send it to the data computing system, so that the data computing system can assign values ​​to the data objects in the relevant task calculation process. The operational data refers to the data of all variables in the calculation process.

[0161] Step 503: Send a data calculation instruction to the data calculation system so that the data calculation system can perform calculations on the data in the relevant task calculation process according to the data calculation instruction.

[0162] The data calculation instructions are Python script code. In some implementations, the script code is an expression that corresponds to a calculated value. This value can be a data type or a judgment result of true or false.

[0163] Step 504: Send a data output instruction to the data computing system so that the data computing system can store the relevant computing process data into the data object database according to the data output instruction.

[0164] The data output instructions include: variable name, type, size, and object name / ID from user-stored data. The variable name identifies the variable to be output, and the type distinguishes between input and output. The object name / ID from user-stored data is used to create a data object in the user-stored data and assign values ​​based on the variable's numerical value.

[0165] Furthermore, it also includes:

[0166] The system receives a data query request from a user for a target task within its authorized scope, sends the data query instruction to the data computing system, and then the target task's computing process queries data from the object database and returns it according to the data query instruction.

[0167] In practice, user data query requests are also recorded to facilitate subsequent tracking of user information and ensure data security.

[0168] In one embodiment of this article, such as Figure 6 As shown, an experimental data processing method for use in a data computing system is also provided, including:

[0169] Step 601: Receive the task preparation sequence instruction sent by the task management system, start the task-related calculation process according to the task preparation sequence instruction, create data objects in the calculation process and assign initial values.

[0170] The task preparation sequence instructions include: experimental task creation instructions, data object creation instructions, and data input instructions. The specific implementation process of this step includes: receiving the experimental task creation instruction sent by the task management system, and establishing a task-specific computing process according to the instruction; receiving the data object creation instruction sent by the task management system, and creating a data object in the relevant computing process using an object creation script according to the instruction; receiving the data input instruction sent by the task management system, and assigning the user-defined stored data to the data object in the computing process using a second input script according to the instruction.

[0171] Step 602: Receive data modification instructions sent by the task management system, and assign values ​​to data objects in the relevant task calculation process using the input script according to the data modification instructions.

[0172] Step 603: Receive data calculation instructions sent by the task management system, push the calculation script to the calculation process of the relevant task according to the data calculation instructions, and realize data calculation through the calculation script.

[0173] Step 604: Receive the data output instruction sent by the task management system, send the query script to the calculation process of the relevant task according to the data output instruction, and store the data queried by the query script into the data object database.

[0174] Furthermore, it also includes:

[0175] It receives data interaction instructions sent by the data interaction front end and sends the data interaction instructions to the computing process of the relevant task so that the computing process can realize the interaction.

[0176] The data interaction front end displays an interactive interface, which shows data objects within the user's permissions and interactive controls for those data objects. Data interaction commands are issued through the interactive controls of the data objects, including but not limited to commands for viewing, modifying, adding, and deleting data.

[0177] Based on the same inventive concept, this paper also provides an experimental data processing device, as described in the following embodiments. Since the principle by which the experimental data processing device solves the problem is similar to that of the experimental data processing method, the implementation of the experimental data processing device can refer to the experimental data processing method, and repeated details will not be elaborated further.

[0178] Specifically, such as Figure 7 As shown, the experimental data processing device applied to the task management system includes:

[0179] The task creation control unit 701 is used to send task preparation sequence instructions to the data computing system, so that the data computing system can create a task-specific computing process according to the task preparation sequence instructions, create data objects in the computing process and assign initial values;

[0180] The data acquisition and control unit 702 is used to send data acquisition commands to the device control terminal, receive data acquired by the device control terminal from operating the experimental equipment, and generate data modification commands to the data calculation system based on the acquired data, so that the data calculation system can assign values ​​to data objects in the relevant task calculation process.

[0181] The data calculation control unit 703 is used to send data calculation instructions to the data calculation system, so that the data calculation system can calculate the data in the relevant task calculation process according to the data calculation instructions;

[0182] The data output control unit 704 is used to send data output instructions to the data computing system, so that the data computing system can store the relevant computing process data to the data object database according to the data output instructions.

[0183] like Figure 8 As shown, the experimental data processing device applied to the data computing system includes:

[0184] The task creation unit 801 is used to receive the task preparation sequence instruction sent by the task management system, start the task-related calculation process according to the task preparation sequence instruction, create data objects in the calculation process and assign initial values;

[0185] The data modification unit 802 is used to receive data modification instructions sent by the task management system, and to assign values ​​to data objects in the relevant task calculation process using the first input script according to the data modification instructions;

[0186] The data calculation unit 803 is used to receive calculation instructions sent by the task management system, push calculation scripts to the calculation process of relevant tasks according to the calculation instructions, and realize data calculation through the calculation scripts.

[0187] The data output unit 804 is used to receive data output instructions sent by the task management system, send query scripts to the calculation process of the relevant tasks according to the data output instructions, and store the data queried by the query scripts into the data object database.

[0188] This embodiment achieves a complete business process from data acquisition, on-site processing, storage, and user interaction by establishing a computing process based on a general-purpose scripting language. All data storage and computation are based on a standard scripting language with a broad user base, offering advantages in ease of use, learning, and versatility.

[0189] The experimental data processing method, apparatus, and system presented in this paper support script requests from multiple users and multiple experimental tasks. Utilizing the factory pattern and containerization principles, each experimental task is isolated and managed, with each task assigned an independent WebSocket connection. A blocking priority queue is used to control the sequential execution of each script within a task. Based on object-oriented principles, user-defined data objects are supported to meet the needs of various biological experimental data, ensuring the data computation requirements of biological experiments and improving experimental flexibility. Data generated by each experimental task is cached or permanently stored, allowing users to conveniently and quickly download and view their data in real time, and to observe the Python script execution process in Jupyter Notebooks.

[0190] In one embodiment of this document, a computer device is also provided, such as... Figure 9As shown, computer device 902 may include one or more processors 904, such as one or more central processing units (CPUs), each of which may implement one or more hardware threads. Computer device 902 may also include any memory 906 for storing information of any kind, such as code, settings, data, etc. Non-limitingly, for example, memory 906 may include any type of RAM, any type of ROM, flash memory, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Furthermore, any memory may provide volatile or non-volatile retention of information. Furthermore, any memory may represent a fixed or removable component of computer device 902. In one case, when processor 904 executes an associated request stored in any memory or combination of memories, computer device 902 may perform any operation of the associated request. Computer device 902 also includes one or more drive mechanisms 908 for interacting with any memory, such as hard disk drive mechanisms, optical disk drive mechanisms, etc.

[0191] Computer device 902 may also include an input / output module 910 (I / O) for receiving various inputs (via input device 912) and providing various outputs (via output device 914). A specific output mechanism may include a presentation device 916 and an associated graphical user interface (GUI) 918. In other embodiments, the input / output module 910 (I / O), input device 912, and output device 914 may be omitted, and the device may function solely as a computer device within a network. Computer device 902 may also include one or more network interfaces 920 for exchanging data with other devices via one or more communication links 922. One or more communication buses 924 couple the components described above together.

[0192] Communication link 922 can be implemented in any way, such as via a local area network (LAN), a wide area network (WAN) (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication link 922 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0193] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the above-described method.

[0194] This embodiment also provides a computer-readable request, wherein when a processor executes the request, the program therein causes the processor to perform the method of any of the foregoing embodiments.

[0195] It should be understood that in the various embodiments of this document, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.

[0196] It should also be understood that, in the embodiments herein, the term "and / or" is merely a description of the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following associated objects have an "or" relationship.

[0197] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this document.

[0198] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0199] In the embodiments provided herein, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, or they may be electrical, mechanical, or other forms of connection.

[0200] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments described herein, depending on actual needs.

[0201] Furthermore, the functional units in the various embodiments of this document can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0202] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this document, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several requests to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this document. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0203] This document uses specific embodiments to illustrate the principles and implementation methods of this document. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this document. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this document. Therefore, the content of this specification should not be construed as a limitation of this document.

Claims

1. A method for processing experimental data, characterized in that, Applications in task management systems include: Send task preparation sequence instructions to the data computing system so that the data computing system can create a task-specific computing process according to the task preparation sequence instructions, create data objects in the computing process and assign initial values; after the task-specific computing process is established, create and record the websocket address of the computing process; connect the websocket address of the computing process to the Jupyter notebook server process of the user to which the relevant experimental task belongs, so that the Jupyter notebook server process can access the connected computing process through the websocket address; put the connection relationship between the user's Jupyter notebook server process and the computing process's websocket address in a ConcurrentHashMap; Send data acquisition instructions to the device control terminal, receive operation data from the device control terminal to operate the experimental equipment, and obtain operating data; generate data modification instructions based on the obtained operating data and send them to the data computing system, so that the data computing system can assign values ​​to data objects in the relevant task calculation process; Send data calculation instructions to the data calculation system so that the data calculation system can calculate the data in the relevant task calculation process according to the data calculation instructions; A data output instruction is sent to the data computing system, so that the data computing system stores the relevant computing process data into the data object database according to the data output instruction.

2. The method as described in claim 1, characterized in that, The task preparation sequence instructions include: experimental task creation instructions, data object creation instructions, and data input instructions; The experimental task creation command is used by the data computing system to establish a task-specific computing process; The data object creation instruction is used by the data computing system to create data objects and assign initial values ​​during the computing process. The data input instruction is used to create and assign data objects to the computing process based on user-stored data.

3. The method as described in claim 1, characterized in that, Also includes: The system receives a data query instruction from a user for a target task within its authorized scope, and sends the data query instruction to the data computing system so that the computing process of the target task can query data from the data object database according to the data query instruction and return it.

4. A method for processing experimental data, characterized in that, Applied to data computing systems, including: The system receives task preparation sequence instructions from the task management system, starts a task-specific computing process according to the instructions, creates data objects and assigns initial values ​​within the computing process, establishes and records the WebSocket address of the computing process, connects the WebSocket address of the computing process to the Jupyter Notebook Server process of the user to which the relevant experimental task belongs, so that the Jupyter Notebook Server process can access the connected computing process through the WebSocket address, and places the connection relationship between the user's Jupyter Notebook Server process and the computing process's WebSocket address in a ConcurrentHashMap. Receive data modification instructions sent by the task management system, and assign values ​​to data objects in the relevant task calculation process using the first input script according to the data modification instructions; Receive data calculation instructions sent by the task management system, push calculation scripts to the calculation process of relevant tasks according to the data calculation instructions, and realize data calculation through the calculation scripts; The system receives data output instructions from the task management system, sends query scripts to the calculation processes of relevant tasks according to the data output instructions, and stores the data retrieved by the query scripts into the data object database.

5. The method as described in claim 4, characterized in that, The task preparation sequence instructions include: experimental task creation instructions, data object creation instructions, and data input instructions; Receiving a task preparation sequence instruction sent by the task management system, starting a task-related computing process according to the task preparation sequence instruction, and creating data objects and assigning initial values ​​in the computing process include: Receive the experimental task creation instruction sent by the task management system, and establish a task-specific computing process according to the experimental task creation instruction; Receive a data object creation instruction sent by the task management system, and create a data object in the relevant computing process according to the data object creation instruction using an object creation script; The system receives a data input instruction from the task management system and uses a second input script to assign user-stored data to the data object of the computing process according to the data input instruction.

6. The method as described in claim 5, characterized in that, Also includes: Receive data query instructions for the target task sent by the task management system; According to the data query instruction, a query script is sent to the computing process of the target task, so that the computing process of the target task can query data from the data object database according to the query script and return it to the task management system.

7. The method as described in claim 5, characterized in that, Also includes: The system receives data interaction instructions sent by a data interaction front-end, which displays an interactive interface. The interactive interface displays data objects within the user's permissions and interactive controls for the data objects. Data interaction instructions are issued through the interactive controls of the data objects. The data interaction instructions are sent to the computing process of the relevant task so that the computing process can perform the interaction.

8. An experimental data processing system, characterized in that, include: Task management system, data computing system, and data object database; The task management system is used to send task preparation sequence instructions to the data computing system; Send data acquisition instructions to the device control terminal, receive data acquired by the device control terminal during operation of the experimental equipment; generate data modification instructions based on the acquired data and send them to the data calculation system; send data calculation instructions to the data calculation system; send data output instructions to the data calculation system. The data computation system is used to initiate a task-specific computation process according to the task preparation sequence instructions; create data objects and assign initial values ​​within the computation process; after establishing the task-specific computation process, establish and record the WebSocket address of the computation process; connect the WebSocket address of the computation process to the Jupyter Notebook server process of the user to which the relevant experimental task belongs, so that the Jupyter Notebook server process can access the connected computation process through the WebSocket address; place the connection relationship between the user's Jupyter Notebook server process and the computation process's WebSocket address in a ConcurrentHashMap; assign values ​​to the data objects in the relevant task computation process according to the data modification instructions; perform calculations on the data in the relevant task computation process according to the data computation instructions; and store the data of the relevant computation process into the data object database according to the data output instructions.

9. The system as described in claim 8, characterized in that, The data object database also stores a data dictionary and data display methods; The data dictionary is used to describe the storage and parsing methods of various types of data objects and is invoked during data storage and data retrieval. The data display method is used to describe how the data interaction front end displays the data to be displayed, and is called when the data is read.

10. An experimental data processing device, characterized in that, Applications in task management systems include: The task creation control unit sends task preparation sequence instructions to the data computing system, enabling the system to create a task-specific computing process, create data objects within that process, and assign initial values. After establishing the task-specific computing process, it creates and records the process's WebSocket address. It then connects the computing process's WebSocket address to the Jupyter Notebook server process of the user to which the relevant experimental task belongs, allowing the Jupyter Notebook server process to access the connected computing process via the WebSocket address. Finally, it stores the connection relationship between the user's Jupyter Notebook server process and the computing process's WebSocket address in a ConcurrentHashMap. The data acquisition and control unit is used to send data acquisition commands to the device control terminal, receive data acquired by the device control terminal from operating the experimental equipment, and generate data modification commands to the data calculation system based on the acquired data, so that the data calculation system can assign values ​​to data objects in the relevant task calculation process. A data calculation control unit is used to send data calculation instructions to the data calculation system, so that the data calculation system can calculate the data in the relevant task calculation process according to the data calculation instructions; A data output control unit is used to send data output instructions to the data computing system, so that the data computing system can store the relevant computing process data into the data object database according to the data output instructions.

11. An experimental data processing device, characterized in that, Applied to data computing systems, including: The task creation unit is used to receive task preparation sequence instructions sent by the task management system, start a task-specific computing process according to the task preparation sequence instructions, create data objects in the computing process and assign initial values; after establishing the task-specific computing process, establish and record the WebSocket address of the computing process; connect the WebSocket address of the computing process to the Jupyter Notebook Server process of the user to which the relevant experimental task belongs, so that the Jupyter Notebook Server process can access the connected computing process through the WebSocket address; and place the connection relationship between the user's Jupyter Notebook Server process and the computing process's WebSocket address in a ConcurrentHashMap. The data modification unit is used to receive data modification instructions sent by the task management system, and to assign values ​​to data objects in the relevant task calculation process according to the data modification instructions using a first input script; The data calculation unit is used to receive data calculation instructions sent by the task management system, push calculation scripts to the calculation process of relevant tasks according to the data calculation instructions, and realize data calculation through the calculation scripts; The data output unit is used to receive data output instructions sent by the task management system, send query scripts to the calculation process of the relevant tasks according to the data output instructions, and store the data queried by the query scripts into the data object database.

12. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 7.

13. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor of the computer device, it implements the method of any one of claims 1 to 7.

Images