Experimental data processing method and apparatus, and system

Abstract

An experimental data processing method and apparatus, and a system, relating to the field of automatic data processing. The method comprises: sending a task preparation sequence instruction to a data computation system, so that the data computation system, on the basis of the task preparation sequence instruction, creates a task-related computation process, creates a data object in the computation process, and assigns an initial value (501); sending a data acquisition instruction to a device control end, and receiving data acquired by the device control end operating an experiment device; on the basis of the acquired data, generating a data modification instruction to the data computation system, so that the data computation system assigns a value to the data object in the related task computation process (502); sending a data computation instruction to the data computation system, so that, on the basis of the data computation instruction, the data computation system performs computation on data of the related task computation process (503); and sending a data output instruction to the data computation system, so as to, on the basis of the data output instruction, store in a data object database the data of the related computation process (504). According to the described method, a full-service process can be implemented. In addition, the method has the advantages of being easy to use and easy to learn and being versatile.

Description

Experimental data processing method, device and system Technical Field

[0001] This article relates to the field of data processing, and in particular to an experimental data processing method, device and system. Background Art

[0002] Laboratory automation and intelligence are important development directions for future life science research. Many domestic and foreign companies and research institutions have begun to develop integrated laboratory automation systems, integrating robots and various types of experimental equipment to achieve automated experiments. Standardizing and universalizing the operating parameters and collected operating data required by various types of laboratory equipment is essential for achieving comprehensive laboratory automation and intelligence. However, various types of experimental equipment in laboratories currently lack unified data collection and on-site processing methods. The current laboratory integration technology solutions on the market are highly dependent on the copying and moving of various data files and customized programming scripts. As a result, the current automated experimental system has low integration and poor versatility, making it difficult to analyze experimental data on-site and use it for subsequent experiments. At the same time, the flow of data and files on the experimental site is complex and difficult to track and monitor, resulting in poor security.

[0003] Summary of the Invention

[0004] This paper aims to solve the problems of low integration, poor versatility and complex field data flow in existing experimental data processing methods.

[0005] In order to solve the above problems, in one embodiment of the present invention, a method for processing experimental data is provided, which is applied to a task management system, including:

[0006] Sending a task preparation sequence instruction to the data computing system, so that the data computing system creates a task-specific computing process according to the task preparation sequence instruction, creates a data object in the computing process and assigns an initial value;

[0007] Sending data acquisition instructions to the device control terminal, which then operates the experimental equipment and obtains operating data; generating data modification instructions based on the obtained operating data to the data computing system, so that the data computing system can assign values ​​to data objects in the relevant task computing process;

[0008] Sending a data calculation instruction to the data calculation system, so that the data calculation system calculates the data in the relevant task calculation process according to the data calculation instruction;

[0009] A data output instruction is sent to the data computing system, so that the data computing system stores the data of the related computing process into the data object database according to the data output instruction.

[0010] In a further embodiment, the task preparation sequence instructions include: an experimental task creation instruction, a data object creation instruction, and a data input instruction;

[0011] The experimental task creation instruction is used for the data computing system to establish a task-specific computing process;

[0012] The data object creation instruction is used for the data computing system to create a data object and assign an initial value in the computing process;

[0013] The data input instruction is used to create and assign data objects to the computing process based on user stored data.

[0014] In a further embodiment, it also includes:

[0015] Receive a data query request from a user for a target task within the scope of authority, and send the data query instruction to the data computing system so that the computing process of the target task queries and returns data from the object database according to the data query instruction.

[0016] The second aspect of this invention provides an experimental data processing method, which is applied to a data computing system, comprising:

[0017] Receive a task preparation sequence instruction sent by the task management system, start a task-related computing process according to the task preparation sequence instruction, create a data object in the computing process and assign an initial value;

[0018] Receiving a data modification instruction sent by the task management system, and assigning a value to a data object in a related task computing process using a first input script according to the data modification instruction;

[0019] Receive data calculation instructions sent by the task management system, push calculation scripts to the calculation process of related tasks according to the data calculation instructions, and realize data calculation through the calculation scripts;

[0020] Receive the data output instruction sent by the task management system, send the query script to the computing process of the relevant task according to the data output instruction, and store the data queried by the query script in the data object database.

[0021] In a further embodiment, the task preparation sequence instructions include: an experimental task creation instruction, a data object creation instruction, and a data input instruction;

[0022] Receiving a task preparation sequence instruction sent by a task management system, starting a task-related computing process according to the task preparation sequence instruction, creating a data object in the computing process and assigning an initial value includes:

[0023] Receive an experimental task creation instruction sent by the task management system, and establish a task-specific computing process according to the experimental task creation instruction;

[0024] receiving a data object creation instruction sent by the task management system, and creating a data object in a related computing process according to the data object creation instruction;

[0025] Receive the data input instruction sent by the task management system, and assign initial values ​​to data objects of related computing processes according to the data input instruction.

[0026] In a further embodiment, after establishing a task-specific computing process according to the experimental task creation instruction, the method further includes:

[0027] Connect the task-specific computing process to the Web application to which the task belongs, so that the Web application can access the WebSocket address of the task-specific computing process.

[0028] In a further embodiment, it also includes:

[0029] Receive a data query instruction for a target task sent by the task management system; send the data query instruction to a computing process of the target task, so that the computing process of the target task queries data from the object database and returns the data to the task management system.

[0030] In a further embodiment, it also includes:

[0031] Receiving a data interaction instruction sent by a data interaction front end, wherein the data interaction front end displays an interaction interface, wherein the interaction interface displays data objects within the user's authority and interaction controls of the data objects, and issuing a data interaction instruction through the interaction controls of the data objects;

[0032] The data interaction instruction is sent to the computing process of the relevant task so that the computing process can realize the interaction.

[0033] The third aspect of this invention provides an experimental data processing system, comprising: a task management system, a data calculation system and a data object database;

[0034] The task management system is used to send task preparation sequence instructions to the data computing system; send data acquisition instructions to the equipment control terminal, receive data obtained by the equipment control terminal operating the experimental equipment; generate modification instructions to the data computing system based on the obtained data; send data calculation instructions to the data computing system; and send data output instructions to the data computing system;

[0035] The data computing system is used to receive task preparation sequence instructions sent by the task management system, start the task-related computing process according to the task preparation sequence instructions, create data objects in the computing process and assign initial values; receive modification instructions sent by the task management system, assign values ​​to data objects in the computing process of the relevant task according to the modification instructions; receive data computing instructions sent by the task management system, push computing scripts to the computing process of the relevant task according to the data computing instructions, and realize data computing through the computing scripts; receive data output instructions sent by the task management system, and store the assigned data or computing results in the data object database according to the data output instructions.

[0036] In a further embodiment, the data object database further stores a data dictionary and a data display method;

[0037] The data dictionary is used to describe the storage and parsing methods of various types of data objects;

[0038] The data display method is used to describe the way in which the data interaction front end displays the required display data.

[0039] A fourth aspect of the present invention provides an experimental data processing device, which is applied to a task management system and includes:

[0040] The task creation control unit is used to send a task preparation sequence instruction to the data computing system, so that the data computing system creates a task-related computing process according to the task preparation sequence instruction, creates a data object in the computing process and assigns an initial value;

[0041] The data acquisition control unit is used to send data acquisition instructions to the device control terminal, receive data acquired by the device control terminal operating the experimental equipment, and generate modification instructions to the data computing system based on the acquired data, so that the data computing system can assign values ​​to data objects in the relevant task computing process;

[0042] A data calculation control unit, configured to send a data calculation instruction to the data calculation system, so that the data calculation system pushes a calculation script to the calculation process of the relevant task according to the data calculation instruction, and realizes data calculation through the calculation script;

[0043] The data output control unit is used to send a data output instruction to the data computing system, so that the data computing system stores the assigned data or calculation results into the data object database according to the data output instruction.

[0044] A fifth aspect of the present invention provides an experimental data processing device, which is applied to a data computing system and includes:

[0045] A task creation unit, configured to receive a task preparation sequence instruction sent by the task management system, start a task-related computing process according to the task preparation sequence instruction, and create a data object in the computing process;

[0046] The data modification unit is used to receive the modification instructions sent by the task management system and assign values ​​to the data objects in the computing process according to the modification instructions;

[0047] The data calculation unit is used to receive data calculation instructions sent by the task management system, push the calculation script to the calculation process of the relevant task according to the data calculation instruction, and realize data calculation through the calculation script;

[0048] The data output unit is used to receive the data output instruction sent by the task management system, and store the assigned data or calculation results in the data object database according to the data output instruction.

[0049] A sixth aspect of this document provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any one of the aforementioned embodiments when executing the computer program.

[0050] A seventh aspect of this document provides a computer storage medium having a computer program stored thereon, which implements the method described in any of the aforementioned embodiments when the computer program is executed by a processor of a computer device.

[0051] The present invention provides a method, device, and system for processing experimental data. By developing a computing process based on a universal scripting language, the system implements a full business process from data acquisition, on-site processing, storage, and user interaction. All data storage and calculations are based on a standard scripting language with a broad public base, offering the advantages of ease of use, ease of learning, and versatility. In addition, each experimental task establishes a computing process, allowing each task to have its own computing client. The serial execution of each computing process ensures orderliness, and each computing variable is persistently updated in the data object database, ensuring data security.

[0052] In order to make the above and other purposes, features and advantages of this article more obvious and easy to understand, the following specifically cites preferred embodiments and provides detailed descriptions in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of this article or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of this article. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] FIG1 shows a structural diagram of the experimental data processing system according to an embodiment of the present invention;

[0055] FIG2 is a schematic diagram showing data stored in a data object database according to an embodiment of the present invention;

[0056] FIG3 shows an interactive flow chart of the experimental data processing system according to an embodiment of the present invention;

[0057] FIG4 shows another structural diagram of the experimental data processing system according to the embodiment of this invention;

[0058] FIG5 shows a flow chart of the experimental data processing method of the task management system side of the embodiment of this invention;

[0059] FIG6 shows a flow chart of the experimental data processing method on the data calculation system side of the embodiment of this invention;

[0060] FIG7 shows a structural diagram of an experimental data processing device on the task management system side of an embodiment of this invention;

[0061] FIG8 shows a structural diagram of an experimental data processing device on the data calculation system side of an embodiment of this invention;

[0062] FIG9 shows a structural diagram of a computer device according to an embodiment of the present invention;

[0063] FIG10 shows a flowchart of the task processing process of the embodiment of this invention.

[0064] Explanation of the symbols in the accompanying drawings: 101, task management system; 102, data computing system; 103, data object database; 104, data interaction front end; 701, task creation control unit; 702, data acquisition control unit; 703, data computing control unit; 704, data output control unit; 801, task creation unit; 802, data modification unit; 803, data computing unit; 804, data output unit; 902, computer equipment; 904, processor; 906, memory; 908, drive mechanism; 910, input / output module; 912, input device; 914, output device; 916, presentation device; 918, graphical user interface; 920, network interface; 922, communication link; 924, communication bus. DETAILED DESCRIPTION

[0065] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of this document. Obviously, the embodiments described are only part of the embodiments of this document, not all of the embodiments. Based on the embodiments of this document, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this document.

[0066] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of this document and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products or equipment.

[0067] This specification provides method operation steps as described in the embodiments or flowcharts, but more or fewer operation steps may be included based on routine or non-creative work. The order of steps listed in the embodiments is only one way of executing the steps among many orderings and does not represent the only execution order. When a system or device product is actually executed, the method can be executed in the order shown in the embodiments or the drawings or in parallel.

[0068] It should be noted that the experimental data processing method and device in this article can be used in the biological field, and can also be used in any field other than the biological field. The application field of the experimental data processing method and device in this article is not limited.

[0069] In one embodiment, an experimental data processing system is provided to address the problems of existing experimental data processing methods, such as low integration, poor versatility, and complex field data flow. Specifically, as shown in FIG1 , the experimental data processing system includes a task management system 101 , a data calculation system 102 , and a data object database 103 .

[0070] As shown in Figure 2, the data object database 103 is used to persistently store experimental data objects. Specifically, the data objects are lists, arrays, etc. in Python, and are usually personalized JSON strings. The content includes experimental measurement parameters, user stored data, and experimental measurement results (the data objects of each task in the running data in Figure 2). Among them, user stored data serves as the raw material of the experiment and can be set according to user needs.

[0071] Taking the optical absorption of a microplate reader as an example, experimental measurement settings include parameters such as wavelength and measurement range. The experimental measurement results are an 8 x 12 matrix (a two-dimensional array) that includes the sampling name, task ID, user ID, data size, usage instructions, result output method, measurement direction, emission mode wavelength, device name, etc. During implementation, users can establish various data types based on actual experimental task requirements, allowing subsequent users to create data objects based on these data types.

[0072] During specific implementation, the data object database 103 also stores a data dictionary and front-end display methods (data type library) for various data objects. The data dictionary is used to describe the encoding used when storing each type of data object in binary form in the database or reading from the database. By setting the data dictionary, various data formats can be adapted, including but not limited to XML, protoBuf, etc. The front-end display method is used to describe the graphical display method of the required display data by the data interaction front-end. The data interaction front-end obtains the data display method required for the data object by querying the front-end display method library and draws the graph according to the method. By maintaining the data dictionary and front-end display method, the data type support supported by the entire system can be increased. The data object database is implemented using database technology, such as the relational database MySQL or the non-relational database Redis or MongoDB.

[0073] The task management system 101 is a subsystem that controls the initiation, execution, and termination of the experimental task process. It is used to implement at least the following services, which are triggered by the user:

[0074] Business 1, corresponding to the experimental task creation instruction, is used to create a new experimental task and start the task-specific computing process.

[0075] Business 2, corresponding to the data object creation instruction, is used to create a new data object and assign an initial value during the experiment (the source of the assigned value can be: experimental instrument measurement data, data calculation results, user input, etc.).

[0076] Business 3, corresponding to data input instructions, is used to input user stored data into the calculation process (user stored data can be regarded as raw materials for experimental preparation).

[0077] Business 4, corresponding to data modification instructions, is used to assign values ​​to data objects during experimental operation.

[0078] Business 5, corresponding calculation instructions, is used to query data during the experiment and perform calculations according to the expression in the instruction. In one embodiment, the expression is used to determine whether the condition is met, for example:

[0079] For example, if a>10, then aluminum film sealing is performed, and if a<=10, then absorbance measurement is performed.

[0080] Business 6, corresponding data output instructions, are used to output experimental data objects from the calculation process to user storage data.

[0081] Service 7, corresponding to the task completion instruction, is used to complete the task and end the task-specific computing process.

[0082] The above services 1 to 3 are sent when the experiment task is created. At this time, the experiment has not yet started.

[0083] The data computing system 102 is the core module for managing data object storage and computing, and directly manages the data object database 103. It is specifically used to receive business information from the task management system 101, encapsulate it into objects according to business types, and store it in a queue waiting for response.

[0084] When the instruction of business 1 (creating a new experimental task and starting a task-specific computing process) is received, a dedicated computing process will be established for the experimental task. In some implementations, as shown in Figure 10, each user will have a jupyter notebook server process, and the computing process established for each experimental task is an IPython Kernel process. These IPython Kernel processes are connected to the jupyter notebook server process of the user to whom the experimental task belongs. In the jupyter notebook server process, the dedicated processes of these experimental tasks are accessed through a specific websocket address. After the establishment of the computing process is completed, the data computing system will record the websocket address of the computing process for the instruction distribution of the experimental task, and at the same time establish an instruction queue for the experimental task to receive, store, and send subsequent instructions for these tasks.

[0085] In some implementations, the connection between the user's Jupyter Notebook server process and the websocket address of the computing process is placed in a ConcurrentHashMap (websocket connection manager). This approach allows for secure retrieval without locking.

[0086] After receiving instructions from tasks 2 to 6, these instructions are packaged according to the different tasks and translated into different Python statements. These tasks are distributed to the instruction queue of each task according to the task ID. During implementation, PriorityBlockingQueue (blocking priority queue) is used to process Python script tasks in sequence. The dedicated computing process of each experimental task will consume the instructions in the queue and execute the operation in turn:

[0087] Business 2: corresponds to the data object creation instruction, which is used to create a new data object and assign values ​​during the experiment.

[0088] Specifically, the data object creation instruction includes: variable name, type, size, and initial value.

[0089] Based on the information in the data object creation instruction, the corresponding Python statement (i.e., the new script) is used to create a data object in the IPython process and assign initial values. Then, a data entry for the data object is created in the running database using the data dictionary for that data type recorded in the data type library. Upon completion, a success response is returned. The initial value is optional; all data object creation methods assign a default value.

[0090] Business 3: Corresponding to the data input instruction, the experimental data object inputs the user stored data into the calculation process.

[0091] Data input instructions include: variable name, type, size, object name / ID in user stored data.

[0092] Create a data object in the IPython process using the generated Python statements (input script) according to the data input instructions. The value entered for the data object is retrieved from the user-stored data in the data object database using the object name / ID and assigned to the created data object. Then, create or modify the data entry for the object in the running data according to the data dictionary for the data type recorded in the data type library. Upon completion, the success rate is returned.

[0093] Service 4: corresponds to the data modification instruction, which is used to assign values ​​to data objects during the experiment operation.

[0094] Data modification instructions include: variable name, type, and value.

[0095] According to the data modification instructions, the corresponding generated Python statements (input scripts) are used to modify the value of the data object in the IPython process. According to the data dictionary of the data type recorded in the data type library, the data entry of the object is created or modified in the running data. After completion, whether it is successful is returned.

[0096] Service 5: corresponds to data calculation instructions, which are used to query data and calculate data based on expressions during experiment operation.

[0097] Data calculation instructions include: type and expression.

[0098] According to the data calculation instruction, the corresponding Python statement (calculation script) is used to calculate the value of the expression in the IPython process, and the data is encoded according to the data dictionary of the data type. After completion, the encoded expression value is returned.

[0099] Service 6: corresponds to the data output instruction, which is used to output the experimental data object from the calculation process to the user storage data.

[0100] Data output instructions include: variable name, type, size, object name / ID in user stored data.

[0101] The generated Python statement (query script) is used in the IPython process to query the value of the data object according to the data output instruction. The data object is created based on the object name / ID in the user stored data in the data object database. The query value is then encoded according to the data dictionary of the data type recorded in the data type library and assigned to the data object to complete the data output. Upon completion, the success rate is returned.

[0102] When the end instruction for the business 7 experimental task is received, the IPython Kernel process will be closed when the instruction queue for the experimental task is cleared. At that time, the process will no longer be visible from Jupyter Notebooks (used to create computing processes and install Python on them). At the same time, the instruction queue for the experimental task is deleted, and related resources are released, which means that the experimental task is completely completed.

[0103] The device control end, in this context, is an independent process. It responds to data collection instructions from the task management system, executes the operational data collection process, and returns the collected operational data to the task management system using a data encoding of a specific data type. The task management system then sends this data encoding and the data object to be assigned a value to the data calculation system via service 4, completing the entire process from data collection to data assignment in the data calculation process.

[0104] The scripts in this article can be implemented using standard Python statements, that is, device acquisition, user storage and downloading are all implemented using Python data objects, thus ensuring the ease of use and learning of the system. Through the standardization of scripts, the following effects can be achieved: (1) Data can automatically enter and exit experimental tasks (input and output); (2) Data can change during the experiment; (3) Data can be used as a parameter of a step in the experimental process. By changing the value of the data, the value of the parameter can be changed, thereby affecting the experimental operation.

[0105] Specifically, as shown in FIG3 , the interaction process between the task management system 101 , the data computing system 102 and the data object database 103 includes:

[0106] Step 1: The task management system 101 sends a task preparation sequence instruction to the data computing system 102 .

[0107] Among them, the task preparation sequence instructions include: experimental task creation instructions, data object creation instructions and data input instructions;

[0108] The experimental task creation instruction is used for the data computing system to establish a task-specific computing process;

[0109] The data object creation instruction is used for the data computing system to create a data object and assign an initial value in the computing process;

[0110] The data input instruction is used to create and assign data objects to the computing process based on user stored data.

[0111] Step 2: The data computing system 102 creates a task-related computing process according to the task preparation sequence instruction, creates a data object in the computing process, and assigns an initial value.

[0112] Step 3: The task management system 101 sends a data acquisition instruction to the device control terminal, receives the operating data obtained by the device control terminal operating the experimental equipment, generates a data modification instruction based on the obtained operating data, and sends it to the data calculation system 102.

[0113] Among them, the equipment control terminal is used to respond to the data collection instructions of a certain experimental task from the task management system, operate the experimental equipment to complete the operation and obtain data, and feed it back to the task management system in a formatted manner.

[0114] Step 4: The data computing system 102 receives the data modification instruction and uses the first input script to assign values ​​to data objects in the relevant task computing process according to the data modification instruction.

[0115] Step 5: The task management system 101 sends a data calculation instruction to the data calculation system 102 .

[0116] In step 7, the data calculation system 102 receives the data calculation instruction, pushes the calculation script to the calculation process of the relevant task according to the calculation instruction, and realizes the data calculation through the calculation script.

[0117] In step 8 , the task management system 101 sends a data output instruction to the data computing system 102 .

[0118] Step 9: The data computing system 102 receives the data output instruction sent by the task management system, sends the query script to the computing process of the relevant task according to the data output instruction, and stores the data retrieved by the query script into the data object database 103.

[0119] The experimental data processing system presented in this article implements the entire business process, from data acquisition and on-site processing to storage and user interaction, by creating computational processes based on a common scripting language within the data computing system according to task requirements and issuing computational scripts to these processes during data processing. All data storage and computation are based on a widely used standard scripting language, offering advantages such as ease of use, ease of learning, and versatility.

[0120] In a further embodiment of the present invention, as shown in Figure 4, a data interaction front end 104 is also included. The data interaction front end 104 is used for user operations, such as issuing instructions such as viewing, modifying, adding, and deleting data. Instructions such as task preparation sequence, data acquisition instructions, calculation instructions, and output instructions can also be issued. Data objects within the user's authority range can also be displayed, including data objects in the user's stored data and data objects in the experimental process. The task management system can also respond to the experiment completion instruction and terminate the calculation process. Since the data calculation system is asynchronous processing, the data interaction front end can serve multiple experimental tasks at the same time, but the calculation process of each experimental task is carried out synchronously, and the data of all experimental tasks are executed and fed back in sequence in the form of a queue.

[0121] When implementing this, you can use a series of thread concurrency tools provided by java.util.concurrent in JDK8:

[0122] A) Containerization technology uses a thread-safe ConcurrentHashMap to place each task of each user and creates a separate websocket connection to the jupyter notebooks.

[0123] B) Use PriorityBlockingQueue (blocking priority queue) to process Python script tasks in sequence.

[0124] Data persistence is achieved using a Redis non-relational database and a MySQL relational database. Jupyter notebooks are deployed in a containerized manner based on Kubernetes, ensuring flexible use and cluster reliability.

[0125] In addition to being able to automatically acquire and process experimental data, this embodiment also allows users to monitor and intervene in all data operations, thereby improving user experience.

[0126] In one embodiment of the present invention, a method for processing experimental data applied to a task management system is provided, as shown in FIG5 , including:

[0127] Step 501 : Sending a task preparation sequence instruction to a data computing system, so that the data computing system creates a task-specific computing process according to the task preparation sequence instruction, creates a data object in the computing process, and assigns an initial value.

[0128] The task preparation sequence instructions in this step include: experimental task creation instructions, data object creation instructions and data input instructions.

[0129] The experimental task creation instruction allows the data computing system to establish a task-specific computing process. The experimental task creation instruction includes user information and task information. User information uniquely identifies the user and allows the experimental task data to be associated with the user process. Task information, including the task name and identification information, uniquely identifies the experimental task. In specific implementations, in addition to this information, the Python library is also imported.

[0130] The data object creation instruction allows the data computing system to create a data object and assign initial values ​​during the computing process. The data object creation instruction includes the variable name, type, size, and value. The variable type can be a Python list or array, typically a personalized JSON string. The variable value is specified in the data object creation instruction. The size constrains the array to a two-dimensional array, for example, 3x4.

[0131] The data input instruction is used to create and assign data objects to the computing process based on user stored data. The data input instruction includes: variable name, type, size, and object name / ID in the user stored data.

[0132] Step 502 sends a data acquisition instruction to the device control terminal. The receiving device control terminal operates the experimental equipment and obtains operating data. Based on the obtained operating data, a data modification instruction is generated and sent to the data computing system, so that the data computing system can assign values ​​to data objects in the relevant task computing process. The operating data refers to the data of all variables in the computing process.

[0133] Step 503: Send a data calculation instruction to the data calculation system, so that the data calculation system calculates the data in the relevant task calculation process according to the data calculation instruction.

[0134] The data calculation instruction is a Python script code. In some implementations, the script code is an expression that can correspond to a calculated value. This value can be a data type or a judgment result of true or false.

[0135] Step 504 : Send a data output instruction to the data computing system, so that the data computing system stores the data of the related computing process into the data object database according to the data output instruction.

[0136] Data output instructions include the variable name, type, size, and the object name / ID in the user stored data. The variable name identifies the variable to be output, and the type distinguishes the variable's input and output types. The object name / ID in the user stored data is used to create a data object in the user stored data and assign a value based on the variable's value.

[0137] Furthermore, it also includes:

[0138] Receive a data query request from a user for a target task within the scope of authority, and send the data query instruction to the data computing system so that the computing process of the target task queries and returns data from the object database according to the data query instruction.

[0139] During specific implementation, the user's data query request is also recorded to facilitate subsequent tracking of user information and ensure data security.

[0140] In one embodiment of the present invention, as shown in FIG6 , a method for processing experimental data applied to a data computing system is also provided, including:

[0141] Step 601: Receive a task preparation sequence instruction sent by the task management system, start a task-related computing process according to the task preparation sequence instruction, create a data object in the computing process and assign an initial value.

[0142] The task preparation sequence instructions include: experimental task creation instructions, data object creation instructions, and data input instructions. The specific implementation process of this step includes: receiving the experimental task creation instruction sent by the task management system, establishing a task-specific computing process according to the experimental task creation instruction; receiving the data object creation instruction sent by the task management system, using the object creation script to create a data object in the relevant computing process according to the data object creation instruction; receiving the data input instruction sent by the task management system, and using the second input script to assign the user-given stored data to the data object of the computing process according to the data input instruction.

[0143] Step 602: Receive a data modification instruction sent by the task management system, and assign values ​​to data objects in the relevant task computing process using an input script according to the data modification instruction.

[0144] Step 603: Receive the data calculation instruction sent by the task management system, push the calculation script to the calculation process of the relevant task according to the data calculation instruction, and implement data calculation through the calculation script.

[0145] Step 604: Receive a data output instruction sent by the task management system, send a query script to the computing process of the relevant task according to the data output instruction, and store the data retrieved by the query script in the data object database.

[0146] Furthermore, it also includes:

[0147] Receive data interaction instructions sent by the data interaction front end, and send the data interaction instructions to the computing process of the relevant task so that the computing process can realize the interaction.

[0148] Among them, the data interaction front end displays an interactive interface, which displays data objects within the user's authority and interactive controls of data objects. Data interaction instructions are issued through the interactive controls of data objects. Data interaction instructions include but are not limited to data viewing, modification, addition, and deletion instructions.

[0149] Based on the same inventive concept, this article also provides an experimental data processing device, as described in the following embodiments. Since the principles of the experimental data processing device for solving problems are similar to those of the experimental data processing method, the implementation of the experimental data processing device can refer to the experimental data processing method, and the repeated parts will not be repeated.

[0150] Specifically, as shown in FIG7 , the experimental data processing device applied to the task management system side includes:

[0151] The task creation control unit 701 is used to send a task preparation sequence instruction to the data computing system, so that the data computing system creates a task-specific computing process according to the task preparation sequence instruction, creates a data object in the computing process and assigns an initial value;

[0152] The data acquisition control unit 702 is used to send data acquisition instructions to the device control terminal, receive data acquired by the device control terminal operating the experimental equipment, and generate data modification instructions based on the acquired data to the data computing system so that the data computing system can assign values ​​to data objects in the relevant task computing process;

[0153] The data calculation control unit 703 is used to send a data calculation instruction to the data calculation system, so that the data calculation system calculates the data in the relevant task calculation process according to the data calculation instruction;

[0154] The data output control unit 704 is configured to send a data output instruction to the data computing system, so that the data computing system stores the data of the related computing process into the data object database according to the data output instruction.

[0155] As shown in Figure 8, the experimental data processing device applied to the data computing system side includes:

[0156] The task creation unit 801 is configured to receive a task preparation sequence instruction sent by the task management system, start a task-related computing process according to the task preparation sequence instruction, create a data object in the computing process, and assign an initial value;

[0157] The data modification unit 802 is configured to receive a data modification instruction sent by the task management system, and assign values ​​to data objects in the relevant task computing process using the first input script according to the data modification instruction;

[0158] The data calculation unit 803 is used to receive the calculation instructions sent by the task management system, push the calculation script to the calculation process of the relevant task according to the calculation instruction, and realize data calculation through the calculation script;

[0159] The data output unit 804 is used to receive a data output instruction sent by the task management system, send a query script to the computing process of the relevant task according to the data output instruction, and store the data queried by the query script in the data object database.

[0160] This embodiment implements the entire business process from data collection, on-site processing, storage, and user interaction by developing a computing process based on a universal scripting language. All data storage and calculations are based on a standard scripting language with a broad user base, offering advantages such as ease of use, ease of learning, and versatility.

[0161] The experimental data processing method, device and system provided in this paper support script requests for multiple users and multiple experimental tasks, and use the factory mode and containerization idea to isolate and manage each experimental task, assigning each experimental task to an independent

[0162] The system uses a websocket connection and a blocking priority queue to control the sequential execution of each script within a task. Based on object-oriented principles, it supports user-defined data objects to meet the data needs of various biological experiments, ensuring the data computation requirements of the biological experiment process and improving experimental flexibility. The data generated by each experimental task is cached or permanently stored, allowing users to quickly download and view their data in real time and visualize the Python script execution process in Jupyter notebooks.

[0163] In one embodiment, a computer device is provided. As shown in FIG9 , computer device 902 may include one or more processors 904, such as one or more central processing units (CPUs), each of which may implement one or more hardware threads. Computer device 902 may also include any memory 906 for storing any type of information, such as code, settings, data, and the like. For example, and without limitation, memory 906 may include any one or more combinations of the following: any type of RAM, any type of ROM, a flash memory device, a hard disk, an optical disk, and the like. More generally, any memory may use any technology to store information. Furthermore, any memory may provide volatile or non-volatile retention of information. Furthermore, any memory may represent a fixed or removable component of computer device 902. In one embodiment, when processor 904 executes an associated request stored in any memory or combination of memories, computer device 902 may perform any operation associated with the associated request. Computer device 902 also includes one or more drive mechanisms 908, such as a hard disk drive mechanism, an optical disk drive mechanism, and the like, for interacting with any memory.

[0164] The computer device 902 may also include an input / output module 910 (I / O) for receiving various inputs (via input devices 912) and for providing various outputs (via output devices 914). A specific output mechanism may include a presentation device 916 and an associated graphical user interface (GUI) 918. In other embodiments, the input / output module 910 (I / O), input devices 912, and output devices 914 may not be included, and the computer device 902 may simply be a computer device in a network. The computer device 902 may also include one or more network interfaces 920 for exchanging data with other devices via one or more communication links 922. One or more communication buses 924 couple the components described above together.

[0165] The communication link 922 may be implemented in any manner, for example, via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 922 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0166] The embodiments of the present invention further provide a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above method are executed.

[0167] The embodiments of this document also provide a computer-readable request, wherein when a processor executes the request, the program therein causes the processor to execute the method of any of the aforementioned embodiments.

[0168] It should be understood that in the various embodiments of this document, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.

[0169] It should also be understood that in the embodiments herein, the term "and / or" merely describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0170] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this document.

[0171] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0172] In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices, or units, or can be an electrical, mechanical, or other form of connection.

[0173] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments herein.

[0174] In addition, the functional units in the various embodiments herein may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0175] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several requests for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0176] This article uses specific embodiments to illustrate the principles and implementation methods of this article. The description of the above embodiments is only used to help understand the methods and core ideas of this article. At the same time, for those skilled in the art, based on the ideas of this article, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation to this article.

Claims

1. An experimental data processing method, characterized in that: Applied to task management systems, including: Sending a task preparation sequence instruction to a data computing system, so that the data computing system creates a task-specific computing process according to the task preparation sequence instruction, creates a data object in the computing process and assigns an initial value; Sending data acquisition instructions to the device control end, receiving the device control end to operate the experimental equipment and obtain operation data; generating data modification instructions to the data computing system according to the obtained operation data, so that the data computing system can assign values ​​to data objects in the relevant task calculation process; Sending a data calculation instruction to the data calculation system, so that the data calculation system calculates the data in the relevant task calculation process according to the data calculation instruction; A data output instruction is sent to the data computing system, so that the data computing system stores the data of the related computing process into the data object database according to the data output instruction.

2. The method according to claim 1, characterized in that The task preparation sequence instructions include: experimental task creation instructions, data object creation instructions and data input instructions; The experimental task creation instruction is used for the data computing system to establish a task-specific computing process; The data object creation instruction is used for the data computing system to create a data object and assign an initial value in the computing process; The data input instruction is used to create and assign data objects to the computing process based on user stored data.

3. The method according to claim 1, characterized in that Also includes: A data query instruction for a target task within the scope of authority is received from a user, and the data query instruction is sent to the data computing system, so that the computing process of the target task queries and returns data from a data object database according to the data query instruction.

4. An experimental data processing method, characterized in that: Applied to data computing systems, including: Receive a task preparation sequence instruction sent by the task management system, start a task-related computing process according to the task preparation sequence instruction, create a data object in the computing process and assign an initial value; Receiving a data modification instruction sent by the task management system, and assigning a value to a data object in a related task computing process using a first input script according to the data modification instruction; Receive the calculation instructions sent by the task management system, push the calculation script to the calculation process of the relevant task according to the calculation instructions, and realize data calculation through the calculation script; Receive a data output instruction sent by the task management system, send a query script to the computing process of the relevant task according to the data output instruction, and store the data queried by the query script in the data object database.

5. The method according to claim 4, characterized in that The task preparation sequence instructions include: experimental task creation instructions, data object creation instructions and data input instructions; Receiving a task preparation sequence instruction sent by a task management system, starting a task-related computing process according to the task preparation sequence instruction, creating a data object in the computing process and assigning an initial value includes: Receiving the experimental task creation instruction sent by the task management system, and establishing a task-specific computing process according to the experimental task creation instruction; Receiving a data object creation instruction sent by the task management system, and creating a data object in a related computing process using an object creation script according to the data object creation instruction; A data input instruction sent by the task management system is received, and according to the data input instruction, the user-given storage data is assigned to the data object of the computing process using a second input script.

6. The method according to claim 5, characterized in that After establishing a task-specific computing process according to the experimental task creation instruction, the method further includes: Establish and record the websocket address of the computing process; Connect the websocket address of the computing process to the jupyter notebook server process of the user to which the relevant experimental task belongs, so that the computing process connected to it can be accessed through the websocket address in the jupyter notebook server process; Place the connection relationship between the user's jupyter notebook server process and the websocket address of the computing process in ConcurrentHashMap.

7. The method according to claim 5, characterized in that Also includes: Receiving a data query instruction for a target task sent by the task management system; According to the data query instruction, a query script is sent to the computing process of the target task, so that the computing process of the target task queries data from the data object database according to the query script and returns the data to the task management system.

8. The method according to claim 5, characterized in that Also includes: Receiving a data interaction instruction sent by a data interaction front end, wherein the data interaction front end displays an interaction interface, wherein the interaction interface displays data objects within the user's authority and interaction controls of the data objects, and issuing the data interaction instruction through the interaction controls of the data objects; The data interaction instruction is sent to the computing process of the relevant task so that the computing process can realize the interaction.

9. An experimental data processing system, characterized in that: include: Task management system, data computing system and data object database; The task management system is used to send task preparation sequence instructions to the data computing system; Send data acquisition instructions to the device control end, receive the data obtained by operating the experimental equipment by the device control end; generate data based on the obtained data Modify instructions to the data computing system; send data computing instructions to the data computing system; send data output instructions to the data computing system; The data computing system is used to start the task-related computing process according to the task preparation sequence instructions, create data objects in the computing process and assign initial values; assign values ​​to data objects in the related task computing process according to the data modification instructions; calculate the data in the related task computing process according to the data calculation instructions; and store the data of the related computing process in the data object database according to the data output instructions.

10. The system according to claim 9, characterized in that The data object database also stores a data dictionary and a data display method; The data dictionary is used to describe the storage and parsing methods of various types of data objects, and is called when storing and reading data; The data display method is used to describe the display method of the data interaction front end for the required display data, and is called when the data is read.

11. An experimental data processing device, characterized in that: Applied to task management systems, including: The task creation control unit is used to send a task preparation sequence instruction to the data computing system, so that the data computing system creates a task-specific computing process according to the task preparation sequence instruction, creates a data object in the computing process and assigns an initial value; The data acquisition control unit is used to send data acquisition instructions to the device control terminal, receive data acquired by the device control terminal operating the experimental equipment; generate data modification instructions to the data computing system according to the acquired data, so that the data computing system can assign values ​​to data objects in the relevant task computing process; A data calculation control unit, used for sending a data calculation instruction to the data calculation system, so that the data calculation system calculates the data in the relevant task calculation process according to the data calculation instruction; The data output control unit is used to send a data output instruction to the data computing system, so that the data computing system stores the data of the relevant computing process into the data object database according to the data output instruction.

12. An experimental data processing device, characterized in that: Applied to data computing systems, including: The task creation unit is used to receive the task preparation sequence instruction sent by the task management system, start the task-related computing process according to the task preparation sequence instruction, create a data object in the computing process and assign an initial value; A data modification unit, configured to receive a data modification instruction sent by the task management system, and assign values ​​to data objects in a related task computing process using a first input script according to the data modification instruction; A data calculation unit is used to receive a data calculation instruction sent by the task management system, push a calculation script to the calculation process of the relevant task according to the data calculation instruction, and realize data calculation through the calculation script; The data output unit is used to receive the data output instruction sent by the task management system, send the query script to the computing process of the relevant task according to the data output instruction, and store the data queried by the query script in the data object database.

13. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

14. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor of a computer device, the method according to any one of claims 1 to 8 is implemented.

Images