Real-time visual remote microscopic imaging experiment system

Abstract

The invention discloses a real-time visual remote microscopic imaging experiment system, which belongs to the field of digitization capable of remote coordination and comprises a reservation management module, a sample logistics tracking module, an augmented reality interaction module, a remote control module, a cloud storage module and an equipment management module. According to the invention, remote reservation, operation and data management functions are integrated, geographical restrictions are broken, sharing of high-value microscopic imaging instruments by more users becomes possible, and the use efficiency of the instruments is maximized; according to the invention, through linkage of the AR interaction module and the remote control module, real-time cooperation between a user and an instrument operator is supported, and through combination of voice communication and visual annotation, virtual on-site experiment control experience is realized.

Description

Technical Field

[0001] The present invention belongs to the field of remote collaborative digital technology, and in particular relates to a real-time visual remote microscopy imaging experiment system. Background Art

[0002] At present, there are certain limitations in the configuration and use of scientific research instruments in domestic universities, research institutes and some enterprises. Due to resource constraints, many units are unable to equip themselves with instruments and equipment that meet all scientific research needs, so they often need to borrow scientific research instruments from external units. The operation of some instruments is relatively simple, mainly completed through computer software connected to the instrument, and samples can be transported over long distances. The sample preparation process before loading is relatively simple, and usually only the sample needs to be placed in the instrument to complete the preparation work. In addition, some instrument units provide external sharing services, allowing users to send samples for testing to meet the needs of different researchers.

[0003] However, the existing model of instrument sharing and use has many shortcomings. On the one hand, scientific researchers have heavy research tasks and tight time constraints. If the borrowed instrument is far away, most of the time will be wasted on the round trip, resulting in a significant increase in time and labor costs. On the other hand, although some instrument units provide sample delivery and testing services, they are often unable to accurately understand the user's experimental purpose and find it difficult to prepare samples that meet the user's needs, resulting in a large deviation between the collected experimental data and the user's expectations. In this case, the user cannot control the entire experimental process as if they were operating the instrument on site, which not only affects the experimental efficiency, but also reduces the utilization rate of the instrument. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention proposes a real-time visual remote microscopy imaging experiment system to solve the problems existing in the above prior art.

[0005] To achieve the above objectives, the present invention provides a real-time visual remote microscopy imaging experimental system, comprising:

[0006] The reservation management module is used to receive the instrument use reservation request input by the user and communicate with the instrument operator to confirm the reservation time and sample information;

[0007] The sample logistics tracking module is connected to the logistics information system to monitor the transportation status and receipt information of user mailed samples;

[0008] The augmented reality interaction module includes AR glasses worn by the instrument operator, a real-time video transmission unit, and a labeling feedback unit. The AR glasses are used to collect visual data of sample preparation and on-machine operation. The real-time video transmission unit sends the visual data to the user end. The labeling feedback unit receives the labeling instructions from the user end and synchronously displays them on the AR glasses interface.

[0009] A remote control module, communicating with a computer connected to the microscopic imaging instrument, includes a remote desktop access unit and an instrument control interface unit. The remote desktop access unit enables a user to remotely operate the microscopic imaging software. The instrument control interface unit is used to adjust parameters of the microscopic imaging instrument and trigger data acquisition and analysis processing.

[0010] The cloud storage module communicates with the computer connected to the microscopic imaging instrument and the user terminal to receive experimental data and store it in the network storage space specified by the user;

[0011] The equipment management module is used by instrument operators to control the power on and off process of microscopic imaging instruments and record instrument usage logs and operator information.

[0012] Preferably, the reservation management module includes:

[0013] The reservation information input unit is used to receive the sample attributes, experimental purpose and time preference submitted by the user;

[0014] The reservation confirmation unit interacts with the instrument operator end, generates a reservation success notification and synchronizes it to the user end and the instrument operator end.

[0015] Preferably, the sample logistics tracking module includes:

[0016] Logistics status query interface, connected to the third-party logistics platform API, used to obtain sample transportation trajectory in real time;

[0017] The receipt feedback unit is used to send confirmation information to the user end and the instrument operator end after the sample is delivered.

[0018] Preferably, the augmented reality interaction module includes:

[0019] The real-time video transmission unit uses a low-latency encoding protocol to ensure synchronization of visual data between the user end and the AR glasses;

[0020] The annotation feedback unit supports users to draw marked areas in real-time video and highlight the operation steps through the display interface of AR glasses.

[0021] Preferably, the low-latency encoding protocol is WebRTC or H.265 protocol, and the annotation feedback unit supports multi-layer overlay display, and the annotation content of the user terminal is rendered in layers with the original picture of the AR glasses.

[0022] Preferably, the instrument control interface unit includes a focus adjustment submodule, a light source intensity control submodule and an image resolution setting submodule, and the focus adjustment submodule, light source intensity control submodule and image resolution setting submodule directly interact with the hardware driver of the microscopic imaging instrument.

[0023] Preferably, the focus adjustment submodule drives the objective lens of the microscopic imaging instrument to move via a stepping motor, and the light source intensity control submodule integrates a PID algorithm to stabilize the light output.

[0024] Preferably, the cloud storage module includes:

[0025] Data encryption unit, used to protect uploaded experimental data using end-to-end encryption protocol;

[0026] The storage path allocation unit is used to store data in a designated directory according to user account information.

[0027] Preferably, the end-to-end encryption protocol is AES-256, and the storage path allocation unit is directly connected to the API interface of the user's network disk account to achieve data upload without human intervention.

[0028] Preferably, the device management module includes:

[0029] A shutdown process control unit, used to shut down the power supply and auxiliary equipment of the microscopic imaging instrument in a preset order;

[0030] The log generation unit is used to record the instrument usage time, operator identity and experimental data file identification.

[0031] Compared with the prior art, the present invention has the following advantages and technical effects:

[0032] The present invention provides a real-time visual remote microscopy imaging experiment system, comprising: an appointment management module for receiving an instrument use appointment request input by a user and communicating with the instrument operator to confirm the appointment time and sample information; a sample logistics tracking module, connected to the logistics information system, for monitoring the transportation status and receipt information of the user's mailed samples; an augmented reality interaction module, comprising AR glasses worn by the instrument operator, a real-time video transmission unit and a marking feedback unit, wherein the AR glasses are used to collect visual data of sample preparation and on-machine operation, the real-time video transmission unit sends the visual data to the user, and the marking feedback unit receives the user's The marking instructions of the end are synchronously displayed on the AR glasses interface; the remote control module communicates with the computer connected to the microscopic imaging instrument, including a remote desktop access unit and an instrument control interface unit. The remote desktop access unit enables the user end to remotely operate the microscopic imaging software, and the instrument control interface unit is used to adjust the parameters of the microscopic imaging instrument and trigger data acquisition and analysis processing; the cloud storage module communicates with the computer connected to the microscopic imaging instrument and the user end, and is used to receive experimental data and store it in the network storage space specified by the user; the equipment management module is used for the instrument operator to control the power on and off process of the microscopic imaging instrument, and record the instrument usage log and operator information.

[0033] The present invention optimizes instrument usage time planning and sample transportation monitoring through the collaboration of the appointment management module and the sample logistics tracking module, reduces waiting time in the experiment preparation stage, ensures seamless connection of the experiment process, and improves the efficiency of remote experiments.

[0034] With the help of the real-time video transmission and annotation feedback functions of the augmented reality (AR) interaction module, users can intuitively guide instrument operators to complete sample preparation and machine operation, avoiding misoperation caused by information asymmetry, achieving precise remote guidance, and significantly improving the success rate of experiments.

[0035] The present invention directly operates the parameter adjustment and data acquisition process of the microscopic imaging instrument through a remote control module, allowing users to complete complex experiments without having to visit the site in person, breaking through physical distance limitations and saving time and travel costs.

[0036] The cloud storage module in the present invention adopts end-to-end encryption protocol and directional storage mechanism to ensure the security of experimental data during transmission and storage, while supporting users to download data from the network disk at any time, improving the convenience of data management.

[0037] The device management module in the present invention reduces the risk of human operational errors, extends the service life of the instrument, and enhances the standardization of equipment management through preset shutdown processes and automated log recording functions, providing traceable operation records for subsequent experiments.

[0038] The system provided by this invention integrates remote appointment, operation, and data management functions, breaking geographical restrictions and enabling more users to share high-value microscopy instruments, maximizing instrument efficiency. By linking an AR interaction module with a remote control module, this invention supports real-time collaboration between users and instrument operators, combining voice communication with visual annotation to achieve a "virtual presence" experimental control experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0040] Figure 1 Schematic diagram of a system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0042] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] Example 1

[0044] In response to the problems existing in the external sharing of scientific research instruments, "Scientific researchers now have heavy research tasks and tight time constraints. If the distance to borrow the instrument is far, most of the time will be spent on the journey, wasting time, manpower and costs; some instrument units also share externally and provide sample delivery and testing services, but they do not know how to prepare the tested samples or cannot accurately understand the user's experimental purpose, making it difficult to collect the experimental data the user wants." This embodiment provides a real-time visual remote microscopy imaging experiment system, which includes six modules: appointment management module, sample logistics tracking module, augmented reality interaction module, remote control module, cloud storage module, and equipment management module. Figure 1 As shown, specifically including:

[0045] The reservation management module is used to receive the instrument use reservation request input by the user and communicate with the instrument operator to confirm the reservation time and sample information;

[0046] Furthermore, the reservation management module includes:

[0047] The reservation information input unit is used to receive the sample attributes, experimental purpose and time preference submitted by the user;

[0048] The reservation confirmation unit interacts with the instrument operator end, generates a reservation success notification and synchronizes it to the user end and the instrument operator end.

[0049] The sample logistics tracking module is connected to the logistics information system to monitor the transportation status and receipt information of user mailed samples;

[0050] Furthermore, the sample logistics tracking module includes:

[0051] Logistics status query interface, connected to the third-party logistics platform API, used to obtain sample transportation trajectory in real time;

[0052] The receipt feedback unit is used to send confirmation information to the user end and the instrument operator end after the sample is delivered.

[0053] Specifically, the sample requirements are: suitable for conventional long-distance transportation, easy to store, and prepared or pre-processed and only require simple preparation before loading onto the machine.

[0054] The samples used in this example are 5-day-old Arabidopsis seedling plates labeled with red and green fluorescence.

[0055] The augmented reality interaction module includes AR glasses worn by the instrument operator, a real-time video transmission unit, and a labeling feedback unit. The AR glasses are used to collect visual data of sample preparation and on-machine operation. The real-time video transmission unit sends the visual data to the user end. The labeling feedback unit receives the labeling instructions from the user end and synchronously displays them on the AR glasses interface.

[0056] Furthermore, the augmented reality interaction module includes:

[0057] The real-time video transmission unit adopts a low-latency encoding protocol to ensure the synchronization of visual data between the user end and the AR glasses; wherein, the low-latency encoding protocol is WebRTC or H.265 protocol, and the annotation feedback unit supports multi-layer overlay display, and the annotation content of the user end is rendered in layers with the original picture of the AR glasses.

[0058] The annotation feedback unit supports users to draw marked areas in real-time video and highlight the operation steps through the display interface of AR glasses.

[0059] Specifically, the instrument operator turns on the machine, connects to the Internet, wears AR glasses, and connects to the user's AR corresponding software. Under the user's guidance, the seedlings are placed on a glass slide with sterilized water, covered with a coverslip, and then the prepared sample is placed on a laser confocal microscope.

[0060] In this embodiment, the user installs VCR control software and connects it to the VCR worn by the instrument operator. The user can then watch the operator's every step of the operation (including but not limited to sample preparation and sample loading) through video, take screenshots and edit the video, and transmit it to the operator in real time. The user can also communicate in real time via voice. This allows the user to fully control the entire experimental process in real time, as if they were actually operating the experiment.

[0061] A remote control module, communicating with a computer connected to the microscopic imaging instrument, includes a remote desktop access unit and an instrument control interface unit. The remote desktop access unit enables a user to remotely operate the microscopic imaging software. The instrument control interface unit is used to adjust parameters of the microscopic imaging instrument and trigger data acquisition.

[0062] Furthermore, the instrument control interface unit includes a focus adjustment submodule, a light source intensity control submodule and an image resolution setting submodule, and the focus adjustment submodule, light source intensity control submodule and image resolution setting submodule directly interact with the hardware driver of the microscopic imaging instrument.

[0063] The focus adjustment submodule drives the objective lens of the microscope to move via a stepping motor, and the light source intensity control submodule integrates a PID algorithm to stabilize the light output.

[0064] Specifically, the computer screen connected to the instrument is equipped with operating software and a network drive for controlling the instrument and analyzing data, along with remote control software. The user then installs the corresponding remote control software on their own computer and connects the two remote control software systems via a wireless network. Once the instrument operator loads the prepared sample onto the instrument, the user can use the remote control software to operate the instrument control software, locate the sample, adjust parameters, collect and save data, and then analyze the data.

[0065] The cloud storage module communicates with the computer connected to the microscopic imaging instrument and the user terminal to receive experimental data and store it in the network storage space specified by the user;

[0066] Furthermore, the cloud storage module includes:

[0067] Data encryption unit, used to protect uploaded experimental data using end-to-end encryption protocol;

[0068] The storage path allocation unit is used to store data in a designated directory according to user account information.

[0069] Furthermore, the end-to-end encryption protocol is AES-256, and the storage path allocation unit is directly connected to the API interface of the user's network disk account to achieve data upload without human intervention.

[0070] Specifically, the user logs in to his own network disk (such as Baidu Netdisk, etc.) on the computer connected to the instrument, and then uploads the experimental data to his own network disk; finally, the user logs in to the network disk on his own computer, and then downloads the experimental data from the network disk to his own computer.

[0071] The equipment management module is used to control the power on and off process of the microscope and record the instrument usage log and operator information.

[0072] Furthermore, the device management module includes:

[0073] A shutdown process control unit, used to shut down the power supply and auxiliary equipment of the microscopic imaging instrument in a preset order;

[0074] The log generation unit is used to record the instrument usage time, operator identity and experimental data file identification.

[0075] Specifically, after the user completes uploading the experimental data, the instrument operator removes the sample from the instrument, then turns off the switch and power supply of the instrument according to the shutdown procedure, registers it in the instrument equipment usage record book, and cleans up the instrument table.

[0076] Beneficial effects of this embodiment:

[0077] In the system provided in this embodiment, the user only needs to mail the sample to the instrument operator. The instrument operator wears AR glasses and connects the AR glasses and the instrument to the user's computer via the network. The user can then provide remote guidance and control the instrument under real-time visualization.

[0078] This embodiment not only enables remote point-to-point guidance for sample preparation and instrumentation, but also remote control and instrument operation, resolving the issue of users being unable to access instruments due to distance, thus reducing travel time. This embodiment not only monitors the entire experimental process, enabling timely and effective communication with on-site technicians, but also enables timely acquisition of experimental data, improving experimental efficiency and instrument utilization, effectively ensuring experimental success and enabling remote experimentation. For fully electric microscopes equipped with a computer, this embodiment uses the microscope imaging data acquisition software on the computer to control the microscope's adjustment and imaging.

[0079] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A real-time visual remote microscopy imaging experimental system, characterized in that: include: The reservation management module is used to receive the instrument use reservation request input by the user and communicate with the instrument operator to confirm the reservation time and sample information; The sample logistics tracking module is connected to the logistics information system to monitor the transportation status and receipt information of user mailed samples; The augmented reality interaction module includes AR glasses worn by the instrument operator, a real-time video transmission unit, and a labeling feedback unit. The AR glasses are used to collect visual data of sample preparation and on-machine operation. The real-time video transmission unit sends the visual data to the user end. The labeling feedback unit receives the labeling instructions from the user end and synchronously displays them on the AR glasses interface. A remote control module, communicating with a computer connected to the microscopic imaging instrument, includes a remote desktop access unit and an instrument control interface unit. The remote desktop access unit enables a user to remotely operate the microscopic imaging software. The instrument control interface unit is used to adjust parameters of the microscopic imaging instrument and trigger data acquisition and analysis by operating the microscopic imaging software. The cloud storage module communicates with the computer connected to the microscopic imaging instrument and the user terminal to receive experimental data and store it in the network storage space specified by the user; The equipment management module is used by instrument operators to control the power on and off process of microscopic imaging instruments and record instrument usage logs and operator information.

2. The system according to claim 1, wherein: The reservation management module includes: The reservation information input unit is used to receive the sample attributes, experimental purpose and time preference submitted by the user; The reservation confirmation unit interacts with the instrument operator end, generates a reservation success notification and synchronizes it to the user end and the instrument operator end.

3. The system according to claim 1, wherein: The sample logistics tracking module includes: Logistics status query interface, connected to the third-party logistics platform API, used to obtain sample transportation trajectory in real time; The receipt feedback unit is used to send confirmation information to the user end and the instrument operator end after the sample is delivered.

4. The system according to claim 1, wherein: The augmented reality interaction module includes: The real-time video transmission unit uses a low-latency encoding protocol to ensure synchronization of visual data between the user end and the AR glasses; The annotation feedback unit supports users to draw marked areas in real-time video and highlight the operation steps through the display interface of AR glasses.

5. The system according to claim 4, characterized in that The low-latency encoding protocol is WebRTC or H.265 protocol, and the annotation feedback unit supports multi-layer overlay display, and the annotation content on the user side is rendered in layers with the original picture of the AR glasses.

6. The system according to claim 1, wherein: The instrument control interface unit includes a focus adjustment submodule, a light source intensity control submodule and an image resolution setting submodule, which directly interact with the hardware driver of the microscopic imaging instrument.

7. The system according to claim 6, characterized in that The focus adjustment submodule drives the objective lens of the microscopic imaging instrument to move via a stepping motor, and the light source intensity control submodule integrates a PID algorithm to stabilize the light output.

8. The system according to claim 1, wherein: The cloud storage module includes: Data encryption unit, used to protect uploaded experimental data using end-to-end encryption protocol; The storage path allocation unit is used to store data in a designated directory according to user account information.

9. The system according to claim 8, characterized in that The end-to-end encryption protocol is AES-256, and the storage path allocation unit is directly connected to the API interface of the user's network disk account to achieve data upload without human intervention.

10. The system according to claim 1, wherein: The device management module includes: A shutdown process control unit, used to shut down the power supply and auxiliary equipment of the microscopic imaging instrument in a preset order; The log generation unit is used to record the instrument usage time, operator identity and experimental data file identification.

Images