Deep fusion unattended intelligent experiment system

Abstract

The invention discloses a deep fusion unattended intelligent experiment system. The experiment system comprises a microscope heating stage base structure, an XY moving motor, a CCD image acquisition module and a control module. The microscope heating stage base structure is used for placing a loading area of a sample; two groups of high-precision XY moving motors are arranged in the horizontal plane direction of the microscope heating stage and are respectively responsible for the movement of the heating stage in the horizontal transverse X-axis direction and the horizontal longitudinal Y-axis direction; the CCD image acquisition module is used for acquiring microscopic images of a sample in real time; and the control module is connected with the CCD image acquisition module and is used for controlling experiment operation according to the microscopic image. And full-process automatic cycle operation is realized.

Description

Technical Field

[0001] The present invention relates to the field of microscopic temperature-variable observation experimental operation, and in particular to a deeply integrated unattended intelligent experimental system. Background Art

[0002] Under current technological conditions, microscopic temperature-variable observation experiments rely primarily on manual operation: the experimenter must manually adjust the microscope's optical parameters, simultaneously set the temperature gradient of the temperature stage, the heating and cooling rate, and other control parameters, conduct observation tests on each sample one by one, and manually record the experimental data. This traditional operation process is not only labor-intensive and time-consuming, making it difficult to meet the needs of high-throughput sample testing, but also due to differences in operator technique and recording errors, the repeatability and consistency of experimental data are poor, making it impossible to guarantee the accuracy and reliability of experimental results.

[0003] Existing technologies have significant flaws in the operation of microscopic temperature-variable observation experiments. These shortcomings are mainly reflected in two aspects: First, high labor costs and low efficiency. A large number of samples require manual microscope adjustment, temperature stage parameter setting, observation and testing, and data recording. This process is cumbersome and time-consuming, making it difficult to meet the needs of high-throughput testing. Second, experimental data has poor reproducibility and consistency. During manual operation, different experimenters have different operating techniques, observation and judgment standards, and recording methods, which can easily introduce human errors, resulting in insufficient accuracy and reliability of experimental results.

[0004] These shortcomings are primarily due to an over-reliance on manual operation. Specifically, manual adjustment of microscope and temperature stage parameters, lacking the assistance of automated equipment, prevents the rapid and precise unified setup and processing of multiple samples. Furthermore, manual data recording lacks standardized processes and automated verification mechanisms, leading to a high degree of arbitrariness in data recording and difficulty ensuring data integrity and accuracy. Under manual operation, it is difficult to eliminate interference caused by individual operational differences, which affects the repeatability and consistency of experimental data. Summary of the Invention

[0005] In view of the above problems, the present invention is proposed to provide a deep fusion unattended intelligent experiment system that overcomes the above problems or at least partially solves the above problems.

[0006] According to one aspect of the present invention, a deep fusion unattended intelligent experimental system is provided, the experimental system comprising: a microscope hot stage infrastructure, an XY moving motor, a CCD image acquisition module and a control module;

[0007] The microscope hot stage basic structure is used to place the sample loading area;

[0008] Two sets of high-precision XY moving motors are installed in the horizontal plane direction of the microscope hot stage, which are responsible for the movement of the hot stage in the horizontal horizontal X-axis and horizontal vertical Y-axis directions respectively;

[0009] The CCD image acquisition module is used to acquire microscopic images of samples in real time;

[0010] The control module is connected to the CCD image acquisition module and is used to control the experimental operation according to the microscopic image.

[0011] Optionally, the XY moving motor is driven by a stepper motor and is equipped with a precision screw transmission mechanism to achieve micron-level precision displacement control with a minimum moving step of 1 micron.

[0012] Electric focusing Z-axis robotic arm: In the vertical direction of the microscope, an electric focusing Z-axis robotic arm is added, and the robotic arm is connected to the objective lens adjustment mechanism of the microscope;

[0013] The robotic arm has a built-in servo motor and encoder, which adjusts the distance between the objective lens and the sample according to instructions to achieve an autofocus function.

[0014] Optionally, the CCD image acquisition module specifically includes:

[0015] A high-resolution CCD camera is installed above the eyepiece of the microscope to capture microscopic images of the sample in real time;

[0016] The CCD camera has automatic exposure and automatic white balance functions, and transmits the collected image data to the control module in real time.

[0017] Optionally, the control module specifically includes: a coordinate management unit, a temperature control calling unit, an image monitoring unit and a point switching unit.

[0018] Optionally, the coordinate management unit is used to record and manage XYZ three-dimensional coordinate information of each observation point;

[0019] The user sets the observation points by manually inputting them through the software interface or marking them in the microscope field of view, and the software automatically saves and generates a list of point coordinates.

[0020] Optionally, the temperature control calling unit specifically includes:

[0021] According to the preset observation process, the temperature control instructions of the temperature control system are automatically called to achieve precise control of the sample temperature, supporting multiple temperature control modes such as linear heating, cooling, and step-by-step temperature changes.

[0022] Optionally, the image monitoring unit specifically includes:

[0023] Communicate with the CCD image acquisition system in real time to receive and process the collected microscopic image data;

[0024] The image recognition algorithm is used to monitor the morphological changes of samples in real time. When key nodes of phase change and structural change of the sample are detected, the data recording function is automatically triggered.

[0025] Optionally, the point switching unit specifically includes:

[0026] After completing the measurement of a single observation point, the control command is automatically generated to drive the XY moving motor and the electric focusing Z-axis robotic arm to move the hot stage to the next observation point and adjust the focal length to the appropriate state.

[0027] The present invention provides a deep fusion unattended intelligent experimental system, which includes: a microscope hot stage basic structure, an XY moving motor, a CCD image acquisition module and a control module; the microscope hot stage basic structure is used to place the sample loading area; two sets of high-precision XY moving motors are installed in the horizontal plane direction of the microscope hot stage, which are respectively responsible for the movement of the hot stage in the horizontal transverse X-axis and the horizontal longitudinal Y-axis directions; the CCD image acquisition module is used to collect the microscopic image of the sample in real time; the control module is connected to the CCD image acquisition module and is used to control the experimental operation according to the microscopic image. The whole process is realized by automated cyclic operation. The core value of the artificial intelligence unattended intelligent microscopic temperature change system is to break the technical bottleneck of traditional research and production by means of automation and intelligence. Through precise temperature control and real-time in-situ microscopic observation, combined with in-depth analysis of massive data using AI algorithms, the system can achieve dynamic process visualization tracking, quantitative analysis of key parameters, and intelligent process optimization in multiple fields such as materials science, biomedicine, and drug development. This significantly improves research efficiency and the ability to transform research results, promotes the upgrade of the entire chain from basic scientific research to industrial production, and provides core technical support for new material development, drug innovation, and industrial intelligent transformation.

[0028] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 A structural diagram of a deep fusion unattended intelligent experiment system provided by an embodiment of the present invention;

[0031] Figure 2 A flowchart of automatic control of a deep fusion unattended intelligent experiment system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0033] The terms "comprises" and "comprising" and any variations thereof in the description, embodiments, claims and drawings of the present invention are intended to cover non-exclusive inclusions, for example, including a series of steps or units.

[0034] The technical solution of the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0035] This invention focuses on the microscope hot stage to build an AI artificial intelligence automated experimental system with deep integration of software and hardware.

[0036] By adding an XY motion motor for the hot stage and a motorized focusing Z-axis robotic arm, a precise three-dimensional automated motion system is created. The XY motion motor, like a flexible "two-dimensional dance" for the hot stage, enables horizontal, forward, backward, and left / right movement with micron-level precision. The motorized focusing Z-axis robotic arm, like a robotic hand with intelligent joints, uses AI image recognition algorithms to automatically adjust focus based on experimental needs, ensuring a consistently clear and sharp field of view. Working in tandem, these two systems completely eliminate the clumsy and inefficient traditional manual adjustments.

[0037] The integrated AI control program builds an intelligent hub for multi-module collaboration. It has powerful coordinate management capabilities and can accurately record the XYZ three-dimensional coordinate information of each observation point, like drawing a detailed "navigation map" for the microscopic world. Before the experiment begins, researchers only need to preset the observation points (such as point 1, point 2, point 3, etc.), and the program will automatically call the temperature control software in sequence to accurately control the temperature of the sample. At the same time, through the high-resolution CCD image acquisition system, the software captures sample changes in real time at a rate of dozens of frames per second, like a "electronic eye" on duty 24 hours a day. Once key nodes such as phase change and structural change in the sample are detected, the system will immediately lock and automatically save the temperature data of the change point. After completing the single-point measurement, the software and hardware systems are seamlessly connected, automatically driving the hot stage to move to the next observation point, and the entire process can be automated and cycled without human intervention. The core value of the artificial intelligence unmanned intelligent microscopy temperature change system lies in breaking the technical bottlenecks of traditional research and production through automated and intelligent means. Through precise temperature control and real-time in-situ microscopic observation, combined with in-depth analysis of massive data using AI algorithms, the system can achieve dynamic process visualization tracking, quantitative analysis of key parameters, and intelligent process optimization in multiple fields such as materials science, biomedicine, and drug development. This significantly improves research efficiency and the ability to transform research results, promotes the upgrade of the entire chain from basic scientific research to industrial production, and provides core technical support for new material development, drug innovation, and industrial intelligent transformation.

[0038] The innovative system, through the deep coupling of hardware automation and software intelligent control, not only increases experimental efficiency several times, but also fundamentally eliminates human operational errors, significantly enhances the repeatability and accuracy of experimental data, and brings revolutionary technological breakthroughs to the field of microscopic temperature-dependent observation.

[0039] Focusing on microscopic temperature-variable observation experiments, this paper systematically transforms existing microscope hot stages, constructing an experimental system that deeply integrates hardware and software. Through the coordinated operation of hardware automated motion systems and software intelligent control programs, this system automates the entire microscopic temperature-variable observation process, effectively addressing the high labor costs, low efficiency, and poor data reproducibility associated with existing technologies.

[0040] (1) Hardware structure

[0041] Microscope hot stage basic structure: The main frame of the existing microscope hot stage is used as the basic platform for sample loading and temperature control, and is provided with a loading area for placing samples.

[0042] XY Motion Motors: Two high-precision XY motion motors are installed in the horizontal plane of the microscope's heat stage, responsible for moving the heat stage in the X-axis (horizontal) and Y-axis (vertical) directions, respectively. These motors are driven by stepper motors and coupled with a precision lead screw drive mechanism, enabling micron-level displacement control, with a minimum step size of 1 micron.

[0043] Motorized Z-axis Focusing Arm: An additional motorized Z-axis focusing arm is installed vertically on the microscope, connected to the microscope's objective lens adjustment mechanism. Equipped with a built-in servo motor and high-precision encoder, the arm precisely adjusts the distance between the objective lens and the sample according to software instructions, enabling autofocus with a focus accuracy of up to 0.1 micron.

[0044] CCD Image Acquisition System: A high-resolution CCD camera is installed above the microscope's eyepiece to capture real-time microscopic images of the sample. The CCD camera features automatic exposure and white balance, and transmits captured image data to the software control system in real time.

[0045] Temperature Control System: The original microscope hot stage temperature control system is retained and upgraded, with a new programmable temperature controller. This controller allows for software-defined temperature settings and adjustments, with a temperature control range of -195°C to 600°C and a temperature control accuracy of ±0.1°C.

[0046] Control module: Self-developed AI artificial intelligence dedicated control program, including coordinate management unit, temperature control call unit, image monitoring unit and point switching unit.

[0047] Coordinate Management Unit: Used to record and manage the XYZ coordinates of each observation point. Users can manually enter observation points through the software interface or mark them in the microscope field of view. The program automatically saves and generates a list of point coordinates.

[0048] Temperature Control Calling Unit: Automatically calls the temperature control system's temperature control instructions based on the preset observation process to achieve precise control of the sample temperature. Supports multiple temperature control modes such as linear heating and cooling, and step-by-step temperature changes.

[0049] Image Monitoring Unit: Communicates in real time with the CCD image acquisition system to receive and process acquired microscopic image data. Using image recognition algorithms, it monitors sample morphological changes in real time and automatically triggers data recording when key points such as phase or structural changes are detected.

[0050] Point switching unit: After completing the measurement of a single observation point, it automatically generates control instructions to drive the XY moving motor and the electric focusing Z-axis robotic arm to move the hot stage to the next observation point and adjust the focal length to the appropriate state.

[0051] (3) Working principle

[0052] The working principle of the present invention is based on the coordination of hardware automated movement and software artificial intelligence control. Before the experiment begins, the user presets the observation points and temperature change parameters through the software interface. The coordinate management module of the software records the XYZ coordinates of each observation point, and the temperature control call module generates temperature control instructions according to the preset parameters and sends them to the temperature control system. During the experiment, the CCD image acquisition system collects sample images in real time and transmits them to the image monitoring module of the software. The module analyzes the image through an image recognition algorithm. Once a sample change is detected, the data recording function is immediately triggered to save the current temperature information. After completing the measurement of an observation point, the point switching module automatically generates a control instruction to drive the XY moving motor and the electric focusing Z-axis robot to move the hot stage to the next observation point, adjust the focal length, and continue to observe until the measurement of all preset points is completed.

[0053] (1) Hardware installation and connection

[0054] On the horizontal plane of the existing microscope hot stage, the XY motion motor and its transmission mechanism are fixed by high-precision guide rails and mounting brackets. Ensure that the motor is firmly mounted and that the connecting parts of the transmission screw and the hot stage have good coaxiality and straightness to ensure the accuracy and smoothness of the hot stage movement.

[0055] Mount the motorized focus Z-axis robotic arm to the microscope's objective lens holder. Connect the arm's drive shaft to the transmission component of the objective lens adjustment mechanism via a coupling. Precise calibration is required during installation to ensure the arm's movement is perpendicular to the microscope's optical axis to avoid focusing errors.

[0056] Connect the CCD image acquisition system to the microscope eyepiece via an interface adapter, ensuring that the camera's optical axis coincides with the microscope's optical axis to obtain a clear microscopic image. Simultaneously, connect the CCD camera to the computer's image acquisition card via a data cable to transmit image data.

[0057] The programmable temperature controller is connected to the heating / cooling components of the microscope hot stage and connected to the computer control program through the USB communication interface to realize the transmission and feedback of temperature control instructions.

[0058] (2) Software system setup and debugging

[0059] Install the self-developed dedicated control program into the computer system. After running the program, enter the hardware parameters of the microscope hot stage in the parameter setting interface, such as the step angle and lead screw pitch of the XY moving motor, the transmission ratio of the electric focusing Z-axis robot arm, etc., so that the software can accurately control the hardware movement.

[0060] In the Coordinate Management module, observation points are set by manual input or by marking them in the microscope field of view. For example, for a rectangular sample, nine observation points can be evenly selected on the sample surface (3×3 array), and the program automatically records the XYZ coordinates of each point.

[0061] In the temperature control call module, set the temperature change parameters, such as starting at 25°C, increasing the temperature to 100°C at a rate of 5°C / min, and maintaining it at 100°C for 5 minutes. Also select an appropriate temperature control mode, such as linear temperature rise mode.

[0062] Debug the image monitoring module and set the threshold parameters of the image recognition algorithm to ensure accurate detection of sample changes. For example, by adjusting the grayscale threshold and edge detection parameters, the system can identify characteristic changes such as phase change and crack growth in the sample.

[0063] (III) Experimental operation steps

[0064] Place the sample to be measured on the loading area of ​​the microscope hot stage and ensure that the sample is firmly fixed.

[0065] Turn on the computer, run the dedicated control program, select the experimental mode on the software interface, and enter the parameter setting interface.

[0066] Follow the above software system setting and debugging steps to complete the setting of observation points, temperature change parameters, etc.

[0067] Click the "Start Experiment" button on the software interface, and the system will automatically start the temperature control system to heat the sample. At the same time, the CCD image acquisition system will start to collect sample images in real time.

[0068] The image monitoring module of the software performs real-time analysis on the collected images and automatically records the current temperature information when changes in the sample are detected.

[0069] After completing the measurement of the first observation point, the point switching module automatically drives the XY moving motor and the electric focusing Z-axis robotic arm to move the hot stage to the next observation point, adjust the focal length, and continue observation.

[0070] Repeat steps 5 and 6 until all preset observation points are measured.

[0071] After the experiment is over, the system automatically stops temperature control and generates an experimental data report, which includes the coordinates of each observation point, sample temperature change, image data, etc.

[0072] Attachment Figure 1This diagram shows the hardware structure of the modified microscope hot stage. In the figure, 1 is the main frame of the microscope hot stage, 2 is the sample loading area, 3 is the X-axis motion motor, 4 is the Y-axis motion motor, 5 is the motorized focus Z-axis robotic arm, 6 is the CCD image acquisition system, and 7 is the programmable temperature controller. The X-axis motion motor 3 and the Y-axis motion motor 4 are connected to the main frame 1 via drive screws and are responsible for horizontal movement of the hot stage. The motorized focus Z-axis robotic arm 5 is connected to the microscope's objective lens adjustment mechanism for automatic focusing. The CCD image acquisition system 6 is mounted above the microscope eyepiece to capture sample images. The programmable temperature controller 7 is connected to the hot stage's heating / cooling components to achieve temperature control.

[0073] Attachment Figure 2 This is a flow chart of the system of the present invention. In the flow chart, A is a coordinate management module, which is used to record and manage the XYZ coordinates of the observation point; B is a temperature control calling module, which is responsible for calling the temperature control system to perform temperature control according to preset parameters; C is an image monitoring module, which is used to monitor the sample image in real time and detect change points; D is a point switching module, which automatically switches the observation point after completing the single-point measurement. After the program is started, first enter the coordinate management module A to set the observation point, then enter the temperature control calling module B to set the temperature parameters, then start the CCD image acquisition system to start image acquisition, and the image monitoring module C analyzes the image in real time. When a change point is detected, the temperature information is recorded. After completing the single-point measurement, the point switching module D drives the hardware to move to the next observation point, and repeats the above process until all point measurements are completed.

[0074] Hardware automation transformation: The microscope hot stage is equipped with a hot stage XY moving motor and an electric focusing Z-axis robotic arm to build a three-dimensional automated motion system, realizing automatic adjustment of the observation position and precise control of the focal length.

[0075] Software intelligent control program: Develop a dedicated control program with the functions of recording the XYZ coordinates of the observation point, automatically calling the temperature control software, monitoring the sample changes in real time and recording the temperature information through the CCD image acquisition system, and automatically switching the observation point to achieve full process automation.

[0076] Software and hardware collaboration mechanism: The deep integration of hardware automated motion system and software intelligent control program forms a closed-loop control to ensure the consistency of experimental operations and the accuracy of data collection.

[0077] Beneficial Effects: An XY motor installed on the microscope's heated stage enables precise horizontal movement under software control, with micron-level accuracy. A motorized Z-axis focus arm automatically adjusts focus according to software instructions, ensuring a consistently clear field of view. Working together, these two elements create a three-dimensional automated motion system, enabling rapid and precise unified setup and processing of multiple samples.

[0078] The above specific implementation methods further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A deep fusion unattended intelligent experiment system, characterized by: The experimental system includes: a microscope hot stage basic structure, an XY moving motor, a CCD image acquisition module and a control module; The microscope hot stage basic structure is used to place the sample loading area; Two sets of high-precision XY moving motors are installed in the horizontal plane direction of the microscope hot stage, which are responsible for the movement of the hot stage in the horizontal horizontal X-axis and horizontal vertical Y-axis directions respectively; The CCD image acquisition module is used to acquire microscopic images of samples in real time; The control module is connected to the CCD image acquisition module and is used to control the experimental operation according to the microscopic image.

2. A deep fusion unattended intelligent experiment system according to claim 1, characterized in that: The XY moving motor is driven by a stepper motor and matched with a precision screw transmission mechanism, which can achieve micron-level precision displacement control with a minimum moving step of 1 micron. Electric focusing Z-axis robotic arm: In the vertical direction of the microscope, an electric focusing Z-axis robotic arm is added, and the robotic arm is connected to the objective lens adjustment mechanism of the microscope; The robotic arm has a built-in servo motor and encoder, which adjusts the distance between the objective lens and the sample according to instructions to achieve an autofocus function.

3. A deep fusion unattended intelligent experiment system according to claim 1, characterized in that: The CCD image acquisition module specifically includes: A high-resolution CCD camera is installed above the eyepiece of the microscope to capture microscopic images of the sample in real time; The CCD camera has automatic exposure and automatic white balance functions, and transmits the collected image data to the control module in real time.

4. A deep fusion unattended intelligent experiment system according to claim 1, characterized in that: The control module specifically includes: a coordinate management unit, a temperature control calling unit, an image monitoring unit and a point switching unit.

5. A deep fusion unattended intelligent experiment system according to claim 4, characterized in that: The coordinate management unit is used to record and manage the XYZ three-dimensional coordinate information of each observation point; The user sets the observation points by manually inputting them through the software interface or marking them in the microscope field of view, and the software automatically saves and generates a list of point coordinates.

6. A deep fusion unattended intelligent experiment system according to claim 4, characterized in that: The temperature control calling unit specifically includes: According to the preset observation process, the temperature control instructions of the temperature control system are automatically called to achieve precise control of the sample temperature, supporting multiple temperature control modes such as linear heating, cooling, and step-by-step temperature changes.

7. A deep fusion unattended intelligent experiment system according to claim 4, characterized in that: The image monitoring unit specifically includes: Communicate with the CCD image acquisition system in real time to receive and process the collected microscopic image data; The image recognition algorithm is used to monitor the morphological changes of samples in real time. When key nodes of phase change and structural change of the sample are detected, the data recording function is automatically triggered.

8. The deep fusion unattended intelligent experiment system according to claim 4 is characterized in that: The point switching unit specifically includes: After completing the measurement of a single observation point, the control command is automatically generated to drive the XY moving motor and the electric focusing Z-axis robotic arm to move the hot stage to the next observation point and adjust the focal length to the appropriate state.

Images