Virtual and real synchronization control systems, devices, storage media, and software products for ultra-depth-of-field microscopy equipment
By using a virtual-real synchronous control system, the entire 3D model is loaded and its status is synchronized and safety is verified. This solves the problem that ultra-depth-of-field microscopes are difficult to demonstrate intuitively in demonstrations and training, and enables convenient and safe virtual operation and image fusion display.
Patent Information
- Application Number
- CN202611208049.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-11
- Publication Date
- 2026-11-17
AI Technical Summary
Existing super depth-of-field digital microscopes rely on real equipment for demonstrations, training, and understanding of complex mechanisms, making it difficult to intuitively showcase operations and functions. Furthermore, the two-dimensional software interface lacks three-dimensional posture display, and the control methods are not concrete enough.
A virtual-real synchronous control system was designed, including a physical microscopic device execution layer, a real-time data transceiver layer, and a host computer simulation layer. By loading the whole machine 3D model, synchronizing state data, and performing virtual interactive operations, the system achieves consistency between the state of virtual components and physical components, supports drag, rotate, click and other operations, and performs security verification and image rendering.
It enables the demonstration of the entire machine structure and operation without the need for actual equipment, reducing training costs and the risk of misoperation, improving image understanding and spatial orientation, and enhancing the convenience and safety of demonstrations and training.
Smart Images

Figure CN123114622A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of digital microscopy equipment, and specifically relates to a virtual-real interaction technology for ultra-depth-of-field microscopy equipment. Background Technology
[0002] Super depth-of-field digital microscopes are a type of high-end optical digital microscope. These microscopes mainly include interchangeable magnification objectives, image sensors, a focusing z-axis, a motorized platform, a switchable light source, and a matching computer host and monitor. They can perform functions such as image navigation, automatic / manual focusing, illumination control, exposure control, tilt observation, observation annotation, image stabilization, depth synthesis, 2D / 3D image stitching, 3D display, contour measurement, volume measurement, line / surface roughness measurement, and point cloud export.
[0003] While existing ultra-depth-of-field digital microscopes possess strong electrification, automation, and image analysis capabilities, they still primarily rely on physical ultra-depth-of-field microscopes for product demonstrations, training, after-sales review, and understanding complex structural states. For example, physical ultra-depth-of-field microscopes may not be readily available at customer sites, trade shows, remote conferences, or pre-sales communication, making it difficult to intuitively demonstrate their operation and functions. Furthermore, training new users on product operation and functionality requires repeated use of physical equipment, resulting in high training costs and the risk of misoperation. Additionally, while current product demonstrations and training rely on 2D software interfaces, these only display partial parameters, focus views, and 3D data, lacking a 3D orientation display corresponding to the overall structure of the device. This makes it difficult to understand the relationships between objectives, samples, platforms, fields of view, point clouds, and illumination within the same spatial context. Operations such as magnification, illumination, platform movement, and tilt angle are often performed via buttons, dropdown menus, or parameter panels, lacking a sufficiently concrete control method. Summary of the Invention
[0004] To address the technical problems mentioned in the background section, this invention proposes a virtual-real synchronization control system, device, storage medium, and program product for ultra-depth-of-field microscopy equipment.
[0005] To achieve the above-mentioned technical objectives, the technical solution of the present invention is as follows: A virtual-real synchronous control system for ultra-depth-of-field microscopy equipment is provided. The virtual-real synchronous control system includes a physical microscopy equipment execution layer, a real-time data transceiver layer, and a host computer simulation layer. The physical microscopy equipment execution layer includes a physical ultra-depth-of-field microscopy equipment. The host computer simulation layer includes a 3D model loading submodule, a virtual-real state synchronization mapping submodule, a virtual interactive control submodule, and a virtual instruction parsing submodule. After the virtual-real synchronization control system is started, the 3D model loading submodule loads the full-machine 3D model corresponding to the physical ultra-depth-of-field microscope equipment, and establishes the parameter mapping and binding relationship between each virtual component in the full-machine 3D model and each physical component in the physical ultra-depth-of-field microscope equipment; the physical microscope equipment execution layer periodically collects various status data of the physical ultra-depth-of-field microscope equipment and transmits them to the real-time data transceiver layer; the real-time data transceiver layer adds timestamps to the various status data collected by the physical microscope equipment execution layer, performs time sequence alignment and parameter standardization, forms a data packet that can be parsed by the host computer computing simulation layer, and uploads the data packet to the host computer computing simulation layer; the virtual-real state synchronization mapping submodule, based on the various status data of the physical ultra-depth-of-field microscope equipment, combines... Based on the aforementioned parameter mapping and binding relationship, the state of each virtual component in the overall 3D model is synchronously refreshed, ensuring that the state of each virtual component is consistent with that of its corresponding physical component. The user interacts with each virtual component in the virtual scene through the virtual interaction control submodule. The virtual instruction parsing submodule identifies the interaction operation instructions triggered by the virtual interaction control submodule, converts the interaction operation instructions from the virtual coordinate system to the spatial coordinate system of the physical ultra-depth-of-field microscope, and sends the converted interaction operation instructions to the real-time data transceiver layer. The real-time data transceiver layer converts the received interaction operation instructions into control signals that can be responded to by the corresponding physical components and sends them to the physical microscope execution layer, thereby realizing the state update of the corresponding physical components and the acquisition of microscope images.
[0006] Furthermore, when establishing the parameter mapping and binding relationship between each virtual component in the overall 3D model and each physical component in the physical ultra-depth-of-field microscope, it is necessary to configure the travel range of each physical component and the safety threshold of each control parameter in the upper computer simulation layer.
[0007] Furthermore, after the virtual command parsing submodule converts the interactive operation command from the virtual coordinate system to the spatial coordinate system where the physical ultra-depth-of-field microscope is located, if the actual control result corresponding to the interactive operation command exceeds the travel range of the physical component or the safety threshold of the control parameter, it intercepts the issuance of the interactive operation command and triggers an early warning prompt.
[0008] Furthermore, the host computer simulation layer also includes an image rendering submodule. The microscopic images and their measurement and analysis data acquired by the physical ultra-depth-of-field microscope are sequentially uploaded to the host computer simulation layer through the physical microscope execution layer and the real-time data transceiver relay layer. The acquired microscopic image textures are mapped to the virtual camera field of view of the whole machine 3D model, the depth-synthesized point cloud or image stitching result textures are mapped to the virtual stage of the whole machine 3D model, and the measurement and analysis results are superimposed on the depth-synthesized point cloud or image stitching result.
[0009] Furthermore, the host computer simulation layer also includes an offline demonstration submodule. The offline demonstration submodule is pre-loaded with a general-purpose ultra-depth-of-field microscope 3D model, different types of sample data, and control scripts. When the host computer simulation layer is not connected to or bound to the physical ultra-depth-of-field microscope, the user selects the corresponding sample and control script to run the general-purpose ultra-depth-of-field microscope 3D model, thereby demonstrating the operation of each component and / or function in the general-purpose ultra-depth-of-field microscope 3D model.
[0010] Secondly, the present invention also designs a computer device, including a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the computer program to realize the above-mentioned virtual and real synchronous control system for ultra-depth-of-field microscopes.
[0011] Furthermore, the present invention also designs a computer storage medium storing a computer program, which, when executed on a processor, implements the aforementioned virtual-real synchronization control system for ultra-depth-of-field microscopy equipment.
[0012] Finally, the present invention also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the aforementioned virtual-real synchronization control system for ultra-depth-of-field microscopy equipment.
[0013] The beneficial effects of adopting the above technical solution are as follows: (1) More convenient demonstration: There is no need to move the actual microscope equipment. The structure, operation method and special functions of the whole machine can be demonstrated in conferences, exhibitions and online communication. (2) Safer teaching and training: New users can practice magnification selection, focusing, tilting observation, platform movement, illumination switching and splicing measurement repeatedly on the virtual whole machine model. Incorrect operation will not damage the real objective lens, platform or sample; (3) Eliminate operating distance limitations: Operators can grasp the real-time attitude of the platform, Z-axis, tilting mechanism, objective lens, light source and camera field of view from a position far away from the microscopic execution device through the 3D twin model; (4) Improve debugging and scanning planning efficiency: Visual drag, rotation, click and box selection replace a large amount of coordinate input, and multi-angle observation and path planning are more intuitive; (5) Reduce the risk of hardware damage: Before issuing instructions, collision, overtravel and safety threshold judgments are performed to reduce objective lens impact, stage overtravel and sample damage; (6) Enhance image understanding and spatial orientation: Microscopic images, true color / false color point clouds, 2D / 3D stitching trajectories, measurement annotations and equipment 3D structures are integrated and displayed in the same space, making remote operation closer to on-site observation. Attached Figure Description
[0014] Figure 1 This is a schematic diagram of the virtual-real synchronous control system for ultra-depth-of-field microscopes involved in the embodiment; Figure 2 This is a flowchart of the virtual-real synchronization control system involved in the embodiment. Detailed Implementation
[0015] The technical solution of this embodiment will be described in detail below with reference to the accompanying drawings.
[0016] like Figure 1 As shown, the virtual-real synchronous control system for ultra-depth-of-field microscopy equipment involved in this embodiment includes a physical microscopy equipment execution layer, a real-time data transmission and reception relay layer, and a host computer simulation layer. The physical microscopy equipment execution layer includes a physical ultra-depth-of-field microscopy equipment. The host computer simulation layer includes a 3D model loading submodule, a virtual-real state synchronization mapping submodule, a virtual interactive control submodule, a virtual instruction parsing submodule, an image rendering submodule, and an offline demonstration submodule.
[0017] like Figure 2 As shown, the working process of the virtual-real synchronization control system for ultra-depth-of-field microscopes involved in this embodiment is as follows: 1. Establish a mapping between physical equipment and virtual models. The 3D model loading submodule loads the complete 3D model corresponding to the physical ultra-depth-of-field microscope equipment and establishes parameter mapping and binding relationships between each virtual component in the complete 3D model and each physical component in the physical ultra-depth-of-field microscope equipment. The complete 3D model can be a lightweight model, retaining key movable structures and spatial relationships, and simplifying unnecessary details (such as screws, nameplates, etc.) to meet real-time rendering requirements.
[0018] 2. Real-time synchronization of the status of physical devices and virtual models The physical microscope execution layer periodically collects various status data of the physical ultra-depth-of-field microscope (including Z-axis position, platform position, tilt angle, objective lens type / magnification, illumination mode, light source intensity, camera exposure, focus status, etc.) and transmits them to the real-time data transceiver layer. The real-time data transceiver layer adds timestamps to the various status data collected by the physical microscope execution layer, performs time sequence alignment and parameter standardization, and forms a data packet that can be parsed by the host computer simulation layer, and uploads the data packet to the host computer simulation layer. The virtual-real state synchronization mapping submodule, based on the various status data of the physical ultra-depth-of-field microscope and combined with the aforementioned parameter mapping binding relationship, synchronously refreshes the state of each virtual component in the whole machine 3D model, so that the state of each virtual component is consistent with the state of the corresponding physical component.
[0019] 3. Perform virtual interactive operations Users can interact with various virtual components in the virtual scene through the virtual interaction control submodule, including dragging, rotating, clicking, or selecting. For example, users can drag a virtual platform to a target position, adjust the virtual z-axis height, click the virtual objective disk to switch magnification, and select a 2D / 3D stitching area.
[0020] 4. Virtual operation command parsing and security verification The virtual instruction parsing submodule identifies the interactive operation instructions triggered by the virtual interactive control submodule and transforms these instructions from the virtual coordinate system to the spatial coordinate system of the physical ultra-depth-of-field microscope. Based on the parsed instructions, it checks whether any instructions exceed the component's travel range (e.g., whether the z-axis / platform exceeds soft or hard limits, whether the tilt angle exceeds the maximum safe angle, whether the stitching path exceeds the platform boundary, etc.) or the safety threshold of control parameters (e.g., the distance between the lens and the sample, light intensity, platform movement speed, etc.). If no instructions exceed the component's travel range or the safety threshold of the control parameters, the instructions are sent to the real-time data transceiver layer; otherwise, the instructions are intercepted, and a warning is triggered.
[0021] 5. Physical equipment execution and feedback The real-time data transceiver layer converts received interactive operation commands into control signals that can be responded to by the corresponding physical components and sends them to the execution layer of the physical microscopy device. After the physical device completes the response action according to the control signals, it sends back new status data, refreshes the overall 3D model, and forms a continuous closed loop.
[0022] 6. Microscopic image rendering and display The microscopic images and their measurement and analysis data acquired by the physical ultra-depth-of-field microscope are sequentially uploaded to the host computer computing and simulation layer through the physical microscope execution layer and the real-time data transceiver relay layer. The acquired microscopic image textures are mapped onto the virtual camera field of view of the whole machine 3D model, the depth-synthesized point cloud or image stitching result textures are mapped onto the virtual stage of the whole machine 3D model, and the measurement and analysis results are superimposed and displayed on the depth-synthesized point cloud or image stitching result.
[0023] The above steps illustrate the operation process of the system in this embodiment when connected to a real device. When not connected to a real device, the system enters offline demonstration mode. The offline demonstration submodule is pre-loaded with a general-purpose ultra-depth-of-field microscope 3D model, different types of sample data, and control scripts. The user selects the corresponding sample and control script to run the general-purpose ultra-depth-of-field microscope 3D model, thereby demonstrating the operation of each component and / or function in the general-purpose ultra-depth-of-field microscope 3D model.
[0024] This embodiment also relates to a computer device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program to implement the above-described contents. This embodiment also relates to a computer storage medium storing a computer program, which, when executed on a processor, implements the above-described contents. This embodiment also relates to a computer program product, including a computer program or instructions, which, when executed by a processor, implements the above-described contents.
[0025] The embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.
Claims
1. A virtual-real synchronous control system for ultra-depth-of-field microscopy equipment, characterized in that: The virtual-real synchronous control system includes a physical microscopic device execution layer, a real-time data transceiver layer, and a host computer simulation layer; the physical microscopic device execution layer includes a physical ultra-depth-of-field microscopic device, and the host computer simulation layer includes a 3D model loading submodule, a virtual-real state synchronization mapping submodule, a virtual interactive control submodule, and a virtual instruction parsing submodule. After the virtual-real synchronous control system is started, the 3D model loading submodule loads the whole machine 3D model corresponding to the physical ultra-depth-of-field microscope, and establishes the parameter mapping and binding relationship between each virtual component in the whole machine 3D model and each physical component in the physical ultra-depth-of-field microscope. The physical microscope execution layer periodically collects various status data of the physical ultra-depth-of-field microscope and transmits them to the real-time data transceiver layer. The real-time data transceiver layer adds timestamps to the various status data collected by the physical microscope execution layer, performs time sequence alignment and parameter standardization, forming a data packet that can be parsed by the host computer simulation layer, and uploads the data packet to the host computer simulation layer. The virtual-real state synchronization mapping submodule, based on the various status data of the physical ultra-depth-of-field microscope and combined with the aforementioned parameter mapping binding relationship, synchronously refreshes the status of each virtual component in the whole machine 3D model, so that the status of each virtual component is consistent with the status of the corresponding physical component. The user interacts with each virtual component in the virtual scene through the virtual interaction control submodule. The virtual instruction parsing submodule identifies the interaction operation instruction triggered by the virtual interaction control submodule, converts the interaction operation instruction from the virtual coordinate system to the spatial coordinate system where the physical ultra-depth-of-field microscope is located, and sends the converted interaction operation instruction to the real-time data transceiver layer. The real-time data transceiver layer converts the received interactive operation commands into control signals that can be responded to by the corresponding physical components and sends them to the physical microscope execution layer to realize the status update of the corresponding physical components and the acquisition of microscopic images.
2. The virtual-real synchronization control system for ultra-depth-of-field microscopy equipment according to claim 1, characterized in that: When establishing the parameter mapping and binding relationship between each virtual component in the overall 3D model and each physical component in the physical ultra-depth-of-field microscope, it is necessary to configure the travel range of each physical component and the safety threshold of each control parameter in the upper computer simulation layer.
3. The virtual-real synchronization control system for ultra-depth-of-field microscopy equipment according to claim 2, characterized in that: After converting the interactive operation command from the virtual coordinate system to the spatial coordinate system of the physical ultra-depth-of-field microscope, the virtual command parsing submodule intercepts the issuance of the interactive operation command and triggers an early warning if the actual control result corresponding to the interactive operation command exceeds the travel range of the physical component or the safety threshold of the control parameter.
4. The virtual-real synchronization control system for ultra-depth-of-field microscopy equipment according to claim 1, characterized in that: The host computer simulation layer also includes an image rendering submodule. The microscopic images and their measurement and analysis data acquired by the physical ultra-depth-of-field microscopic device are uploaded to the host computer simulation layer through the physical microscopic device execution layer and the real-time data transceiver relay layer. The acquired microscopic image textures are mapped to the virtual camera field of view of the whole machine 3D model, the depth-synthesized point cloud or image stitching result textures are mapped to the virtual stage of the whole machine 3D model, and the measurement and analysis results are superimposed on the depth-synthesized point cloud or image stitching result.
5. The virtual-real synchronization control system for ultra-depth-of-field microscopy equipment according to claim 1, characterized in that: The host computer simulation layer also includes an offline demonstration submodule. The offline demonstration submodule is pre-loaded with a general-purpose ultra-depth-of-field microscope 3D model, different types of sample data, and control scripts. When the host computer simulation layer is not connected to or bound to the physical ultra-depth-of-field microscope, the user selects the corresponding sample and control script to run the general-purpose ultra-depth-of-field microscope 3D model, thereby demonstrating the operation of each component and / or function in the general-purpose ultra-depth-of-field microscope 3D model.
6. A computer device comprising a processor and a memory, wherein the memory stores a computer program, characterized in that: The processor is used to execute the computer program to implement the virtual-real synchronization control system for ultra-depth-of-field microscopes as described in any one of claims 1-5.
7. A computer storage medium storing a computer program, characterized in that: When the computer program is executed on the processor, it implements the virtual-real synchronization control system for ultra-depth-of-field microscopes as described in any one of claims 1-5.
8. A computer program product, comprising a computer program or instructions, characterized in that: When the computer program or instructions are executed by the processor, they implement the virtual-real synchronization control system for ultra-depth-of-field microscopes as described in any one of claims 1-5.
