Multi-terminal distributed digital microscope system based on VR glasses

Abstract

The invention provides a multi-end distributed digital microscope system based on VR glasses. The multi-end distributed digital microscope system comprises a digital camera module, an electric microscope objective lens module, an electric objective table module, a slice management module, a data interface module, local / remote end VR glasses, an operation rocker and a control host module. A traditional eyepiece and a mechanical knob are replaced by electric control, dual-mode observation of an entity glass slide and a digital slice is supported, after the entity slice is automatically called by a mechanical arm, multiple observation is carried out through an electric objective lens and an objective table, and an image is displayed through VR glasses; digital slices can be scaled, translated and subjected to image acquisition. A remote user can synchronously observe and reversely control system operation in real time through a network, the sending opportunity is reselected according to a back-off algorithm in a CSMA / CA mechanism, and multiple users can operate different slices in parallel and do not conflict with one another. The system solves the problems of uneven resource distribution, difficulty in remote cooperation, low operation efficiency and the like in pathological diagnosis, and is suitable for medical diagnosis, teaching and scientific research scenes.

Description

Technical Field

[0001] The present application relates to the field of microscope technology, and in particular to a digital microscope system that integrates virtual reality (VR) technology, motorized control, and remote collaboration functions, and is suitable for remote work in clinical pathology diagnosis, remote teaching and scientific research, and industrial testing. Background Art

[0002] As an important pillar of the modern medical diagnosis and treatment system, the accuracy and timeliness of pathological diagnosis directly affect the clinical treatment effect. However, the current medical system faces the following severe challenges:

[0003] 1. There is a serious imbalance in pathology medical resources. Traditional consultation models are limited by difficulties such as the transportation of physical slides, slow upload of digital slides, and the difficulty of achieving real-time, remote, synchronized microscopy viewing, making it difficult to transfer high-quality resources to the grassroots level.

[0004] 2. Bottlenecks in pathology training: The training cycle for pathologists is as long as 10-15 years, with microscopic diagnostic training requiring at least five years. Traditional teaching microscopes have significant limitations: multi-head microscopes are bulky and restrict their use cases. Teachers and students must coexist in the same room, making remote, real-time instruction impossible. The lack of intelligent annotation and case review functionality reduces teaching efficiency. Primary care physicians face difficulties in further training and face a lack of continuing education opportunities.

[0005] 3. Occupational health and work efficiency: Traditional microscopes have significant drawbacks. Fixed sitting postures lead to a 72% incidence of cervical and lumbar spondylosis. Observing through the eyepiece causes visual fatigue, resulting in an average daily working time of less than four hours. Mechanical knobs are cumbersome to operate, making single diagnosis time-consuming. Existing digital microscope systems still retain traditional user interfaces, resulting in poor human-computer interaction and weak remote collaboration capabilities, making them unable to meet the needs of multi-consultation consultations. They also lack intelligent auxiliary diagnosis modules.

[0006] 4. Current state of technology development: Existing telepathology systems primarily utilize: Whole-slide digital scanning (WSI) solutions: high cost (>200 yuan per case) and poor timeliness (pre-scanning required). Traditional microscope video transmission: low resolution (usually ≤1080p) and lack of three-dimensional depth of field information. VR technology has achieved breakthroughs: the new generation of VR glasses weighs <200g, has a resolution of 8K / eye, and supports natural interaction methods such as gesture recognition and voice control. Latency is controlled within 20ms to meet real-time operation requirements. Mature automation technology: precision motion control accuracy can reach 0.1μm. Robotic arm positioning error is <0.01mm. Image recognition algorithm accuracy exceeds 99%.

[0007] 5. Existing Technology Shortcomings: Remote consultation systems rely on specialized equipment, resulting in high deployment costs. Existing VR microscope solutions on the market have the following shortcomings: they only support digital slide data, lack an automated slide management system, and fail to integrate intelligent diagnostic algorithms. Concurrent access by multiple users can easily lead to conflicts.

[0008] Therefore, traditional microscopes rely on eyepiece observation and manual adjustment, resulting in cumbersome operation, difficulty in multi-person collaboration, and inability to share remotely. While existing digital microscopes support image digitization, they lack automated management of physical sections, support for VR interaction, and multi-terminal collaborative operation. Therefore, a microscope system integrating automation, VR visualization, and remote control capabilities is urgently needed.

[0009] The present invention is proposed to address the above-mentioned technological gaps and to build a new generation of intelligent pathology diagnosis platform by innovatively integrating VR technology, automated control technology and remote collaboration technology. Summary of the Invention

[0010] In light of this, it is necessary to provide a multi-terminal distributed digital microscope system based on VR glasses to address the existing problems of inconvenient operation, poor collaboration, and insufficient remote support. The system uses motorized modules, automated slide management, and VR interaction technology to achieve efficient and flexible pathology diagnosis and teaching.

[0011] The present application provides a multi-terminal distributed digital microscope system based on VR glasses, including a digital camera module, an electric microscope objective module, an electric stage module, a slice management module, a data interface module, local / remote VR glasses, an operating joystick and a control host module. By replacing traditional eyepieces and mechanical knobs with electric control, dual-mode observation of physical slides and digital slices is supported: after the physical slices are automatically retrieved by the robotic arm, they are observed at multiple times through the electric objective lens and stage, and the images are displayed through VR glasses; digital slices can be zoomed, translated and imaged. Remote users can observe synchronously in real time through the network and reversely control system operations, reselect the sending time according to the backoff algorithm in the CSMA / CA mechanism, and multiple users can operate different slices in parallel without conflict. It solves the problems of uneven resource distribution, difficulty in remote collaboration, and low operational efficiency in pathological diagnosis, and is suitable for medical diagnosis, teaching and scientific research scenarios.

[0012] In a first aspect, an embodiment of the present application provides a multi-terminal distributed digital microscope system based on VR glasses, the system comprising:

[0013] A digital camera module (1) for collecting microscope magnified images;

[0014] An electric microscope objective lens module (2), comprising a rotary drive device for switching objective lenses of different magnifications;

[0015] The electric stage module (3) includes a three-axis precision moving drive device for front-back, left-right, and up-down, capable of moving in the horizontal plane and fine-moving in the vertical direction for adjusting the observation area and focusing;

[0016] A slide management module (5) comprising a barcode scanning camera (52) for identifying textual information on slide labels, a robotic arm (51) with an electric suction cup (53), and a slide placement slot (54) for automatically identifying, retrieving, and returning physical slides;

[0017] A data interface module (4) is used to connect the hardware modules with the control host module (8) and transmit control signals and data;

[0018] Local VR glasses (6) and joysticks (7) for local interactive operation and immersive observation;

[0019] A control host module (8) is used to coordinate the operation of various modules of the system through control instructions;

[0020] The remote VR glasses (10) and the remote mobile device (9) are connected to the control host module (8) via a network to achieve remote observation and control;

[0021] The system eliminates the traditional eyepiece system and mechanical adjustment knobs, and achieves observation through motorized control and VR interaction.

[0022] Furthermore, the robotic arm (51) of the slice management module (5) absorbs the designated physical slide through the electric suction cup (53) and places it on the electric stage module (3) according to the instruction of the control host module (8), and automatically completes the return and replacement of the slice after the observation is completed.

[0023] Furthermore, the barcode scanning camera (52) for identifying text information on the slice label is used to scan the QR code information on the slide and enter it into the system, thereby realizing automatic identification and management of the slide.

[0024] Furthermore, the rotation drive device of the electric microscope objective module (2) and the three-axis precision movement drive device of the electric stage module (3) are both electrically connected to the control host module (8) and receive control instructions from the operating joystick (7) or the remote mobile terminal device (9).

[0025] Furthermore, the system supports dual-mode observation of physical slides and digital whole sections, wherein: for physical slides, the slice management module (5) is controlled by operating the joystick (7) or the remote mobile terminal device (9) to retrieve a specific slice, the motorized microscope objective lens module (2) is controlled to switch the magnification, and the motorized stage module (3) is controlled to move and adjust the observation area; for digital whole sections, the magnification is adjusted by operating the joystick (7) or the remote mobile terminal device (9) to control the software to zoom in and out and to translate in any direction to adjust the observation area.

[0026] Furthermore, the system supports simultaneous multi-user operation. Multiple local and remote users can simultaneously observe and operate different physical or digital slices. In order to prevent conflicts between operations, the implementation is as follows:

[0027] The local VR glasses (6) share the observation image with the remote VR glasses (10), and the joystick (7) is operated to enjoy the first priority of the operation;

[0028] The remote user uses a remote mobile terminal device (9) to sense the network allocation vector NAV information terminal allocated by the device through a carrier, and determines whether the communication channel of the network allocation vector NAV information allocated by the device is idle;

[0029] The control host module (8) determines whether the communication channel is idle, and if so, sends a reply instruction to the remote mobile terminal device (9); if not, uses a collision algorithm to determine when the communication channel is idle, calculates the waiting time, sends the calculation result to the remote mobile terminal device (9), and continues to listen;

[0030] Among them, the probability of collision is:

[0031]

[0032] The current NAV value reflects the remaining time when the channel is busy, and the maximum NAV value is the time when the channel is completely idle. If the NAV value is high, the collision probability is high;

[0033] The waiting time depends on the rate at which the NAV value decreases and the time it takes for the channel to return to an idle state. The NAV value decreases with the transmission of each data frame. When the NAV value drops to 0, the channel is considered idle, and the collision probability is estimated using the following formula:

[0034]

[0035] Among them, V NAV Indicates the rate at which the NAV value decreases per second;

[0036] Virtual carrier sensing technology determines whether the channel is idle by monitoring the NAV value. When the NAV value is greater than 0, it indicates that the channel is busy. When the NAV value drops to 0, the channel is idle. If the NAV value is higher, it will take longer to retry sending data frames. At this time, the remote mobile terminal device (9) will continue to monitor the channel status and adjust the waiting time according to the change of the NAV value.

[0037] After a collision occurs, the remote mobile device (9) will reselect the transmission time according to the backoff algorithm in the CSMA / CA mechanism; the backoff algorithm usually adopts an exponential backoff strategy, that is, the backoff time doubles after each collision until the data frame is successfully transmitted;

[0038] The backoff time is calculated as follows:

[0039] t tb =2 r ×DIFS,

[0040] Where r is the number of collisions, and DIFS is the distributed coordination function interval, which is a fixed time interval used to distinguish different types of frame transmissions.

[0041] Furthermore, the remote mobile terminal device (9) reversely controls the local control host module (8) through the network, and can operate various modules of the system to complete slice replacement, magnification switching, observation area adjustment and image acquisition operations.

[0042] Furthermore, the image data collected by the digital camera module (1) can be displayed on the local VR glasses (6) and the remote VR glasses (10) at the same time after being transmitted to the control host module (8) via the data interface module (4).

[0043] Furthermore, the operating joystick (7) can control the digital camera module (1) to capture or record images and videos of the area of ​​interest by controlling the host module (8).

[0044] Furthermore, the system supports controlling the operation of each module through voice input.

[0045] In a second aspect, an embodiment of the present application provides an electronic device, including:

[0046] processor;

[0047] a memory for storing processor-executable instructions;

[0048] Wherein, the processor is configured to implement the observation method of the multi-terminal distributed digital microscope system based on VR glasses as described in the first aspect when executing the instructions.

[0049] In a third aspect, an embodiment of the present application provides a computer-readable storage medium storing a program, wherein the program instructs a device to execute the observation method of a multi-terminal distributed digital microscope system based on VR glasses as described in the first aspect.

[0050] Beneficial effects:

[0051] (1) Improve diagnostic efficiency. Through automated slide management and electric control, slides can be quickly retrieved (single operation time < 5 seconds). Multiple magnification objective lenses can be switched with one button, eliminating the need for manual rotation and lens change in traditional microscopes (saving 60% of operation time). Supports instant loading of digital slides, eliminating the time required to prepare physical slides.

[0052] (2) Improve user experience. VR immersive observation replaces traditional eyepieces, alleviating cervical fatigue. Joystick + voice control simplifies complex operations. Ergonomic interactive design reduces the risk of occupational injury.

[0053] (3) Breaking through spatial limitations. Based on the backoff algorithm in the CSMA / CA mechanism, the sending timing is reselected. Multiple users can operate different slices in parallel without conflict. Multi-terminal collaboration: Local and remote users can operate different slices simultaneously without conflict, achieving real-time collaboration across regions. Mobile terminal access means that diagnostic work is no longer restricted to fixed locations.

[0054] (4) Innovation in teaching and scientific research. The teacher and student fields of vision are synchronized in real time with high accuracy. The intelligent annotation system can automatically mark typical lesion areas with high accuracy. The entire operation process can be recorded and played back to support the construction of a teaching case library.

[0055] (5) Optimized resource allocation. A single device can serve multiple medical institutions, improving utilization. Digital slices are permanently stored, reducing the loss of physical slices. Automated management reduces manual operation errors.

[0056] (6) It is easy to promote and apply, and the operation is relatively simple, making it suitable for promotion and application in more application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A three-dimensional architecture diagram of a multi-terminal distributed digital microscope system based on VR glasses provided in one embodiment of the present application.

[0058] Figure 2 A schematic diagram of the slice management module structure provided in one embodiment of the present application. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0060] It should be noted that, in the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art in the art to which this application relates. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0061] It should be noted that, in the embodiments of the present application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order. Features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0062] Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0063] Figure 1 A three-dimensional architecture diagram of a multi-terminal distributed digital microscope system based on VR glasses provided in one embodiment of the present application. Figure 2 This is a schematic diagram of the slice management module structure provided in one embodiment of the present application. The system is a multi-terminal distributed digital microscope system based on VR glasses. Its core feature is that it replaces the eyepiece system and mechanical adjustment knobs in traditional microscopes through electric control and VR interaction technology to achieve a more efficient and convenient observation method. Figure 1 、 2 As shown, the system includes:

[0064] A digital camera module (1) for collecting microscope magnified images;

[0065] An electric microscope objective lens module (2), comprising a rotary drive device for switching objective lenses of different magnifications;

[0066] The electric stage module (3) includes a three-axis precision moving drive device for front-back, left-right, and up-down, capable of moving in the horizontal plane and fine-moving in the vertical direction for adjusting the observation area and focusing;

[0067] A slide management module (5) comprising a barcode scanning camera (52) for identifying textual information on slide labels, a robotic arm (51) with an electric suction cup (53), and a slide placement slot (54) for automatically identifying, retrieving, and returning physical slides;

[0068] A data interface module (4) is used to connect the hardware modules with the control host module (8) and transmit control signals and data;

[0069] Local VR glasses (6) and joysticks (7) for local interactive operation and immersive observation;

[0070] A control host module (8) is used to coordinate the operation of various modules of the system through control instructions;

[0071] The remote VR glasses (10) and the remote mobile device (9) are connected to the control host module (8) via a network to achieve remote observation and control;

[0072] The system eliminates the traditional eyepiece system and mechanical adjustment knobs, and achieves observation through motorized control and VR interaction.

[0073] In addition, the system supports dual-mode observation of physical slides and digital whole slides, and supports simultaneous multi-user operation. Multiple local and remote users can observe and operate different physical or digital slides synchronously, and the operations do not conflict with each other.

[0074] Specifically, the robotic arm (51) of the slice management module (5) absorbs the designated physical slide through the electric suction cup (53) and places it on the electric stage module (3) according to the instruction of the control host module (8), and automatically completes the return and replacement of the slice after the observation is completed.

[0075] Following instructions from the host module, the robotic arm of the slide management module uses a motorized suction cup to pick up a designated physical slide and place it on the motorized stage module. Once observation is complete, the slide is automatically returned to its original position and replaced. This feature significantly improves experimental efficiency and reduces the time and cost of manual operation.

[0076] Specifically, the barcode scanning camera (52) for identifying text information on the slice label is used to scan the QR code information on the slide and enter it into the system, thereby realizing automatic identification and management of the slide.

[0077] The barcode scanning camera, which recognizes textual information on slide labels, scans the QR code on the slide and enters it into the system, enabling automatic slide identification and management. This function ensures the accuracy and traceability of sample information during experiments.

[0078] Specifically, the rotation drive device of the electric microscope objective module (2) and the three-axis precision movement drive device of the electric stage module (3) are both electrically connected to the control host module (8) and receive control instructions from the operating joystick (7) or the remote mobile terminal device (9).

[0079] The rotation drive of the motorized microscope objective module and the three-axis precision motion drive of the motorized stage module are both electrically connected to the control host module, receiving control commands from a joystick or a remote mobile device. This design allows users to easily switch objective magnification and adjust the observation area using VR glasses or a mobile device.

[0080] Specifically, the system supports dual-mode observation of physical slides and digital whole sections, wherein: for physical slides, the slice management module (5) is controlled by operating a joystick (7) or a remote mobile terminal device (9) to retrieve a specific slice, the motorized microscope objective lens module (2) is controlled to switch the magnification, and the motorized stage module (3) is controlled to move and adjust the observation area; for digital whole sections, the magnification is adjusted by operating a joystick (7) or a remote mobile terminal device (9) to control the software to zoom in and out and to translate in any direction to adjust the observation area.

[0081] The system supports dual-mode observation of both physical slides and digital whole slides. For physical slides, users can use the joystick or a remote mobile device to control the slide management module to retrieve specific slides and switch magnification. For digital whole slides, the software adjusts the magnification to zoom in and out and pan to adjust the observation area.

[0082] Specifically, the system supports simultaneous multi-user operation. Multiple local and remote users can simultaneously observe and operate different physical or digital slices. In order to prevent operations from conflicting with each other, the implementation method is as follows:

[0083] The local VR glasses (6) share the observation image with the remote VR glasses (10), and the joystick (7) is operated to enjoy the first priority of the operation;

[0084] The remote user uses a remote mobile terminal device (9) to sense the network allocation vector NAV information terminal allocated by the device through a carrier, and determines whether the communication channel of the network allocation vector NAV information allocated by the device is idle;

[0085] The control host module (8) determines whether the communication channel is idle, and if so, sends a reply instruction to the remote mobile terminal device (9); if not, uses a collision algorithm to determine when the communication channel is idle, calculates the waiting time, sends the calculation result to the remote mobile terminal device (9), and continues to listen;

[0086] Among them, the probability of collision is:

[0087]

[0088] The current NAV value reflects the remaining time when the channel is busy, and the maximum NAV value is the time when the channel is completely idle. If the NAV value is high, the collision probability is high;

[0089] The waiting time depends on the rate at which the NAV value decreases and the time it takes for the channel to return to an idle state. The NAV value decreases with the transmission of each data frame. When the NAV value drops to 0, the channel is considered idle, and the collision probability is estimated using the following formula:

[0090]

[0091] Among them, V NAV Indicates the rate at which the NAV value decreases per second;

[0092] Virtual carrier sensing technology determines whether the channel is idle by monitoring the NAV value. When the NAV value is greater than 0, it indicates that the channel is busy. When the NAV value drops to 0, the channel is idle. If the NAV value is higher, it will take longer to retry sending data frames. At this time, the remote mobile terminal device (9) will continue to monitor the channel status and adjust the waiting time according to the change of the NAV value.

[0093] After a collision occurs, the remote mobile device (9) will reselect the transmission time according to the backoff algorithm in the CSMA / CA mechanism; the backoff algorithm usually adopts an exponential backoff strategy, that is, the backoff time doubles after each collision until the data frame is successfully transmitted;

[0094] The backoff time is calculated as follows:

[0095] t tb =2 r ×DIFS,

[0096] Where r is the number of collisions and DIFS is the Distributed Coordination Function Interval, a fixed time interval used to distinguish different types of frame transmissions. These calculation methods ensure efficient utilization of the communication channel and reduce the possibility of channel conflicts.

[0097] Specifically, the remote mobile terminal device (9) reversely controls the local control host module (8) through the network, and can operate various modules of the system to complete slice replacement, magnification switching, observation area adjustment and image acquisition operations.

[0098] The system supports multi-user simultaneous operation, allowing multiple local and remote users to observe and operate on different physical or digital slides simultaneously without conflicting operations. This feature is particularly suitable for collaborative research.

[0099] Specifically, the image data collected by the digital camera module (1) is transmitted to the control host module (8) via the data interface module (4), and can be displayed simultaneously on the local VR glasses (6) and the remote VR glasses (10).

[0100] The remote mobile device reversely controls the local control host module via the network, operating the various system modules to perform slice changes, magnification switching, observation area adjustment, and image acquisition. Simultaneously, image data captured by the digital camera module is transmitted to the control host module via the data interface module and can be displayed simultaneously on both the local and remote VR glasses.

[0101] Specifically, the operating joystick (7) can control the digital camera module (1) to shoot or record images and videos of the operator's area of ​​interest by controlling the host module (8). The operating joystick can control the digital camera module to shoot or record images and videos of the area of ​​interest by controlling the host module, further enhancing the flexibility and practicality of the system.

[0102] Specifically, the system supports controlling the operation of each module through voice input. The system supports controlling the operation of each module through voice input, which provides users with a more convenient operation method.

[0103] By combining VR technology, automated control, and digital microscopy, the system delivers an efficient, convenient, and multi-modal microscopic observation experience. Its innovation lies in eliminating the mechanical adjustment knobs of traditional microscopes and enabling highly intelligent operation through motorized control and VR interaction. It also supports multi-user collaboration and remote operation, providing new solutions for scientific research and education.

[0104] Specifically, the system includes a digital camera module 1, an electric microscope objective module 2, an electric stage module 3, a slice management module 4, a data interface module 5, local VR glasses 6 and remote VR glasses 10, an operating joystick (7), a control host module 8 and a remote mobile terminal device 9. The electric microscope objective, digital camera module, electric stage module and slice management module of the microscope system are connected to the control host module through the data interface module, while the local VR glasses and operating joystick are also connected to the control host module. The remote VR glasses and the remote mobile terminal device are connected to each other and linked to the local segment control host module through the network.

[0105] The microscope system supports the observation and analysis of both physical slides and digital whole sections. For physical slides, the slides are placed in the slice management module. The slice management module has a robotic arm 51 with a barcode scanning camera 52 and an electric suction cup 53, and a slide placement slot 54. The scanning camera scans the QR code information on the slice and enters it into the system. The slice management module, according to the control module's instructions, uses the robotic arm's electric suction cup to adsorb the specified physical glass slice and places it on the electric stage. The electric microscope objective lens can be used to magnify the target area of ​​the physical glass slide placed on the electric stage at different times. The magnified image is collected by the digital camera module and transmitted to the control host module via the data interface module and displayed on the local VR glasses. The operating joystick can control the slice management module to retrieve specific slices and complete slice replacement through the control host. It can also control the digital camera module to shoot or record images and videos of the area of ​​interest, control the electric microscope objective lens to switch the magnification, and control the electric stage module to move and adjust the observation area.

[0106] For digital full slices, data can be transmitted locally or over the network to the control host module and displayed on the local VR glasses. The joystick can be used to control the host to capture or record images or videos of the area of ​​interest in the digital slice, and the software in the control host can be controlled to adjust the magnification for zooming and translation in any direction to adjust the observation area.

[0107] The above observation and analysis processes can be displayed synchronously on the remote VR glasses and the remote mobile device via the network. The remote mobile device can reversely control the local control host module and operate the microscope system to perform any of the above operations. Multiple users can simultaneously observe different physical and digital slides on the device through the local and remote mobile devices without conflict.

[0108] Example 1: Entity Slice Observation

[0109] The user places the slide into the slide management module slot, uses the scanning camera to identify the text information on the slide label or recognizes the barcode and enters it into the system.

[0110] The robotic arm adsorbs the specified slice to the motorized stage according to the control instructions.

[0111] The user adjusts the objective lens magnification and stage position through the joystick, and the digital camera captures images and transmits them to the VR glasses for display.

[0112] The specific operation process is as follows:

[0113] Slide pretreatment and sample loading stage

[0114] The user places the prepared pathology slide (e.g., size: 25mm×75mm×1mm) into the standardized slot (54) of the slice management module (5). Each slot is equipped with: a radio frequency identification (RFID) tag reader (operating frequency 13.56MHz); a pressure sensor (range 0-5N, accuracy ±0.01N); and a dustproof sealing cover (transmittance >95%).

[0115] A barcode scanning camera (52) (resolution 1280×1024, scanning rate 30 fps) for recognizing text information on slide labels automatically captures the text or QR code information on slide labels (compliant with ISO / IEC 15415 standards). The system performs the following operations:

[0116] Verify that the slide ID matches the pathology information system (LIS); record the slide location coordinates (positioning accuracy ±0.1mm) and update the inventory status database.

[0117] Intelligent adjustment and positioning stage

[0118] After receiving the observation instruction, the control host module (8) sends a control signal to the robotic arm (51) through the ModbusRTU protocol: the robotic arm adopts a 6-axis collaborative robot (repeat positioning accuracy ±0.02mm), the electric suction cup (53) (negative pressure range -80kPa to -20kPa) automatically adjusts the adsorption force according to the weight of the slide (detection range 5-30g), and the motion trajectory planning algorithm ensures that the obstacle avoidance distance is ≥10mm.

[0119] The slide is precisely placed in the observation area of ​​the motorized stage (3) (temperature control range 20-25℃±0.5℃), and the system automatically completes: initial focus (using laser ranging, accuracy ±1μm); lighting intensity adjustment (LED light source, brightness adjustable range 300-6500K).

[0120] VR interactive observation stage. The user controls the rotation drive device (stepper motor, step angle 1.8°) of the electric microscope objective module (2) by operating the joystick (7) (sampling rate 100Hz) to switch the magnification (4× / 10× / 20× / 40× / 100×). The electric stage module (3) performs three-axis motion (X / Y axis travel 100mm, Z axis 20mm; positioning accuracy ±0.5μm). The digital camera module (1) (CMOS sensor, effective pixel 20 million) collects images in real time and transmits them to the image processing unit (FPGA for real-time noise reduction and HDR enhancement) through the data interface module (4) (USB3.0, transmission rate 5Gbps).

[0121] The control host module (8) performs three-dimensional reconstruction (depth of field synthesis layers ≥ 15 layers). The processed image data (resolution 3840×2160@60fps) is output to the local VR glasses (6) via the HDMI 2.1 interface, providing virtual field of view adjustment (FOV 60°-120° adjustable), digital scale overlay (accuracy ±1%), and multi-modal display (bright field / dark field / phase contrast optional).

[0122] It can also be used for intelligent assisted diagnosis, such as using a deep learning-based lesion recognition algorithm (with an accuracy rate >95%) to annotate suspicious areas in real time and automatically generate structured reports (compliant with the DICOM standard). It can also support remote collaboration: observation data is synchronously compressed (H.265 encoding) and uploaded to the cloud, supporting real-time consultations for ≥4 parties (with a latency <200ms). It can also be used for ergonomic optimization: VR glasses support 6DoF head tracking, voice command recognition (supporting mixed Chinese and English input), and a gesture control interface (recognition accuracy ±2mm).

[0123] System maintenance and calibration

[0124] Automatically perform the following daily: white balance calibration, robotic arm zero point calibration, and optical system dust removal (such as pulse airflow cleaning).

[0125] Periodic maintenance: weekly guide rail lubrication, monthly optical component calibration (such as interferometer testing).

[0126] The innovation of this embodiment lies in:

[0127] Through mechatronic design, fully automatic slide management is achieved (300% efficiency improvement compared to traditional methods), the VR interactive interface lowers the operating threshold (novice training time is shortened by 80%), the intelligent diagnostic assistance system significantly improves the accuracy of film reading (false negative rate is reduced to <2%), and the modular design supports rapid maintenance (average repair time is <15 minutes).

[0128] Example 2: Remote Collaboration

[0129] The remote mobile device sends a command to the control host to call a digital slice or request to replace a physical slice. After the local system executes the operation, the real-time image is synchronized to the remote VR glasses and mobile device.

[0130] The specific operation process is as follows:

[0131] Remote access and identity authentication stage

[0132] Remote mobile devices (9) (including but not limited to VR glasses, tablet computers, and workstations) access the system through an encrypted tunnel (TLS1.3 protocol), can pass multi-factor identity authentication (such as biometrics + dynamic tokens), and perform hierarchical authority management (divided into three levels: observer / operator / administrator), and can also bind digital certificates (in compliance with HIPAA security standards).

[0133] The control host module (8) establishes a dedicated session channel: such as allocating independent bandwidth (minimum guarantee of 10Mbps), or creating an operation log blockchain for storage, and loading personalized UI configuration (including display layout, annotation preferences, etc.).

[0134] Bidirectional control and data synchronization

[0135] The remote VR glasses (10) realize the following control functions: real-time control right application (using a token ring mechanism to avoid conflicts), reverse control command transmission (including: objective lens magnification switching command (response time <50ms), stage three-dimensional coordinate setting (accuracy ±0.1μm), image acquisition parameter adjustment (exposure / gain / white balance), voice annotation synchronization (STT conversion accuracy >98%).

[0136] Data synchronization adopts a layered transmission strategy: basic layer: compressed video stream (H.265, bit rate 2-8Mbps adjustable), enhancement layer: original image data (ROI area transmitted on demand), metadata layer: operation instructions / annotation information (JSON format).

[0137] Multi-user concurrency control mechanism

[0138] Resource allocation utilizes a microservices architecture: Physical slice access utilizes round-robin scheduling based on a reservation system. Digital slice access supports ≥16-way parallel processing and dynamic allocation of computing resources. Conflict resolution utilizes command priority management combined with the CSMA / CA mechanism and anti-collision algorithm for adjustment.

[0139] Typical application scenarios:

[0140] Remote consultation mode: The main control end can circle key areas, overlay multiple annotations in real time, and publish diagnostic opinions.

[0141] Teaching and training mode: Forced synchronization of instructor's perspective (locking key fields of view). Operation process playback (supporting 0.1-2x speed adjustment). Automatic generation of assessment reports (including: lesion identification accuracy, operation path optimization, and diagnostic time analysis).

[0142] Network optimization and disaster recovery solutions

[0143] For example, the core of adaptive transmission technology lies in dynamically adjusting transmission parameters based on network conditions to improve data transmission efficiency and stability. It also includes emergency response for disconnections. In the event of a network outage, emergency response mechanisms are key to ensuring business continuity. This includes an operation command queue cache: operation commands are cached for up to two hours, allowing unfinished operations to be resumed after the network is restored. This mechanism is similar to the dynamic bandwidth allocation strategy used in power emergency satellite communication systems, which ensures business continuity by caching important data. Automatically switching to AI-assisted mode: When a network failure occurs, the system can automatically switch to AI-assisted mode, utilizing artificial intelligence technology for data recovery or alternative operations. For example, in emergency communication scenarios, intelligent traffic classification and adaptive optimization mechanisms can improve network performance. Secondary confirmation of important operations: For operations involving important data, the system should require secondary confirmation from the user to prevent data errors or loss due to network failures. This mechanism is similar to the redundant transmission strategy used in disaster recovery solutions, ensuring data integrity and accuracy through multiple verification steps.

[0144] The design of network optimization and disaster recovery solutions requires integrating adaptive transmission technologies, disconnection emergency handling, and comprehensive optimization strategies. Dynamically adjusting protocol parameters, introducing intelligent bitrate adjustments, and local caching mechanisms can significantly improve network performance and stability. Furthermore, measures such as command caching, AI-assisted modes, and secondary confirmation of critical operations can effectively mitigate the risks posed by network failures. The implementation of these solutions requires flexible adjustments based on specific application scenarios, combining multiple technical approaches to achieve optimal results.

[0145] Performance indicators and verification data

[0146] Laboratory testing (based on a 5G network environment): End-to-end latency: 78 ± 12ms for local operation, 142 ± 23ms for remote operation. Multi-user concurrency: supports 12 operating terminals and 24 observation terminals simultaneously online. Data integrity: low bit error rate. Clinical validation results (300 remote consultations): Diagnostic compliance rate: 98.7% (vs. in-person consultation). Average consultation duration: 23.5 minutes (traditional methods require 52 minutes). User satisfaction: 4.8 / 5.0.

[0147] The innovations of this implementation include: a pioneering "hot migration of operational rights" mechanism, enabling seamless handover of control; a multimodal data layered transmission protocol, improving bandwidth utilization by 65%; a medical-grade concurrency control system, ensuring atomicity of operations; and integrated blockchain evidence storage, meeting medical compliance requirements (FDA 510k certified).

[0148] The system is particularly suitable for: remote or cross-institutional consultation of difficult cases, real-time remote pathology teaching, multi-center joint scientific research projects, and emergency medical support scenarios.

[0149] Example 3: Multi-user synchronous operation

[0150] User A observes the physical slice through local VR glasses, while user B operates the digital slice through a remote device, and the control host allocates resources to ensure there is no conflict.

[0151] The specific operation process is as follows:

[0152] Multi-user access and resource allocation. The system adopts a distributed microservice architecture and supports concurrent multi-user access: User A accesses through local VR glasses (6) and obtains physical slice operation permissions. User B accesses through a remote device (9) and obtains digital slice operation permissions. The control host module (8) dynamically allocates hardware resources: Physical slice channel: exclusive access (with queuing mechanism). Digital slice channel: supports multi-instance parallel processing. GPU computing resources: allocated on demand (CUDA core scheduling).

[0153] Solid slicing process (User A). Robotic arm control subsystem: Utilizes a priority arbitration mechanism (local operation priority + 0.5), motion command verification (collision prevention algorithm), and real-time force feedback (pressure sensor sampling rate 1kHz). Image processing pipeline: Dedicated video encoder (H.264 hard-coded), latency optimization (local loop latency <15ms), and image quality enhancement (GAN-based super-resolution reconstruction).

[0154] Digital Slide Operation Process (User B). Digital Slide Management System: Supports the WSI (Whole Slide Image) standard format. Intelligent preloading (caching adjacent areas based on access patterns). Multi-level pyramid display (seamless switching from 1x to 40x). Remote interaction features: Gesture mapping (converting 2D touch to 3D operations), foveated rendering technology, and network QoS assurance (DSCP Differentiated Services Marking).

[0155] Collaborative work function implementation

[0156] Shared annotation system: 3D annotation tools (supporting volume measurement), multi-user annotation fusion (conflict marking and automatic negotiation), version control (Git-style management). Real-time communication channel: low-latency voice, expression capture, and avatar drive.

[0157] System performance verification

[0158] Stress test results: Maximum concurrent users: 32 (8 entities + 24 numbers). Resource conflict rate: <0.1%. Arbitration decision time: <5ms. Clinical usage data: Teaching scenario: Supports one instructor to simultaneously guide 15 students. Consultation scenario: 5-party collaborative diagnosis efficiency increased by 40%. Research scenario: Multi-person annotation consistency reached 93.5%.

[0159] Adaptive interface: dynamic adjustment of operation complexity, attention assistance prompts (eye tracking), fatigue monitoring and reminders.

[0160] The innovation of this embodiment lies in:

[0161] Develop a hybrid resource scheduling algorithm to achieve unified management of physical and digital resources. Establish a medical operation credit system to optimize multi-user collaboration efficiency. Create a pathology operation conflict prediction model to achieve true multimodal parallel operation.

[0162] Typical application scenarios: interdisciplinary joint diagnosis (pathology + radiology + clinical), large-scale pathology teaching (supporting classrooms with hundreds of people), multi-center scientific research collaboration (real-time data sharing), and implementation of tiered diagnosis and treatment (real-time guidance from higher-level hospitals to grassroots hospitals).

[0163] The present application provides a multi-terminal distributed digital microscope system based on VR glasses, including a digital camera module, an electric microscope objective module, an electric stage module, a slice management module, a data interface module, local / remote VR glasses, an operating joystick and a control host module. By replacing traditional eyepieces and mechanical knobs with electric control, dual-mode observation of physical slides and digital slices is supported: after the physical slices are automatically retrieved by the robotic arm, they are observed at multiple times through the electric objective lens and stage, and the images are displayed through VR glasses; digital slices can be zoomed, translated and imaged. Remote users can observe synchronously in real time through the network and reversely control system operations, reselect the sending time according to the backoff algorithm in the CSMA / CA mechanism, and multiple users can operate different slices in parallel without conflict. It solves the problems of uneven resource distribution, difficulty in remote collaboration, and low operational efficiency in pathological diagnosis, and is suitable for medical diagnosis, teaching and scientific research scenarios.

[0164] Those skilled in the art should recognize that the above embodiments are merely intended to illustrate the present application and are not intended to limit the present application. As long as they are within the spirit of the present application, appropriate changes and modifications to the above embodiments are within the scope of protection claimed in the present application.

Claims

1. A multi-terminal distributed digital microscope system based on VR glasses, characterized in that: The system comprises: A digital camera module (1) for collecting microscope magnified images; An electric microscope objective lens module (2), comprising a rotary drive device for switching objective lenses of different magnifications; The electric stage module (3) includes a three-axis precision moving drive device for front-back, left-right, and up-down, capable of moving in the horizontal plane and fine-moving in the vertical direction for adjusting the observation area and focusing; A data interface module (4) is used to connect the hardware modules with the control host module (8) and transmit control signals and data; A slide management module (5) comprising a barcode scanning camera (52) with text information for identifying slide labels, a robotic arm (51) with an electric suction cup (53), and a slide placement slot (54) for automatically identifying, retrieving, and returning physical slides; Local VR glasses (6) and joysticks (7) for local interactive operation and immersive observation; A control host module (8) is used to coordinate the operation of various modules of the system through control instructions; The remote VR glasses (10) and the remote mobile device (9) are connected to the control host module (8) via a network to achieve remote observation and control; The system eliminates the traditional eyepiece system and mechanical adjustment knobs, and achieves observation through motorized control and VR interaction.

2. The system according to claim 1, wherein: The mechanical arm (51) of the slice management module (5) absorbs the designated physical slide through the electric suction cup (53) and places it on the electric stage module (3) according to the instruction of the control host module (8), and automatically completes the return and replacement of the slice after the observation is completed.

3. The system according to claim 1, wherein: The barcode scanning camera (52) is used to scan and identify the text information or QR code information on the slice label and enter it into the system, thereby realizing automatic identification and management of the slide.

4. The system according to claim 1, wherein: The rotation drive device of the electric microscope objective module (2) and the three-axis precision movement drive device of the electric stage module (3) are both electrically connected to the control host module (8) and receive control instructions from the operating joystick (7) or the remote mobile terminal device (9).

5. The system according to claim 1, wherein: The system supports dual-mode observation of physical slides and digital whole sections, wherein: for physical slides, the slice management module (5) is controlled by operating a joystick (7) or a remote mobile terminal device (9) to retrieve a specific slice, the motorized microscope objective module (2) is controlled to switch the magnification, and the motorized stage module (3) is controlled to move and adjust the observation area; for digital whole sections, the magnification is adjusted by operating a joystick (7) or a remote mobile terminal device (9) to control the software to zoom in and out and to translate in any direction to adjust the observation area.

6. The system according to claim 1, wherein: The system supports simultaneous multi-user operation. Multiple local and remote users can observe and operate different physical or digital slices simultaneously. In order to prevent conflicts between operations, the implementation method is as follows: The local VR glasses (6) share the observation image with the remote VR glasses (10), and the joystick (7) is operated to enjoy the first priority of the operation; The remote user uses a remote mobile terminal device (9) to sense the network allocation vector NAV information terminal allocated by the device through a carrier, and determines whether the communication channel of the network allocation vector NAV information allocated by the device is idle; The control host module (8) determines whether the communication channel is idle, and if so, sends a reply instruction to the remote mobile terminal device (9); if not, uses a collision algorithm to determine when the communication channel is idle, calculates the waiting time, sends the calculation result to the remote mobile terminal device (9), and continues to listen; Among them, the probability of collision is: The current NAV value reflects the remaining time when the channel is busy, and the maximum NAV value is the time when the channel is completely idle. If the NAV value is high, the collision probability is high; The waiting time depends on the rate at which the NAV value decreases and the time it takes for the channel to return to an idle state. The NAV value decreases with the transmission of each data frame. When the NAV value drops to 0, the channel is considered idle, and the collision probability is estimated using the following formula: Among them, V NAV Indicates the rate at which the NAV value decreases per second; Virtual carrier sensing technology determines whether the channel is idle by monitoring the NAV value. When the NAV value is greater than 0, it indicates that the channel is busy. When the NAV value drops to 0, the channel is idle. If the NAV value is higher, it will take longer to retry sending data frames. At this time, the remote mobile terminal device (9) will continue to monitor the channel status and adjust the waiting time according to the change of the NAV value. After a collision occurs, the remote mobile device (9) will reselect the transmission time according to the backoff algorithm in the CSMA / CA mechanism; the backoff algorithm usually adopts an exponential backoff strategy, that is, the backoff time doubles after each collision until the data frame is successfully transmitted; The backoff time is calculated as follows: t tb =2 r ×DIFS, Where r is the number of collisions, and DIFS is the distributed coordination function interval, which is a fixed time interval used to distinguish different types of frame transmissions.

7. The system according to claim 1, wherein: The remote mobile terminal device (9) reversely controls the local control host module (8) through the network, and can operate various modules of the system to complete slice replacement, multiple switching, observation area adjustment and image acquisition operations.

8. The system according to claim 1, wherein: After the image data collected by the digital camera module (1) is transmitted to the control host module (8) via the data interface module (4), it can be displayed on the local VR glasses (6) and the remote VR glasses (10) at the same time.

9. The system according to claim 1, wherein: The operating joystick (7) can control the digital camera module (1) to shoot or record images and videos of the area of ​​interest by controlling the host module (8).

10. The system according to claim 1, wherein: The system supports controlling the operation of each module through voice input.

Images