Medical diagnosis and treatment system

Through the combination of a head-mounted display device and an operating handle, combined with a camera and a processing module, the fusion interaction of virtual and reality is achieved, solving the problem of inconvenient AR and VR interaction in medical diagnosis and treatment systems, and improving user experience and operational efficiency.

CN223401394UActive Publication Date: 2025-09-30SHENZHEN LONGHUA DISTRICT PEOPLES HOSPITAL
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202422218181.7
Authority / Receiving Office
CN · China
Patent Type
Utility models(China)
Current Assignee / Owner
Filing Date
2024-09-10
Publication Date
2025-09-30
Estimated Expiration
2034-09-10

AI Technical Summary

Technical Problem

The existing AR and VR devices are not highly integrated in medical diagnosis and treatment systems, which makes it difficult for doctors to switch and operate them smoothly during the diagnosis and treatment process, affecting their efficiency and effectiveness in actual applications.

Method used

A combination of a head-mounted display and an operating handle is used. The head-mounted display displays a virtual training scene and a virtual cursor in the field of view. The operating handle generates control signals based on hand movements to interact with the virtual cursor. Combined with the camera and processing module, the eye gaze position and real scene are collected in real time to achieve the fusion of virtual and reality.

Benefits of technology

It provides a more natural, intuitive and convenient way of interaction, improves user concentration and operational efficiency, is particularly suitable for delicate operations in medical scenarios, simplifies the interaction process and enhances user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN223401394U_ABST
    Figure CN223401394U_ABST
Patent Text Reader

Abstract

The utility model discloses a medical diagnosis and treatment system, which relates to medical detection and comprises a head-mounted display device and an operating handle. The head-mounted display device is used for at least displaying a virtual training scene and a virtual cursor arranged in the virtual training scene in a field of view of the head-mounted display device; the operating handle is in communication connection with the head-mounted display device, and the operating handle is used for generating a control signal according to hand actions of a person wearing the head-mounted display device and performing control interaction with the virtual cursor according to the control signal. The utility model aims to reduce the time and labor cost required by the attention test.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The utility model relates to the technical field of medical detection, in particular to a medical diagnosis and treatment system. Background Art

[0002] In recent years, virtual reality technology and augmented reality technology have developed by leaps and bounds and are becoming more and more popular among consumers. At present, the most common application of virtual reality technology on the market is VR (Virtual Reality) glasses, and the application of augmented reality technology is AR (Augmented Reality) glasses. Both VR glasses and AR glasses allow users to wear them on their heads and can create a virtual scene in front of the user's eyes to help the user experience the virtual world through the virtual scene. However, both virtual reality technology and augmented reality technology need to rely on equipment (such as VR glasses and AR glasses) to realize the application. Therefore, when applying virtual reality technology or augmented reality technology, the interaction between the equipment and the user becomes more important.

[0003] While AR and VR technologies have been widely adopted in fields such as entertainment, education, and the military, their adoption in medical diagnosis and treatment systems remains limited. Currently, many medical institutions still rely on traditional diagnostic and treatment methods, and the use of AR and VR in the medical field has yet to become mainstream. This is primarily due to factors such as technological maturity, cost, and physician and patient acceptance.

[0004] Currently, AR and VR devices are often complex to operate and require specialized training to master. Furthermore, existing AR and VR systems are poorly integrated into medical environments, hindering physicians' ability to smoothly switch and operate these technologies during diagnosis and treatment, thus hindering their efficiency and effectiveness in practical applications. Therefore, there is an urgent need for technologies that can simplify AR or VR interactions within medical diagnosis and treatment systems to increase their usability and adoption in medical practice. Utility Model Content

[0005] The main purpose of this utility model is to provide a medical diagnosis and treatment system, which aims to simplify the technology of AR or VR interaction in medical diagnosis and treatment systems and improve their usability and popularity in medical practice.

[0006] To achieve the above objectives, the present invention provides a medical diagnosis and treatment system for medical testing or medical rehabilitation training, the medical diagnosis and treatment system comprising:

[0007] A head-mounted display device, the head-mounted display device being configured to display at least a virtual training scene and a virtual cursor placed in the virtual training scene in a field of view of the head-mounted display device;

[0008] An operating handle is communicatively connected to the head-mounted display device, and is used to generate a control signal according to the hand movement of the person wearing the head-mounted display device, and to control and interact with the virtual cursor according to the control signal.

[0009] In one embodiment, the head-mounted display device includes a head-mounted device body and a first camera, a second camera, an augmented reality (AR) module, a processing module, and a display screen integrated into the head-mounted device body; wherein:

[0010] The first camera is disposed on the inner side of the head mounted device body, and the first camera is configured to collect the eye gaze position of the test user in real time;

[0011] The second camera is arranged on the outside of the head mounted device body, and the second camera is configured to capture the real scene in real time;

[0012] The processing module is in communication with the first camera, and is configured to receive information of the eye gaze position collected in real time by the first camera, construct the virtual cursor, and transmit the information to the display screen;

[0013] The augmented reality AR module is communicatively connected to the processing module and the second camera. The augmented reality AR module is configured to receive the real scene captured in real time by the second camera, construct the virtual training scene, and transmit it to the display screen.

[0014] In one embodiment, the operating handle is in communication with the processing module, and the operating handle is provided with a voice collection module, and the voice collection module is configured to collect voice information of the test user;

[0015] The processing module is further configured to receive the voice information and perform a control interaction with the virtual cursor according to the voice information.

[0016] In one embodiment, the second camera is a panoramic camera, which is configured to capture a panoramic image of the real scene in real time.

[0017] In one embodiment, the panoramic camera is equipped with a motion recognition module that is in communication with the processing module, and the motion recognition module is configured to detect the hand movements of the test user in real time and generate hand control information;

[0018] The processing module is further configured to receive the hand control information and perform a control interaction with the virtual cursor according to the hand control information.

[0019] In one embodiment, the medical diagnosis and treatment system further comprises a memory configured to send the stored historical training data, attention assessment data, and artificial intelligence knowledge inference rules to the processing module;

[0020] The processing module is also configured to receive historical training data, attention assessment data and artificial intelligence knowledge reasoning rules sent by the memory, evaluate the training results, and construct a virtual-reality fusion scene for training guidance.

[0021] In one embodiment, the first camera is an eye tracker.

[0022] In one embodiment, the operating handle is provided with a manipulation button, and the manipulation button includes a force feedback module, and the force feedback module is used to feed back force information to the processing module;

[0023] The processing module is further configured to receive the force information and perform control interaction with the virtual cursor according to the force information.

[0024] In one embodiment, the head-mounted display device transmits information via a wireless connection to a fixed display device or a mobile smart device, wherein the fixed display device includes one of a television, a computer screen, and an outdoor display screen; and the mobile smart device includes one of a tablet computer, a laptop computer, and a mobile phone.

[0025] In one embodiment, the operating handle and the processing module are communicatively connected via a WI-FI module.

[0026] The present invention enables patients to see the virtual training scene directly in their field of vision by using a head-mounted display device. This intuitive display method reduces dependence on external devices, allowing users to interact more naturally and improving user concentration. A virtual cursor is placed in the virtual scene, and patients can control the cursor through simple head movements or limb movements. This interaction method is more intuitive and convenient than traditional keyboards, mice or touch screens, and is particularly suitable for medical scenarios that require fine operations. In addition, through an operating handle that communicates with the head-mounted display device, users can control the control interaction of the virtual cursor through hand movements. This control interaction can be the movement of the virtual cursor or the selection of a function. This synchronization mechanism allows the user's actions to be directly converted into feedback in the virtual environment, thereby providing a more natural and direct interactive experience. In summary, the present invention provides a more natural, intuitive and convenient interaction method by integrating a head-mounted display device and a limb synchronization wearable device, thereby effectively solving the problem of inconvenient AR or VR interaction in medical diagnosis and treatment systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0028] Figure 1 This is a first structural block diagram of a first embodiment of a medical diagnosis and treatment system provided by the present utility model;

[0029] Figure 2 This is a second structural block diagram of the first embodiment of a medical diagnosis and treatment system provided by the present utility model;

[0030] Figure 3 This is the second embodiment of a medical diagnosis and treatment system provided by the present utility model.

[0031] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0033] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0034] In addition, the descriptions of "first," "second," etc. in this utility model are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this utility model.

[0035] See also Figure 1, the utility model proposes a medical diagnosis and treatment system.

[0036] It should be noted that the medical diagnosis and treatment system provided in this embodiment is an attention testing tool that combines AR or VR technology. This system uses AR or VR technology to generate virtual elements in the user's real field of view, thereby providing the user with a more intuitive and interactive testing experience. For example, in an attention test or limb rehabilitation training, the system may generate a series of virtual targets or tasks, requiring the user to complete specific tasks or responses while maintaining concentration. These tasks may include target tracking, memory tasks, reaction time tests, etc.

[0037] like Figure 1 As shown, the medical diagnosis and treatment system includes a head-mounted display device and an operating handle.

[0038] The head-mounted display device is used to display at least a virtual training scene and a virtual cursor placed in the virtual training scene in the field of view of the head-mounted display device.

[0039] It should be noted that head-mounted display devices generally have a built-in display screen that can generate a three-dimensional virtual environment in the user's field of view. This virtual training scene can be a simulated medical environment or a training game environment. Within the virtual training scene, the system will insert a virtual cursor, which is usually a visual indicator for user interaction. It can represent the user's finger, surgical tools, or other interactive devices, used to indicate the location of the operation or the object to be selected. By displaying the virtual training scene in the field of view of the head-mounted display device, the user can be fully immersed in the simulated medical environment, which is crucial for improving the effectiveness of training and user engagement. The use of the virtual cursor provides an intuitive operation method, allowing users to perform complex medical operations through simple visual cues. In addition, the head-mounted display device allows users to directly interact with the virtual environment without having to consider the real scene. Training in the virtual environment can be repeated indefinitely without worrying about the risks and costs of actual operation.

[0040] The operating handle is communicatively connected to the head-mounted display device, and is used to generate a control signal according to the hand movement of the person wearing the head-mounted display device, and to control and interact with the virtual cursor according to the control signal.

[0041] It should be noted that through the operating handle that is communicated with the head-mounted display device, the user can control the control interaction of the virtual cursor through hand movements. This control interaction can be the movement of the virtual cursor or function selection. This synchronization mechanism enables the user's actions to be directly converted into feedback in the virtual environment, thereby providing a more natural and direct interactive experience.

[0042] In this embodiment, by using a head-mounted display device, the patient can directly see the virtual training scene in the field of view. This intuitive display method reduces the dependence on external devices, allowing users to interact more naturally and improving user concentration. A virtual cursor is placed in the virtual scene, and the patient can control the cursor through simple head movements or body movements. This interaction method is more intuitive and convenient than traditional keyboards, mice or touch screens, and is particularly suitable for medical scenarios that require fine operations. In addition, through the operating handle connected to the head-mounted display device, the user can control the control interaction of the virtual cursor through hand movements. This control interaction can be the movement of the virtual cursor or the selection of a function. This synchronization mechanism allows the user's actions to be directly converted into feedback in the virtual environment, thereby providing a more natural and direct interactive experience. In summary, the present invention provides a more natural, intuitive and convenient interaction method by integrating a head-mounted display device and a body synchronization wearable device, thereby effectively solving the problem of inconvenient AR or VR interaction in medical diagnosis and treatment systems.

[0043] Further, see Figure 2 The head-mounted display device includes a head-mounted device body and a first camera, a second camera, an augmented reality (AR) module, a processing module, and a display screen integrated into the head-mounted device body;

[0044] The first camera is arranged on the inner side of the head mounted device body, and the first camera is configured to collect the eye gaze position of the test user in real time.

[0045] It should be noted that the first camera is configured to capture images of the test user's eyes. Through advanced image processing algorithms, it can track the eye gaze point in real time. The collected eye gaze position data is communicated with the augmented reality AR module through the processing module to achieve data sharing and interaction.

[0046] It is understandable that the first camera can be a camera dedicated to an eye tracker: an eye tracker is specially designed for eye tracking, has high resolution and high frame rate, and can accurately capture tiny movements of the eyes, such as changes in pupil size, eyelid movements, etc.; the first camera can also be a smartphone camera, which can be used as a low-cost alternative to capture the eye gaze position through specific eye tracking software.

[0047] The second camera is arranged on the outside of the head-mounted device body, and the second camera is configured to capture the real scene in real time.

[0048] It should be noted that the second camera is configured to capture images of the user's surrounding environment. These images may include the physical environment of the test user, such as room layout, furniture placement, surrounding people, etc. The collected real-life scene data is connected to the augmented reality AR module through the processing module to achieve data sharing and interaction. In addition, the eye gaze position information collected by the first camera can be combined to construct virtual training scenes. These virtual training scenes can be superimposed on the real scene on the display screen to form an augmented reality experience.

[0049] It is understandable that the second camera can be a high-definition camera or a panoramic camera. The high-definition camera provides higher resolution and clearer video quality, can better capture the details of the real scene, and is suitable for applications that require high-definition images; the panoramic camera can capture panoramic images of the real scene in real time, and is very suitable for applications that need to capture panoramic real scenes.

[0050] Furthermore, in order to improve the human-computer interaction capability of the system, the panoramic camera is equipped with a motion recognition module that is communicatively connected to the processing module. The motion recognition module is configured to detect the hand movements of the test user in real time and generate hand control information.

[0051] It can be understood that the motion recognition module generates corresponding control information based on the detected hand movements. This information can be used to control virtual elements in the test environment, such as virtual cursors, test images, etc. The generated control information is connected to the augmented reality AR module and the display module through the processing module to realize data sharing and interaction.

[0052] The operating handle is in communication with the processing module. The operating handle is provided with a voice collection module. The voice collection module is configured to collect voice information of the test user.

[0053] It can be understood that the operating handle is a user interaction device used to receive and respond to user instructions. In the medical diagnosis and treatment system, the design of the operating handle is intended to simplify the user's operation process and provide an intuitive interaction method.

[0054] The voice acquisition module is located inside the shell and is responsible for collecting the voice information of the test user. The main structure may include a microphone, signal processing circuit and voice recognition software, so that the voice acquisition module can capture the user's voice commands in real time and convert them into commands that the system can understand and respond to.

[0055] The processing module is communicatively connected to the first camera, and is configured to receive information on the eye gaze position collected in real time by the first camera, construct a virtual cursor, and transmit it to the display screen; the processing module is also configured to receive the voice information and interact with the virtual cursor based on the voice information.

[0056] It is understandable that the head-mounted display device itself has the ability to produce different patterns and projections on the display screen. Therefore, in addition to the virtual cursor, a test image can also be displayed on the display screen. It should be noted that the test image can be a virtual object of various shapes, colors, or dynamic effects to attract the user's attention or perform a specific attention test task. Of course, the test image can also be a pattern that can serve as an indicator to indicate an object in the real world, or the test image can also be an image with interference factors, thereby increasing the richness of the test, which helps to simulate the real-world attention test environment and provide more comprehensive test results. Through these functions of the test generation module, the medical diagnosis and treatment system can provide a diversified and dynamic test environment. Users can interact with various movable test images on the display screen to complete various attention test tasks. This test environment not only improves the fun and effectiveness of the test, but also makes the test more in line with the user's natural behavior habits, thereby providing users with more comprehensive attention assessment results.

[0057] When an attention test is required, the position of the virtual cursor can be controlled by collecting the test subject's eye movement trajectory and position, so that the virtual cursor can interact with the test image. For example, the virtual cursor can be overlapped with the test image or with the real object indicated by the test image. The user can use the operating handle or input voice to enable the virtual cursor to select the test image or object to generate corresponding feedback, such as visual prompts, sound feedback, etc., to inform the user whether the operation is successful or further instructions are required. At the same time, the system will also record the test results (such as the time spent on selection or the selection accuracy) for subsequent analysis and evaluation.

[0058] It should be noted that the processing module receives eye gaze position information collected by the first camera in real time via a communication connection with the first camera. This information may include the eye's gaze point and gaze trajectory. The processing module uses the eye gaze position information to construct a virtual cursor on the display screen. The virtual cursor is updated in real time based on the user's gaze point, providing an intuitive user interface. The virtual cursor can move with the test user's eye movements. For example, the test user can control the left and right movement of the virtual cursor by moving their eyes up, down, left, and right, and then use the operating handle to select or cancel.

[0059] Furthermore, the processing module uses the received voice information to interact with the virtual cursor, enabling users to move the virtual cursor, select a target, or perform specific actions based on voice commands. The processing module processes the collected eye gaze position information and voice information and generates corresponding feedback, such as visual cues and audio feedback, to inform the user whether the operation was successful or if further instructions are required.

[0060] Furthermore, a speaker is provided on the head-mounted device body.

[0061] It's important to note that when the virtual cursor interacts with the test pattern in a specific way, the speaker triggers a pre-programmed audio signal, such as a beep or voice prompt, to guide the user to the next stage of the test. With the speaker's audio feedback, users can interact with the system not only through vision and touch, but also through hearing, resulting in a richer and more immersive interactive experience.

[0062] The augmented reality AR module is communicatively connected to the processing module and the second camera. The augmented reality AR module is configured to receive the real scene captured in real time by the second camera, construct a virtual training scene, and transmit it to the display screen.

[0063] In the above-mentioned medical diagnosis and treatment system of this embodiment, the first camera is arranged on the inner side of the head-mounted device body, and the first camera is configured to collect the eye gaze position of the test user in real time; the second camera is arranged on the outer side of the head-mounted device body, and the second camera is configured to collect the real scene in real time; the operating handle is communicatively connected to the processing module, and the operating handle is provided with a voice acquisition module, and the voice acquisition module is configured to collect the voice information of the test user; the processing module is communicatively connected to the first camera, and the processing module is configured to receive the information of the eye gaze position collected in real time by the first camera, realize the construction of a virtual cursor, and transmit it to the display screen; the processing module is also configured to receive the voice information and perform interaction with the virtual cursor according to the voice information; the augmented reality AR module is communicatively connected to the processing module and the second camera, and the augmented reality AR module is configured to receive the real scene collected in real time by the second camera, realize the construction of a virtual training scene, and transmit it to the display screen. The medical diagnosis and treatment system provided by this utility model significantly improves the accuracy and efficiency of the test compared with the traditional test method that relies on manual operation. The first camera captures the test user's eye gaze position in real time, allowing the processing module to accurately construct a virtual cursor, reducing errors caused by manual operation in traditional medical diagnosis and treatment systems. The augmented reality (AR) module communicates with the second camera, capturing real-world scenes in real time and constructing virtual training scenes, further improving test accuracy. The processing module receives eye gaze position information captured in real time by the first camera and quickly constructs a virtual cursor, enabling interaction between the virtual cursor and voice information, simplifying the test process and improving operational efficiency. The display screen can display the eye gaze position (i.e., virtual cursor), real-world scenes, and virtual training scenes, providing the test user with intuitive visual feedback and increasing the interest and effectiveness of the test. Furthermore, through the voice acquisition module of the operating handle, the test user can operate through voice commands, simplifying the interaction process and enhancing the tester's user experience.

[0064] See Figure 2 , Figure 2 This is the second embodiment of the medical diagnosis and treatment system of the present invention.

[0065] In order to enhance the user experience of the test user, the head-mounted display device also includes an optomechanical imaging module, which is configured to project the content on the display screen onto the retina of the test user.

[0066] It should be noted that the optomechanical imaging module is configured to convert the content on the display screen into light signals and project them onto the retina of the test user. Through the optomechanical imaging module, the virtual image is seamlessly integrated with objects in the real world, creating an immersive augmented reality experience for the user.

[0067] To facilitate operation, the operating handle is provided with a control button, which includes a force feedback module, and the force feedback module is used to feedback force information to the processing module; the processing module is also configured to receive the force information and perform interaction with the virtual cursor based on the voice information.

[0068] It should be noted that the user can control certain functions or instructions during the test by manipulating buttons. The force feedback module can sense the force applied by the user to the buttons and feed back the force information to the processing module. The processing module analyzes this information to determine the user's intention and operation. The processing module interacts with the virtual cursor based on the received force information and voice information, such as adjusting the position of the virtual cursor or performing specific operations according to the user's operation. For example, the test user can select or cancel an operation by pressing.

[0069] Furthermore, to enhance the learning capability of the attention testing system, the medical diagnosis and treatment system further comprises a memory configured to send the stored historical training data, attention assessment data, and artificial intelligence knowledge inference rules to the processing module;

[0070] The processing module is also configured to receive historical training data, attention assessment data and artificial intelligence knowledge reasoning rules sent by the memory, evaluate the training results, and construct a virtual-reality fusion scene for training guidance.

[0071] It should be noted that through the combination of memory and processing modules, the medical diagnosis and treatment system can provide a data-driven training method. The system can provide users with personalized training suggestions and guidance based on the user's historical performance and attention assessment data, as well as artificial intelligence knowledge reasoning rules, thereby improving the targetedness and effectiveness of training. This data-driven approach helps users gradually improve in attention tests and provides users with a richer and more interactive training experience.

[0072] Furthermore, the head-mounted display device transmits information with a fixed display device or a mobile smart device through a wireless connection, wherein the fixed display device includes one of a television, a computer screen and an outdoor display screen; and the mobile smart device includes one of a tablet computer, a laptop computer and a mobile phone.

[0073] By wirelessly connecting to various display devices, the medical diagnosis and treatment system offers flexible display options to meet the needs of diverse environments and situations. Users can select the appropriate display device based on their specific needs to better display and analyze test results. This flexible connection method helps enhance the fun and effectiveness of testing while also making it more natural for users to conduct tests.

[0074] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made by using the contents of the present invention specification and drawings under the utility model concept, or direct / indirect application in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A medical diagnosis and treatment system for medical testing or medical rehabilitation training, characterized in that: The medical diagnosis and treatment system comprises: A head-mounted display device, the head-mounted display device being configured to display at least a virtual training scene and a virtual cursor placed in the virtual training scene in a field of view of the head-mounted display device; An operating handle is communicatively connected to the head-mounted display device, and is used to generate a control signal according to the hand movement of the person wearing the head-mounted display device, and to control and interact with the virtual cursor according to the control signal.

2. The medical diagnosis and treatment system according to claim 1, wherein: The head-mounted display device includes a head-mounted device body and a first camera, a second camera, an augmented reality (AR) module, a processing module, and a display screen integrated into the head-mounted device body; wherein: The first camera is disposed on the inner side of the head mounted device body, and the first camera is configured to collect the eye gaze position of the test user in real time; The second camera is arranged on the outside of the head mounted device body, and the second camera is configured to capture the real scene in real time; The processing module is in communication with the first camera, and is configured to receive information of the eye gaze position collected in real time by the first camera, construct the virtual cursor, and transmit the information to the display screen; The augmented reality AR module is communicatively connected to the processing module and the second camera. The augmented reality AR module is configured to receive the real scene captured in real time by the second camera, construct the virtual training scene, and transmit it to the display screen.

3. The medical diagnosis and treatment system according to claim 2, wherein: The operating handle is in communication with the processing module, and the operating handle is provided with a voice acquisition module, and the voice acquisition module is configured to acquire voice information of the test user; The processing module is further configured to receive the voice information and perform a control interaction with the virtual cursor according to the voice information.

4. The medical diagnosis and treatment system according to claim 2, wherein: The second camera is a panoramic camera, which is configured to capture a panoramic image of the real scene in real time.

5. The medical diagnosis and treatment system according to claim 4, wherein: The panoramic camera is equipped with a motion recognition module that is in communication with the processing module, and the motion recognition module is configured to detect the hand movements of the test user in real time and generate hand control information; The processing module is further configured to receive the hand control information and perform a control interaction with the virtual cursor according to the hand control information.

6. The medical diagnosis and treatment system according to claim 2, wherein: The medical diagnosis and treatment system further includes a memory configured to send the stored historical training data, attention assessment data, and artificial intelligence knowledge inference rules to the processing module; The processing module is also configured to receive historical training data, attention assessment data and artificial intelligence knowledge reasoning rules sent by the memory, evaluate the training results, and construct a virtual-reality fusion scene for training guidance.

7. The medical diagnosis and treatment system according to claim 2, wherein: The first camera is an eye tracker.

8. The medical diagnosis and treatment system according to claim 2, wherein: The operating handle is provided with a control button, and the control button includes a force feedback module, and the force feedback module is used to feed back force information to the processing module; The processing module is further configured to receive the force information and perform control interaction with the virtual cursor according to the force information.

9. The medical diagnosis and treatment system according to claim 1, wherein: The head-mounted display device transmits data via a wireless connection to a fixed display device or a mobile smart device. The fixed display device includes one of a television, a computer screen, and an outdoor display screen; the mobile smart device includes one of a tablet computer, a laptop computer, and a mobile phone.

10. The medical diagnosis and treatment system according to claim 2, wherein: The operating handle and the processing module are communicatively connected via a WI-FI module.