Intraoperative frozen pathology sampling simulation system based on VR
By introducing data storage, VR identification and processing distribution modules into the VR surgical simulation system, combining VR glasses and speakers, accurate and efficient labeling and step guidance for pathological materials are achieved, and the pathological materials are not accurate enough in the prior art and The time-consuming problem improves the efficiency and accuracy of pathological material extraction.
Patent Information
- Application Number
- CN202410978612.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-12
AI Technical Summary
The existing medical surgical simulation system based on VR technology cannot effectively integrate drill operations and pathological identification during the pathological material collection process, resulting in inaccurate material collection and long time.
A VR-based intraoperative frozen pathological material sampling simulation system is designed, including data storage module, VR identification module and processing and distribution module. Virtual marking and step guidance of pathological material sampling is performed through VR glasses, and real-time use speakers and cameras. Feedback and monitoring to ensure that the pathologist follows the virtual image steps to retrieve materials.
It improves the accuracy and efficiency of pathological materials, reduces the pressure and search time of pathologists when taking materials, reduces the inaccuracy of pathological materials caused by accidents, and reduces the pain to patients.
Smart Images

Figure CN118942298A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of frozen pathology specimen collection, and in particular to a VR-based intraoperative frozen pathology specimen collection simulation system. Background Art
[0002] In the prior art, when it is necessary to obtain pathological samples from the patient's lesion during surgery, the method usually used is to use the doctor's experience and tools such as puncture needles to obtain pathological samples. However, this method has certain deficiencies in the accuracy of sampling, and it takes a long time to find the pathology, which is not convenient for quickly and efficiently obtaining samples from the lesion. With the development and progress of science and technology, the use of VR technology to assist in the collection of pathological samples is becoming a mainstream tumor treatment method. VR technology can be used to quickly mark and indicate the lesion through simulation and virtual imaging, thereby efficiently obtaining samples from the pathology.
[0003] After searching, the Chinese patent application No. 201610891628.4 discloses a medical surgery simulation system based on VR technology, including: a database storage unit: which stores model element information of various medical devices, performance information of the medical devices and operation type information involving the medical devices, a scene construction unit: in the virtual environment area, the data information in the database storage unit is retrieved to perform image and video editing and virtual reality imaging unit for the entire operation process corresponding to each medical operation.
[0004] A medical surgery simulation system based on VR technology in the above patent has the following shortcomings: although the medical surgery can be practiced in a virtual reality environment, the practice operation and the identification of the pathological site cannot be effectively integrated during the operation, that is, when taking pathological samples, the lesions that need to be treated surgically cannot be accurately and quickly calibrated and found, so that even if the operator can perform actual operations skillfully based on VR technology, it still needs to spend time observing the location of the pathological sampling site with the naked eye, which will prolong the time for sampling below the pathological site and the accuracy of sampling at the pathological site. Summary of the invention
[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a VR-based intraoperative frozen pathology sampling simulation system.
[0006] In order to achieve the above object, the present invention adopts the following technical solutions:
[0007] A VR-based intraoperative frozen pathology sampling simulation system includes a data storage module, a VR recognition module, and a processing allocation module:
[0008] The data storage module records and stores tumor characteristics of various tumor types. When storing, the tumor samples taken after pathological sampling are photographed and classified and summarized to establish multiple subfolders.
[0009] The VR recognition module includes VR glasses and a receiver, a camera and a speaker integrated in the VR glasses;
[0010] When the pathologist puts on VR glasses, he uses the processing allocation module to select the nature of the tumor, the matters required for tumor sampling, and the sampling process in the data storage module, and then uses VR glasses to form a complete medical model to demonstrate the complete steps of pathological sampling based on the medical model.
[0011] Preferably: the processing allocation module is used to establish a connection between the data storage module and the VR recognition module, and the processing allocation module specifically includes a selection module, a processing module and a visual feedback module, and the selection module compares the pathological image with the existing tumor image in the data storage module.
[0012] Further: Each subfolder contains technical features including tumor properties, matters required for tumor sampling, sampling process and matters for special lesion sampling. Tumor properties include
[0013] 1: Whether the tumor is benign or malignant and the degree of malignancy;
[0014] 2: The corresponding treatment options for each tumor type, such as surgery, radiotherapy or chemotherapy.
[0015] Further: The items required for tumor sampling include:
[0016] 1: Collect the patient's name, gender, age and bed number;
[0017] 2: Select the tools for sampling, such as tweezers, scissors and surgical knives, and avoid necrotic and bleeding areas when sampling;
[0018] 3: Record the location, size and quantity of the samples.
[0019] As a preferred solution of the present invention: the material collection process includes:
[0020] 1: Sample preparation: Select the specifications of the puncture needle based on the patient's own imaging examination results;
[0021] 2: Under the guidance of the VR recognition module, the lesion site to be sampled is marked using the VR recognition module, and then the biopsy gun is used to insert the puncture needle into the lesion site for sampling.
[0022] As a further solution of the present invention: the special lesion sampling matters include:
[0023] 1: For the sampling of pathological samples of the digestive system, when sampling the esophagus and stomach, cut open along the axis and sample the lesion and the normal tissues at both ends;
[0024] 2: For the collection of pathological samples of the small intestine and large intestine, mesenteric lymph nodes are collected during the sampling process;
[0025] 3: For pathological sampling of the liver and gallbladder, the liver tissue at the pathological site is sampled.
[0026] As a further solution of the present invention: when the VR recognition module is performing pathological sampling, it specifically includes the following steps:
[0027] S1: The pathologist puts the VR glasses on his head;
[0028] S2: selecting the nature of the tumor, matters required for tumor sampling, and sampling process in the terminal device integrated with the data storage module;
[0029] S3: After the selection is completed, the VR recognition module is activated, and the virtual image is projected onto the pathology using VR glasses;
[0030] S4: The virtual image specifically includes a mark for the pathological lesion, and the content of the mark includes a stroke for the pathological lesion, and the stroke uses a colored line to frame the pathological lesion;
[0031] S5: When the number of lesions is more than two groups, different colored lines are used to select and distinguish the pathological parts during the outlining process;
[0032] S6: After the selection is completed, the sampling steps of the lesion are projected by means of virtual images;
[0033] S7: When taking pathological samples from the lesion, the pathologist follows the steps of virtual impact projection;
[0034] S8: When the footage deviates from the established stroke path, the speaker will issue a warning when the path deviation is detected.
[0035] Based on the above scheme: the camera collects images of the pathological sampling process, and the pathologist's technique during the collection process. When the hand and wrist posture are distorted or shaken significantly, the speaker sends an alarm message to prompt the pathologist to adjust the posture.
[0036] On the basis of the above-mentioned scheme: the processing module is used to input the pathological sampling method into the VR recognition module, so that the VR recognition module samples the lesion according to the pathological sampling method.
[0037] Based on the above scheme: the visual feedback module inputs the generated VR audition data into the VR recognition module according to the pathological sampling method.
[0038] The beneficial effects of the present invention are:
[0039] 1. A VR-based intraoperative frozen pathology sampling simulation system. Compared with the traditional method of sampling pathology based on the experience of pathologists, the use of a simulation system to sample pathology reduces the inaccuracy of pathology sampling due to accidents and the inability to accurately identify the pathology.
[0040] 2. A VR-based intraoperative frozen pathology sampling simulation system uses a simulation system to relieve the pathologist's pressure when sampling. By marking and outlining the lesions, the pathologist only needs to follow the marked path when sampling, eliminating the time spent searching for the pathology, thereby greatly improving the efficiency of sampling the lesions.
[0041] 3. A VR-based intraoperative frozen pathology sampling simulation system, in which the processing allocation module is used to establish a connection between the data storage module and the VR recognition module. By setting up a selection module, the pathology image and the existing tumor image in the data storage module can be compared, so as to select the most effective pathology sampling method according to the pathology image. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 It is a schematic diagram of the overall structure of a VR-based intraoperative frozen pathology sampling simulation system proposed by the present invention;
[0043] Figure 2 It is a schematic diagram of the operation flow of a VR recognition module of a VR-based intraoperative frozen pathology sampling simulation system proposed by the present invention. DETAILED DESCRIPTION
[0044] The technical solution of this patent is further described in detail below in conjunction with specific implementation methods.
[0045] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.
[0046] Embodiment 1:
[0047] A VR-based intraoperative frozen pathology sampling simulation system, such as Figure 1 to Figure 2 As shown, it includes a data storage module, a VR recognition module and a processing allocation module;
[0048] The data storage module records and stores tumor characteristics of various tumor types. When storing, the tumor samples taken after pathological sampling are photographed and classified and summarized to establish multiple subfolders. In this embodiment, the data storage module is integrated in the data storage device, and the storage device model is Dahua DH-ESS5048S-R. The technical features contained in each subfolder include:
[0049] Tumor nature:
[0050] 1: Whether the tumor is benign or malignant and the degree of malignancy;
[0051] 2: The corresponding treatment options for each tumor type, such as surgery, radiotherapy or chemotherapy;
[0052] Things you need to take a sample from a tumor:
[0053] 1: Collect the patient's name, gender, age and bed number;
[0054] 2: Select the tools for sampling, such as tweezers, scissors and surgical knives, and avoid necrotic and bleeding areas when sampling;
[0055] 3: Record the location, size and quantity of the samples;
[0056] Collection process:
[0057] 1: Sample preparation: Select the specifications of the puncture needle based on the patient's own imaging examination results;
[0058] 2: Under the guidance of the VR recognition module, the lesion site to be sampled is marked with the VR recognition module, and then the biopsy gun is used to insert the puncture needle into the lesion site for sampling;
[0059] Special lesion sampling matters:
[0060] 1: For the sampling of pathological samples of the digestive system, when sampling the esophagus and stomach, cut open along the axis and sample the lesion and the normal tissues at both ends;
[0061] 2: For the collection of pathological samples of the small intestine and large intestine, mesenteric lymph nodes are collected during the sampling process;
[0062] 3: For pathological sampling of the liver and gallbladder, the liver tissue at the pathological site is sampled.
[0063] The VR recognition module includes VR glasses and a receiver, a camera and a speaker integrated in the VR glasses, wherein the VR glasses are preferably of the brand pico, which has uniform weight distribution, is comfortable to wear and has a high resolution, and can provide a clearer visual experience when sampling pathology.
[0064] When the pathologist wears VR glasses, he uses the processing allocation module to select the nature of the tumor, the matters required for tumor sampling, and the sampling process in the data storage module, and then uses the VR glasses to form a complete medical model. Based on the medical model, the complete steps of sampling the pathology are demonstrated, such as the insertion angle and direction of the puncture needle, the movement trajectory of the puncture needle when sampling the pathology, and the depth of the puncture;
[0065] When sampling, the camera collects images of the pathological sampling process, and monitors the pathologist's techniques, such as the posture of the hand and wrist during sampling. When the posture of the hand and wrist is distorted or shakes significantly, the speaker sends an alarm message to prompt the pathologist to adjust the posture;
[0066] When the VR recognition module is performing pathological sampling, it specifically includes the following steps:
[0067] S1: The pathologist puts the VR glasses on his head;
[0068] S2: selecting the nature of the tumor, matters required for tumor sampling, and sampling process in the terminal device integrated with the data storage module;
[0069] S3: After the selection is completed, the VR recognition module is activated, and the virtual image is projected onto the pathology using VR glasses;
[0070] S4: The virtual image specifically includes a mark for the pathological lesion, and the content of the mark includes a stroke for the pathological lesion, and the stroke uses a colored line to frame the pathological lesion;
[0071] S5: When the number of lesions is more than two groups, different colored lines are used to select and distinguish the pathological parts for different lesions during the outlining process;
[0072] S6: After the selection is completed, the sampling steps of the lesion are projected by means of virtual images;
[0073] S7: When taking pathological samples from the lesion, the pathologist follows the steps of virtual impact projection;
[0074] S8: When the footage deviates from the established stroke path, the speaker will issue a warning when the path deviation is detected.
[0075] The processing allocation module is used to establish a connection between the data storage module and the VR recognition module. The processing allocation module specifically includes a selection module, a processing module and a visual feedback module. The specific model is preferably CI6273BSE017457R1. The selection module compares the pathological image with the existing tumor image in the data storage module, so as to select the most effective pathological sampling method according to the pathological image;
[0076] The processing module is used to input the pathological sampling method into the VR recognition module, so that the VR recognition module samples the lesion according to the pathological sampling method;
[0077] The visual feedback module inputs the generated VR audition data into the VR recognition module according to the pathological sampling method.
[0078] In this embodiment, when pathological sampling is required, compared with the traditional method of sampling based on the experience of pathologists, the simulation system is used to sample the pathology, thereby reducing the possibility of inaccurate sampling due to accidents, inability to accurately identify the pathological site, and damage to blood vessels and arteries during sampling.
[0079] The use of the simulation system relieves the pathologist's pressure when sampling. By marking and outlining the lesions, the pathologist only needs to follow the marked path to sample, eliminating the time spent searching for the pathology. This greatly improves the efficiency of sampling the lesions and effectively reduces the patient's pain during treatment.
[0080] Embodiment 2:
[0081] A VR-based intraoperative frozen pathology sampling simulation system, wherein the data storage module is in operation, and its specific operation process includes the following steps:
[0082] 1: Data generation, that is, the generation of data depends on the archiving of daily patient pathological samples and various data records. For different pathological sections, such as tumor pathological sections, by recording the analysis results of multiple groups of pathological sections, the type, grade and stage of the tumor can be judged;
[0083] 2: Data processing, using the application server to classify and summarize the data and then transmit it to different storage restorers, which are divided into database servers and file restorers according to storage categories;
[0084] 3: Manage data and store it online, near-line, or offline based on the frequency of data access.
[0085] Embodiment 3:
[0086] A VR-based intraoperative frozen pathology sampling simulation system. Before sampling, samplers can conduct practical operation training based on VR recognition modules to help doctors and medical students improve their surgical skills and reduce surgical risks.
[0087] VR recognition modules can be used for training through dynamic simulation before surgery, allowing doctors and medical students to more skillfully operate the various processes of pathological sampling, thereby reducing patient waiting time and speeding up the sampling rate of pathological sites during actual operations;
[0088] When using the VR recognition module to simulate 3D surgical operations based on VR technology, the following hardware and software equipment are required: VR glasses, handles, motion capture equipment, and computers. The simulation operation includes the following steps:
[0089] 1. Based on the real surgical scene, use 3D modeling software (such as Blender, Maya, etc.) to model the surgical scene and accurately restore the structure and organization of the surgical area;
[0090] 2. Use 3D modeling software to model surgical instruments based on their characteristics and functions. During the modeling process, identify and judge the shape, size, and force feedback of the instruments during operation;
[0091] 3. Use motion capture devices (such as Leap Motion, Kinect, etc.) to obtain the user's hand and body movement information. The motion capture device accurately identifies the user's finger and hand movements and feeds them back to the system;
[0092] 4. Design user UI, including the selection of surgical scenes and surgical instruments. The user interface provides accurate visual and force feedback information to increase immersion;
[0093] 5. According to the characteristics of the surgical process, design a simulation algorithm for surgical operations and give real-time feedback based on the user's operations. When the user's operation is inappropriate, the system can remind the user and give operational suggestions;
[0094] 6. Design training modes of different difficulty levels to gradually improve the user's surgical skills. The training modes include factors such as the speed requirements of surgical operations, the selection of instruments, and the complexity of surgical scenes;
[0095] 7. Design data analysis algorithms to evaluate the user's surgical operations during training, and provide users with personalized training suggestions and skill improvement directions by analyzing the evaluation results.
[0096] The above is a preferred specific implementation manner of the present invention, and the protection scope of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by any technician familiar with the field within the technical scope disclosed by the present invention in combination with the prior art or public common sense, within the spirit and principle of the present invention, shall be covered by the protection scope of the present invention.
Claims
1. A VR-based intraoperative frozen pathology sampling simulation system, comprising a data storage module, a VR recognition module and a processing allocation module, characterized in that: The data storage module records and stores tumor characteristics of various tumor types. When storing, the tumor samples taken after pathological sampling are photographed and classified and summarized to establish multiple subfolders. The VR recognition module includes VR glasses and a receiver, a camera and a speaker integrated in the VR glasses; When the pathologist puts on VR glasses, he uses the processing allocation module to select the nature of the tumor, the matters required for tumor sampling, and the sampling process in the data storage module, and then uses VR glasses to form a complete medical model to demonstrate the complete steps of pathological sampling based on the medical model.
2. The VR-based intraoperative frozen pathology sampling simulation system according to claim 1, characterized in that: The processing allocation module is used to establish a connection between the data storage module and the VR recognition module. The processing allocation module specifically includes a selection module, a processing module and a visual feedback module. The selection module compares the pathological image with the existing tumor image in the data storage module.
3. The VR-based intraoperative frozen pathology specimen collection simulation system according to claim 2, characterized in that: The technical features contained in each subfolder include the nature of the tumor, matters required for tumor sampling, sampling process and matters for special lesion sampling. The nature of the tumor includes 1: benign or malignant tumor and the degree of malignancy; 2: The corresponding treatment options for each tumor type, such as surgery, radiotherapy or chemotherapy.
4. The VR-based intraoperative frozen pathology sampling simulation system according to claim 3, characterized in that: Things needed for tumor sampling include: 1: Collect the patient's name, gender, age and bed number; 2: Select the tools for sampling, such as tweezers, scissors and surgical knives, and avoid necrotic and bleeding areas when sampling; 3: Record the location, size and quantity of the samples.
5. The VR-based intraoperative frozen pathology sampling simulation system according to claim 4, characterized in that: The material collection process includes: 1: Sample preparation: Select the specifications of the puncture needle based on the patient's own imaging examination results; 2: Under the guidance of the VR recognition module, the lesion site to be sampled is marked using the VR recognition module, and then the biopsy gun is used to insert the puncture needle into the lesion site for sampling.
6. The VR-based intraoperative frozen pathology sampling simulation system according to claim 5, characterized in that: The special lesion sampling items include: 1: For the sampling of pathological samples of the digestive system, when sampling the esophagus and stomach, cut open along the axis and sample the lesion and the normal tissues at both ends; 2: For the collection of pathological samples of the small intestine and large intestine, mesenteric lymph nodes are collected during the sampling process; 3: For pathological sampling of the liver and gallbladder, the liver tissue at the pathological site is sampled.
7. The VR-based intraoperative frozen pathology sampling simulation system according to claim 6, characterized in that: When the VR recognition module is performing pathological sampling, it specifically includes the following steps: S1: The pathologist puts the VR glasses on his head; S2: selecting the nature of the tumor, matters required for tumor sampling, and sampling process in the terminal device integrated with the data storage module; S3: After the selection is completed, the VR recognition module is activated, and the virtual image is projected onto the pathology using VR glasses; S4: The virtual image specifically includes a mark for the pathological lesion, and the content of the mark includes a stroke for the pathological lesion, and the stroke uses a colored line to frame the pathological lesion; S5: When the number of lesions is more than two groups, different colored lines are used to select and distinguish the pathological parts during the outlining process; S6: After the selection is completed, the sampling steps of the lesion are projected by means of virtual images; S7: When taking pathological samples from the lesion, the pathologist follows the steps of virtual impact projection; S8: When the footage deviates from the established stroke path, the speaker will issue a warning when the path deviation is detected.
8. The VR-based intraoperative frozen pathology specimen collection simulation system according to claim 7, characterized in that: The camera collects images of the pathological sampling process and the pathologist's techniques during the collection process. When the hand and wrist posture are distorted or shaken significantly, the speaker sends an alarm message to prompt the pathologist to adjust the posture.
9. The VR-based intraoperative frozen pathology sampling simulation system according to claim 8, characterized in that: The processing module is used to input the pathological sampling method into the VR recognition module, so that the VR recognition module samples the lesion according to the pathological sampling method.
10. The VR-based intraoperative frozen pathology specimen collection simulation system according to claim 9, characterized in that: The visual feedback module inputs the generated VR audition data into the VR recognition module according to the pathological sampling method.
Citation Information
Patent Citations
Mixed-reality surgical system with physical markers for registration of virtual models
CN112584789A
Medical learning device based on integrating physical and virtual reality with the aim of studying and simulating surgical approaches at anatomical locations
CN113785343A
Image registration method, device, equipment, medium and program product
CN117252799A
Neuroendoscopic surgery visualization system based on mixed reality technology
CN117323002A
Augmented reality soft tissue biopsy and surgery system
US20230120638A1
