Pathological image training and education control system based on virtual reality

Abstract

The invention provides a virtual reality-based pathological image training and education control system. The virtual reality-based pathological image training and education control system comprises a simulation scene establishment subsystem, a virtual operation interaction subsystem and a training effect evaluation subsystem. According to the invention, a pathological image is combined with a virtual reality technology, and an innovative training and learning mode is provided for medical education and clinical practice; through the virtual reality equipment, an operator can enter a virtual pathological scene in an immersive manner, interact with a pathological image and simulate actual experimental operation and a pathological diagnosis process, the sense of participation and reality of learning can be increased through immersive learning experience, and the learning effect is improved; the virtual reality equipment can provide abundant interaction functions, an operator selects a specific area, amplifies or shrinks an image, marks and annotations and the like and better observes and analyzes a pathological image, and an interactive operation mode can improve the operational ability and observation skills of a learner; and the effects and efficiency of medical education and clinical practice are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a pathological image training and education control system based on virtual reality. Background Art

[0002] Pathological images are images of human tissues or cells observed and analyzed under a microscope. They are obtained by slicing, staining, and sealing the pathological specimens, and then observing and photographing them under a microscope. Pathological images can be used by pathologists and clinicians to diagnose and identify tissue lesions, including inflammation, tumors, and infections. Pathological images can also be used for scientific research and teaching. With the development of digital technology, pathological images can also be converted into digital form through digital scanning and image processing technology, which is convenient for remote sharing, archiving, and analysis. The analysis of pathological images mainly relies on advanced image processing technology and high-precision hardware configuration. Clinical medical staff make judgments with the help of pathological images. However, traditional pathological image training and education mainly rely on laboratory and classroom environments. Learners need to go to designated locations to learn. Under limited real cases and experimental conditions, this traditional method has some limitations and challenges, such as the scarcity of learning resources, limited practice opportunities for learners, and lack of interactivity and feedback. The development of virtual reality (VR) technology has brought new possibilities for pathological image training and education. VR technology can simulate real environments and scenes, allowing users to interact and experience them in an immersive way. In the field of pathological images, VR technology can be used to create virtual pathological image scenes and laboratory environments, providing a broader, more diverse and interactive learning experience.

[0003] Prior art 1, application number: CN202310327834.2 discloses a teaching and training system, including a teacher terminal module, an intelligent teaching assistant module, a live content loading module and a student terminal module. The live content loading module can receive instructions from the teacher terminal to display 3D material models, teacher 3D images and local courseware, and students can conduct real-time remote learning through virtual reality devices. The intelligent teaching assistant module can conduct knowledge training and classroom tests according to the instructions of the teacher terminal, and can answer students' questions independently to achieve intelligent and personalized teaching. Although useful attempts have been made in the display of 3D medical models through virtual reality technology, and artificial intelligence technology has been used to assist teaching, the effect of medical teaching has been significantly improved; however, it lacks realism and interactivity, and learners cannot truly experience the actual operation and touch, which has certain limitations on the improvement of practical ability.

[0004] Prior art 2, application number: CN201710029745.4 discloses a remote medical education system, including a teaching data collection terminal for collecting actual patient medical data information; a consultation and teaching terminal for teaching doctors to diagnose and treat actual patients and to demonstrate and guide students during the diagnosis and treatment process; a learning and practice terminal for students to learn and practice remotely; and a server terminal respectively connected to the teaching data collection terminal, the consultation and teaching terminal, and the learning and practice terminal. Combining the telemedicine consultation system with the simulation teaching system, collecting patient data in real time to form clinical teaching cases, and combining with the situational teaching method, using virtual reality technology to achieve teaching interaction among students, the real clinical education environment helps to cultivate students' initiative, enthusiasm and creativity in learning. Although it trains students to form correct clinical thinking, maximizes the subjective initiative of learning, significantly improves students' clinical thinking ability and improves teaching effectiveness; however, the accuracy and stability of virtual reality interactive devices still need to be improved. Complex operations may result in misidentification or instability. Virtual reality head-mounted displays have problems such as heavy weight, uncomfortable wearing, visual fatigue, and insufficient precision. Long-term use may affect the comfort and health of learners; the accuracy and sensitivity of virtual reality handles or controllers are limited, especially for some subtle image operations, which may be difficult.

[0005] Prior art three, application number CN202010134466.6 discloses a medical imaging image database education platform, which includes 3 modules, namely, image database establishment module, image database management module and image database use module; by setting up 3 modules for standardized management, it realizes classification by system anatomy, pathology or etiology, subject terms, etc., with a large number of typical images as the main form of expression, with general and clinically operational text descriptions and definitions; although it is convenient for readers to search according to the content of interest, so that readers can deepen their understanding of disease knowledge points and improve the visualization of medical imaging (diagnosis) teaching, the cloud-based virtual reality system may have challenges such as network delay, data security and privacy issues.

[0006] Currently, the prior art 1, prior art 2 and prior art 3 have problems such as insufficient realism and interactivity, limited equipment comfort and accuracy, etc. Therefore, the present invention provides a pathological image training and education control system based on virtual reality, which can provide a large number of pathological image simulations and cases through virtual reality technology, enrich learning resources, and enable learners to be exposed to more diverse cases and situations. Summary of the invention

[0007] In order to solve the above technical problems, the present invention provides a pathological image training and education control system based on virtual reality, comprising:

[0008] The simulation scene establishment subsystem is responsible for obtaining pathological images from the pathological image database, establishing a virtual pathological scene, simulating and displaying the pathological images through the virtual pathological scene, establishing a connection with the virtual pathological scene through virtual reality equipment, and observing and analyzing the pathological images in the virtual pathological scene;

[0009] The virtual operation interaction subsystem is responsible for operating the virtual pathology scene through virtual reality equipment, realizing the interaction between the operator and the pathology image, and simulating the actual experimental operation and pathology diagnosis process; the operation includes selecting a specific area, zooming in or out of the image, marking and annotating;

[0010] The training effectiveness evaluation subsystem is responsible for providing training courses and teaching contents at different levels. It evaluates the operator's skills and knowledge level through interactive evaluation tools, provides real-time feedback and personalized learning suggestions; the training courses and teaching contents include the learning of basic knowledge, the cultivation of pathological image diagnosis skills and case analysis.

[0011] Optionally, the virtual reality device includes a head-mounted display, a handle and a tracker to achieve immersive entry into a virtual pathology scene; the virtual pathology scene includes a pathology laboratory or an operating room; and the pathology image database includes healthy pathology images and diseased pathology images.

[0012] Optional, simulation scenario establishment subsystem, including:

[0013] The target information extraction module is responsible for extracting target information from pathological images. By extracting the information, the tissue structure, cell morphology, abnormal structure, staining effect and context information related to the virtual pathological scene are obtained. The spatial relationship embodied in different pathological images is expressed using the tissue structure, cell morphology, abnormal structure, staining effect and context information.

[0014] The spatial relationship division module is responsible for dividing the spatial relationship into the first constraint relationship and the second constraint relationship. The first constraint relationship is used to extract the relative position relationship of the tissue structure and cells from the pathological image. The second constraint relationship is used to infer the existing pathological changes when analyzing the pathological image. The placement of the tissue structure and cells is determined according to the first constraint relationship. The second constraint relationship in the virtual pathological scene is determined according to prior knowledge. The entire placement of the model of the target information of the virtual pathological scene is completed. Instructions are input in Chinese text format to extract the virtual pathological scene information and the spatial relationship within the virtual pathological scene, complete the model placement, generate a scene description file and visualize it.

[0015] The communication connection establishment module is responsible for establishing a connection with the operating system of the virtual pathology scene through the virtual reality device, and observing and analyzing pathological images in the virtual pathology scene.

[0016] Optionally, the tissue structure of the target information extraction module includes: cells, cell nuclei and cell arrangement, reflecting the normal or abnormal state of the tissue; cell morphology includes cell size, shape, staining properties and distribution of organelles, reflecting the physiological state and pathological changes of cells; abnormal structure includes: abnormal changes in tissue structure caused by tumor formation and inflammation; staining effects include nuclear staining and immunohistochemical staining; context information includes the location of the tissue, surrounding structures and clinical information of the patient.

[0017] Optionally, the first constraint relationship of the spatial relationship division module: there is a specific relative position relationship between the tissue structure and cells in the pathological image, the cell arrangement presents a glandular structure, a duct structure and a cluster structure, and the relative position relationship provides important information about the tissue structure and the cell arrangement; the second constraint relationship: based on reasoning and judgment of medical knowledge and experience, doctors will infer the existing pathological changes based on common sense spatial relationships when analyzing pathological images. Tumors cause disorder of tissue structure and abnormal arrangement of cells.

[0018] Optional, target information extraction module, including:

[0019] The extraction strategy generation submodule is responsible for generating the target information extraction strategy, which is used to identify the tissue structure, cell morphology, abnormal structure, staining effect and context information from the pathological image, so as to realize the search and positioning of the target information; the identification of the pathological image containing all the target information is selected from the target information, and the identification is a sign that can express the identity information;

[0020] The target information matching submodule is responsible for matching the target information of each pathological image in turn. For all selected target information, according to the positional relationship of the selected target information, the following matching is performed in turn: matching of the first type of target information with the second type of target information, matching of the second type of target information or the first type of target information, the second type of target information with the third type of target information, matching of the third type of target information, or the second type of target information, the third type of target information, or the first type of target information, the second type of target information, the third type of target information with the fourth type of target information, and so on, until the fifth type of target information is matched; the first type of target information, the second type of target information, the third type of target information, the fourth type of target information and the fifth type of target information correspond to tissue structure, cell morphology, abnormal structure, staining effect and context information respectively;

[0021] The spatial relationship establishment submodule is responsible for extracting description vectors related to establishing virtual pathology scenes from pathology images according to the target information extraction strategy and matching results, generating global information related to establishing virtual pathology scenes based on the description vectors, and using tissue structure, cell morphology, abnormal structure, staining effect and contextual information to express the spatial relationships embodied in different pathology images.

[0022] Optional, spatial relationship establishment submodule, including:

[0023] The description vector splicing unit is responsible for extracting feature points related to the target information from the pathological image, clustering the obtained feature points using the mean clustering algorithm to obtain the cluster center, using the difference vector to generate the local aggregation description vector of each pathological image, dividing each pathological image into regions, obtaining the number of each feature point and the pixel coordinates, and using the local aggregation description vector to splice and generate the spatial local aggregation description vector of each pathological image;

[0024] The information model building unit is responsible for calculating the main direction of the feature point, matching the main direction with the spatial local aggregation description vector according to the nearest neighbor vector matching method, obtaining the local feature matching result of the pathological image, and building a global information model based on the local feature matching result. The global information model is used to describe the distribution and mutual relationship of different tissues in the virtual pathology scene;

[0025] The spatial relationship display unit is responsible for establishing a visual simulation and display of the spatial relationship embodied in different pathological images based on the description results of the global information model, including tissue structure, cell morphology, abnormal structure, staining effect and contextual information.

[0026] Optional, spatial relationship partitioning module, including:

[0027] The database construction submodule is responsible for receiving the content of the first constraint relationship and the second constraint relationship, converting the content into a feature string, and constructing a spatial relationship database according to the feature string;

[0028] The data storage submodule is responsible for traversing each content of the spatial relationship database, extracting the relative position relationship, pathological changes and index position information of the spatial relationship database from the tissue structure and cells, and storing them in the spatial relationship database;

[0029] The scene generation submodule is responsible for inputting instructions in Chinese text format, extracting virtual pathology scene information and spatial relationships within the virtual pathology scene from the spatial relationship database, completing the model placement, realizing the overall placement of the model of the target information of the virtual pathology scene, generating a scene description file and visualizing it.

[0030] Optional, scene generation submodule, including:

[0031] A vertex data determination unit is responsible for obtaining the shape and structure information of the virtual pathology model, parsing the virtual pathology model file, and obtaining vertex data in the local space, where the vertex data represents the shape and structure of the virtual pathology model;

[0032] A vertex data processing unit is responsible for generating target vertex data by processing vertex data in local space based on vertex data, calculating the convex hull of the virtual pathology model in local space, and calculating the convex hull of the virtual pathology model in world space based on the convex hull in local space;

[0033] The position coordinate determination unit is responsible for determining the placement position of the virtual pathology model in the world space by converting vertex data into coordinates in the world space, extracting the virtual pathology scene information and placing it at the placement position; this is achieved by determining the world space matrix of the virtual pathology model in the world space and multiplying the coordinate value of the target vertex with the world space matrix.

[0034] Optional, virtual operation interaction subsystem, including:

[0035] The function permission setting module is responsible for setting the function permissions of each virtual reality device. The virtual reality device includes a teaching end and a student end. Both the teaching end and the student end are operators. The teaching end is responsible for setting the information of the virtual pathology scene, generating virtual pathology scene control information, and sending the virtual pathology scene control information to the server; the student end is responsible for collecting the student's action and voice signals, generating virtual operation role control information, and sending the virtual operation role control information to the server;

[0036] The control information synchronization module is responsible for controlling the information of the virtual pathology scene and the control information of the virtual operation role, controlling the display scene of the student end, displaying the virtual role, receiving action and voice signals, and synchronizing the display information of the virtual pathology scene to each virtual reality device;

[0037] The control information change module is responsible for receiving the movement and attribute information of the elements in the virtual pathology scene model, and changing the position and attributes of the pathology image in the student-side display scene; the movement and attribute information includes selecting a specific area, zooming in or out of the image, marking and annotation.

[0038] The simulation scene establishment subsystem of the present invention obtains pathological images from the pathological image database, establishes a virtual pathological scene, simulates and displays the pathological images through the virtual pathological scene, establishes a connection with the virtual pathological scene through a virtual reality device, and observes and analyzes the pathological images in the virtual pathological scene; wherein the virtual reality device includes a head-mounted display (VR helmet), a handle, a tracker, etc., to achieve immersive entry into the virtual pathological scene; the virtual pathological scene includes a pathological laboratory or an operating room, etc.; the pathological image database includes healthy pathological images and diseased pathological images; the virtual operation interaction subsystem operates the virtual pathological scene through the virtual reality device to achieve interaction between the operator and the pathological image, and simulates the actual experimental operation and pathological diagnosis process; the operation includes selecting a specific area, zooming in or out of the image, marking and annotating, etc.; the training effect evaluation subsystem provides training courses and teaching contents of different levels, evaluates the skills and knowledge level of the operator through interactive evaluation tools, provides real-time feedback and provides personalized learning suggestions; the training courses and teaching contents include learning basic knowledge, cultivating pathological image diagnosis skills and case analysis, etc.; the above scheme combines pathological images with virtual reality technology, and provides an innovative training and learning method for medical education and clinical practice. The specific significance includes: providing immersive learning experience: through virtual reality equipment, operators can immersively enter the virtual pathology scene, interact with pathology images, and simulate actual experimental operations and pathology diagnosis processes. The immersive learning experience can increase the sense of participation and reality in learning and improve learning effects; providing interactive operation and observation: virtual reality equipment can provide rich interactive functions. Operators can select specific areas, zoom in or out images, mark and annotate, etc., to better observe and analyze pathology images. Interactive operation methods can improve learners' hands-on ability and observation skills; providing personalized learning and evaluation: the training effect evaluation subsystem provides personalized learning suggestions and feedback based on the learners' skills and knowledge levels. Through interactive evaluation tools, the skills and knowledge levels of operators are evaluated in real time to help them continuously improve and improve; providing convenient learning resources and case analysis: the pathology image database contains healthy pathology images and diseased pathology images, providing learners with rich learning resources and case analysis. These resources can help learners learn basic knowledge, cultivate pathology image diagnosis skills, and provide analysis and discussion of actual cases. The solution of combining pathological images with virtual reality technology in this embodiment provides a novel, immersive learning and training method, improves the effectiveness and efficiency of medical education and clinical practice, helps medical learners better understand and apply pathological images, and improves the accuracy and professional level of pathological diagnosis.

[0039] The present invention can provide a large number of pathological image simulations and cases through virtual reality technology, enrich learning resources, and enable learners to be exposed to more diverse cases and situations, thus solving the problem that traditional pathological image teaching usually relies on limited real cases, which may not be sufficient and diverse enough for learners; by using virtual reality technology, laboratory conditions and surgical procedures can be simulated in a virtual environment, allowing learners to perform actual operations and experiments, and improve practical ability, thus solving the problem that traditional pathological image training is usually limited by laboratory equipment and conditions, and learners may not be able to personally participate in real pathological operations and experiments; the virtual reality system provides interactive functions, and learners can actively select, operate and mark images, obtain real-time feedback and guidance, and enhance learning effects. It solves the defect that traditional pathological image teaching is often a passive learning method and learners lack the opportunity to interact with images and provide feedback; the pathological image training and education control system based on virtual reality can be studied at any time and place, and learners can learn according to their own needs and schedule, which solves the defect that traditional pathological image training usually requires learners to go to a specific laboratory or classroom to study, and has certain time and space limitations; the virtual reality-based system can convert pathological images into three-dimensional virtual scenes, allowing students to observe and operate pathological tissue samples in an immersive way, providing a more intuitive and vivid learning experience, and solving the defect that traditional pathological image teaching mainly relies on slides or static images and cannot provide real three-dimensional observation and interactive experience.

[0040] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0041] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0043] Figure 1 This is a block diagram of a pathological image training and education control system based on virtual reality in Example 1 of the present invention;

[0044] Figure 2 Establish a subsystem block diagram for the simulation scenario in Example 2 of the present invention;

[0045] Figure 3 This is a block diagram of a target information extraction module in Embodiment 3 of the present invention;

[0046] Figure 4 A block diagram of a submodule for establishing a spatial relationship in Embodiment 4 of the present invention;

[0047] Figure 5 This is a block diagram of a spatial relationship division module in Embodiment 5 of the present invention;

[0048] Figure 6 This is a block diagram of a scene generation submodule in Embodiment 6 of the present invention;

[0049] Figure 7 This is a block diagram of a virtual operation interaction subsystem in Embodiment 7 of the present invention;

[0050] Figure 8 This is a block diagram of the training effect evaluation subsystem in Example 8 of the present invention. DETAILED DESCRIPTION

[0051] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0052] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0053] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as described in detail. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0054] Example 1: Figure 1 As shown, the embodiment of the present invention provides a pathological image training and education control system based on virtual reality, comprising:

[0055] The simulation scene establishment subsystem is responsible for obtaining pathological images from the pathological image database, establishing a virtual pathological scene, simulating and displaying the pathological images through the virtual pathological scene, establishing a connection with the virtual pathological scene through a virtual reality device, and observing and analyzing the pathological images in the virtual pathological scene; wherein the virtual reality device includes a head-mounted display (VR helmet), a handle, a tracker, etc., to achieve immersive entry into the virtual pathological scene; the virtual pathological scene includes a pathological laboratory or an operating room, etc.; the pathological image database includes healthy pathological images and diseased pathological images;

[0056] The virtual operation interaction subsystem is responsible for operating the virtual pathology scene through virtual reality equipment, realizing the interaction between the operator and the pathology image, and simulating the actual experimental operation and pathology diagnosis process; the operation includes selecting a specific area, zooming in or out of the image, marking and annotating, etc.;

[0057] The training effect evaluation subsystem is responsible for providing training courses and teaching contents at different levels. It evaluates the operator's skills and knowledge level through interactive evaluation tools, provides real-time feedback and provides personalized learning suggestions. The training courses and teaching contents include learning basic knowledge, cultivating pathological image diagnosis skills and case analysis, etc.

[0058] The working principle and beneficial effects of the above technical solution are as follows: the simulation scene establishment subsystem of this embodiment obtains pathological images from the pathological image database, establishes a virtual pathological scene, simulates and displays the pathological images through the virtual pathological scene, establishes a connection with the virtual pathological scene through a virtual reality device, and observes and analyzes the pathological images in the virtual pathological scene; wherein the virtual reality device includes a head-mounted display (VR helmet), a handle, a tracker, etc., to achieve immersive entry into the virtual pathological scene; the virtual pathological scene includes a pathological laboratory or an operating room, etc.; the pathological image database includes healthy pathological images and diseased pathological images; the virtual operation interaction subsystem operates the virtual pathological scene through the virtual reality device to achieve interaction between the operator and the pathological image, simulating the actual experimental operation and pathological diagnosis process; the operation includes selecting a specific area, zooming in or out of the image, marking and annotating, etc.; the training effect evaluation subsystem provides training courses and teaching contents at different levels, evaluates the operator's skills and knowledge level through interactive evaluation tools, provides real-time feedback and provides personalized learning suggestions; the training courses and teaching contents include learning basic knowledge, cultivating pathological image diagnosis skills and case analysis, etc.; the above solution combines pathological images with virtual reality technology to provide an innovative training and learning method for medical education and clinical practice. The specific significance includes: providing immersive learning experience: through virtual reality equipment, operators can immersively enter the virtual pathology scene, interact with pathology images, and simulate actual experimental operations and pathology diagnosis processes. The immersive learning experience can increase the sense of participation and reality in learning and improve learning effects; providing interactive operation and observation: virtual reality equipment can provide rich interactive functions. Operators can select specific areas, zoom in or out images, mark and annotate, etc., to better observe and analyze pathology images. Interactive operation methods can improve learners' hands-on ability and observation skills; providing personalized learning and evaluation: the training effect evaluation subsystem provides personalized learning suggestions and feedback based on the learners' skills and knowledge levels. Through interactive evaluation tools, the skills and knowledge levels of operators are evaluated in real time to help them continuously improve and improve; providing convenient learning resources and case analysis: the pathology image database contains healthy pathology images and diseased pathology images, providing learners with rich learning resources and case analysis. These resources can help learners learn basic knowledge, cultivate pathology image diagnosis skills, and provide analysis and discussion of actual cases. The solution of combining pathological images with virtual reality technology in this embodiment provides a novel, immersive learning and training method, improves the effectiveness and efficiency of medical education and clinical practice, helps medical learners better understand and apply pathological images, and improves the accuracy and professional level of pathological diagnosis.

[0059] This embodiment uses virtual reality technology to provide a large number of pathological image simulations and cases, enrich learning resources, and enable learners to be exposed to more diverse cases and situations, which solves the defect that traditional pathological image teaching usually relies on limited real cases, which may not be sufficient and diverse enough for learners; virtual reality technology can be used to simulate laboratory conditions and surgical procedures in a virtual environment, allowing learners to perform actual operations and experiments and improve practical skills, which solves the defect that traditional pathological image training is usually limited by laboratory equipment and conditions, and learners may not be able to personally participate in real pathological operations and experiments; the virtual reality system provides interactive functions, and learners can actively select, operate and mark images, obtain real-time feedback and guidance, and enhance learning effects , which solves the defect that traditional pathological image teaching is often a passive learning method and learners lack the opportunity to interact with images and provide feedback; the pathological image training and education control system based on virtual reality can be studied at any time and place, and learners can learn according to their own needs and schedule, which solves the defect that traditional pathological image training usually requires learners to go to a specific laboratory or classroom to study, and has certain time and space limitations; the virtual reality-based system can convert pathological images into three-dimensional virtual scenes, allowing students to observe and operate pathological tissue samples in an immersive way, providing a more intuitive and vivid learning experience, and solving the defect that traditional pathological image teaching mainly relies on slides or static images and cannot provide real three-dimensional observation and interactive experience.

[0060] This embodiment provides an immersive learning environment: the system based on virtual reality can provide students with an immersive learning environment, making them feel like they are actually standing in a pathology laboratory for observation and diagnosis. Students can move freely, zoom in and out of images, observe details, and perform interactive operations such as annotation and measurement, thereby enhancing the sense of participation and effect of learning; provide diverse learning scenarios: virtual reality technology can simulate various pathology image learning scenarios, such as observation of tumor sections, identification of tissue structures, and location of lesion areas, etc. Students can choose different scenarios for training and practice according to their learning needs to improve their ability to diagnose different pathological conditions; provide personalized learning and evaluation: the system based on virtual reality can provide personalized learning content and evaluation methods according to students' learning progress and abilities. The system can give immediate feedback and guidance based on students' performance to help them correct mistakes and improve their learning effects; promote cooperation and remote teaching: the virtual reality system can realize multi-person collaborative learning and remote teaching. Students can interact and discuss with other students or teachers through the system, solve problems together, and improve their teamwork ability; at the same time, remote students can also participate in real-time pathology image training through virtual reality technology to break through geographical restrictions; the combined application of virtual reality technology and digital pathology images constructs complete educational content and learning resources to meet the needs of medical learners. Accurately simulate the details of pathological images and realize the function of interacting with them, effectively apply virtual reality technology to pathological image training, and provide realistic visual and tactile experience.

[0061] Example 2: Figure 2 As shown, based on Example 1, the simulation scene establishment subsystem provided by the embodiment of the present invention includes:

[0062] The target information extraction module is responsible for extracting target information from pathological images. It obtains tissue structure, cell morphology, abnormal structure, staining effect and context information related to the virtual pathological scene through extraction, and uses tissue structure, cell morphology, abnormal structure, staining effect and context information to express the spatial relationship reflected in different pathological images; among them, tissue structure: the tissue structure in the pathological image is the most basic element, including cells, cell nuclei and cell arrangement, etc., reflecting the normal or abnormal state of the tissue; cell morphology: the cell morphological characteristics in the pathological image are an important basis for diagnosis and analysis. Cell morphology includes cell size, shape, staining properties and distribution of organelles, etc., reflecting the physiological state of the cell. and pathological changes; Abnormal structures: Abnormal structures that may exist in pathological images are important clues for diagnosing diseases. For example, the formation of tumors and the occurrence of inflammation can lead to abnormal changes in tissue structure, and these abnormal structures can be observed in the images; Staining effects: In pathological images, staining techniques are often used to highlight specific structures or cell components. Different staining methods and effects can provide different information, such as nuclear staining and immunohistochemical staining; Contextual information: The contextual information of pathological images refers to the background information related to the pathological images, such as the location of the tissue, the surrounding structures, and the patient's clinical information. This information helps doctors to analyze and diagnose pathological images more accurately;

[0063] The spatial relationship division module is responsible for dividing the spatial relationship into the first constraint relationship and the second constraint relationship. The first constraint relationship extracts the relative position relationship of the tissue structure and the cell from the pathological image. The second constraint relationship infers the possible pathological changes according to the first constraint relationship when analyzing the pathological image; the placement of the tissue structure and the cell is determined according to the first constraint relationship, the second constraint relationship in the virtual pathological scene is determined according to the prior knowledge, the entire placement of the model of the target information of the virtual pathological scene is completed, the instructions are input in Chinese text, the virtual pathological scene information and the spatial relationship in the virtual pathological scene are extracted, the model position is placed, and the scene description file is generated and visualized; Among them, the first constraint relationship: there is a specific relative position relationship between the tissue structure and the cell in the pathological image. For example, the cell arrangement can present glandular structure, duct structure and cluster structure, etc. These relative position relationships can provide important information about the tissue structure and cell arrangement, which is helpful for judging and classifying the disease; the second constraint relationship: based on the reasoning and judgment of medical knowledge and experience, doctors will infer possible pathological changes according to common sense spatial relationships when analyzing pathological images. For example, tumors usually cause disorder of tissue structure and abnormal arrangement of cells. This common sense spatial relationship can help doctors identify possible tumor lesions;

[0064] The communication connection establishment module is responsible for establishing a connection with the operating system of the virtual pathology scene through the virtual reality device, and observing and analyzing the pathology images in the virtual pathology scene;

[0065] The working principle and beneficial effects of the above technical solution are as follows: the target information extraction module of this embodiment extracts target information from the pathological image, obtains the tissue structure, cell morphology, abnormal structure, staining effect and context information related to the virtual pathological scene by extraction, and uses the tissue structure, cell morphology, abnormal structure, staining effect and context information to express the spatial relationship embodied in different pathological images; wherein, tissue structure: the tissue structure in the pathological image is the most basic element, including cells, cell nuclei and cell arrangement, etc., reflecting the normal or abnormal state of the tissue; cell morphology: the cell morphological characteristics in the pathological image are an important basis for diagnosis and analysis. Cell morphology includes cell size, shape, staining properties and distribution of organelles, etc., reflecting Reflect the physiological state and pathological changes of cells; Abnormal structure: The abnormal structure that may exist in the pathological image is an important clue for diagnosing the disease. For example, the formation of tumors and the occurrence of inflammation will lead to abnormal changes in tissue structure, and these abnormal structures can be observed in the image; Staining effect: In pathological images, staining technology is often used to highlight specific structures or cell components. Different staining methods and effects can provide different information, such as nuclear staining and immunohistochemical staining; Contextual information: The contextual information of pathological images refers to the background information related to the pathological image, such as the location of the tissue, the surrounding structure and the patient's clinical information, etc. This information helps doctors to analyze and diagnose the pathological image more accurately; Spatial relationship The system division module divides the spatial relationship into a first constraint relationship and a second constraint relationship. The first constraint relationship extracts the relative position relationship between the tissue structure and the cell from the pathological image. The second constraint relationship infers the possible pathological changes according to the first constraint relationship when analyzing the pathological image; the placement of the tissue structure and the cell is determined according to the first constraint relationship, and the second constraint relationship in the virtual pathological scene is determined according to prior knowledge, and the entire placement of the model of the target information of the virtual pathological scene is completed. The command is input in Chinese text format, the virtual pathological scene information and the spatial relationship within the virtual pathological scene are extracted, the model position is placed, and the scene description file is generated and visualized; Among them, the first constraint relationship: the existence between the tissue structure and the cell in the pathological image In specific relative position relationships, for example, cell arrangement can present glandular structure, duct structure and cluster structure, etc. These relative position relationships can provide important information about tissue structure and cell arrangement, which is helpful for judging and classifying diseases; Second constraint relationship: Based on the reasoning and judgment of medical knowledge and experience, doctors will infer possible pathological changes based on common sense spatial relationships when analyzing pathological images. For example, tumors usually lead to disordered tissue structure and abnormal arrangement of cells. This common sense spatial relationship can help doctors identify possible tumor lesions; The communication connection establishment module establishes a connection with the operating system of the virtual pathology scene through the virtual reality device, and observes and analyzes pathological images in the virtual pathology scene;The above scheme analyzes and extracts the pathological image through the target information extraction module, obtains the tissue structure, cell morphology, abnormal structure, staining effect and context information related to the virtual pathological scene, and divides the spatial relationship into the first constraint relationship and the second constraint relationship through the spatial relationship division module, which helps doctors to analyze and diagnose pathological images more accurately and provide important diagnostic clues and auxiliary information; at the same time, through the communication connection establishment module, the virtual pathological scene can be connected to the virtual reality device, so that the doctor can observe and analyze the pathological image in the virtual scene, and improve the accuracy and efficiency of diagnosis. This embodiment provides a pathological image analysis and diagnosis method based on virtual reality technology, which provides doctors with better tools and platforms, promotes the development of pathology and improves the level of diagnosis. ;

[0066] Example 3: Figure 3 As shown, based on Example 2, the target information extraction module provided by the embodiment of the present invention includes:

[0067] The extraction strategy generation submodule is responsible for generating the target information extraction strategy, which is used to identify the tissue structure, cell morphology, abnormal structure, staining effect and context information from the pathological image, so as to realize the search and positioning of the target information; the identification of the pathological image containing all the target information is selected from the target information, and the identification is a sign that can express the identity information;

[0068] The target information matching submodule is responsible for matching the target information of each pathological image in turn. For all selected target information, according to the positional relationship of the selected target information, the following matching is performed in turn: matching of the first type of target information with the second type of target information, matching of the second type of target information or the first type of target information, the second type of target information with the third type of target information, matching of the third type of target information, or the second type of target information, the third type of target information, or the first type of target information, the second type of target information, the third type of target information with the fourth type of target information, and so on, until the fifth type of target information is matched; the first type of target information, the second type of target information, the third type of target information, the fourth type of target information and the fifth type of target information correspond to tissue structure, cell morphology, abnormal structure, staining effect and context information respectively;

[0069] The spatial relationship establishment submodule is responsible for extracting description vectors related to establishing virtual pathology scenes from pathology images according to the target information extraction strategy and matching results, generating global information related to establishing virtual pathology scenes based on the description vectors, and using tissue structure, cell morphology, abnormal structure, staining effect and context information to express the spatial relationships embodied in different pathology images;

[0070] The working principle and beneficial effects of the above technical solution are as follows: the extraction strategy generation submodule of this embodiment generates a target information extraction strategy, which is used to identify tissue structure, cell morphology, abnormal structure, staining effect and context information from pathological images to achieve the search and positioning of target information; the identification of the pathological image containing all target information is selected from the target information, and the identification is a sign that can express identity information; the target information matching submodule matches the target information of each pathological image in turn, and for all the selected target information, the following matching is performed in turn according to the positional relationship of the selected target information: matching of the first type of target information with the second type of target information, and matching of the second type of target information with the target information of the first type. The first target information, the second target information, the third target information and the fourth target information are matched, and so on and so forth until the fifth target information is matched; the first target information, the second target information, the third target information, the fourth target information and the fifth target information correspond to tissue structure, cell morphology, abnormal structure, staining effect and context information respectively; the spatial relationship establishment submodule extracts information involved in establishing a virtual pathology scene from the pathological image according to the target information extraction strategy and the matching results. Description vectors are generated based on description vectors to establish global information of virtual pathology scenes, and the spatial relationships embodied in different pathology images are expressed using tissue structure, cell morphology, abnormal structure, staining effect and context information; the above scheme realizes automated pathology image analysis and recognition, and extracts and recognizes target information such as tissue structure, cell morphology, abnormal structure, staining effect and context information from pathology images through extraction strategy generation submodules and target information matching submodules, so as to realize the search and positioning of target information; through the spatial relationship establishment submodule, description vectors and global information are generated according to the extracted target information and matching results, so as to express the spatial relationships embodied in different pathology images; Improve efficiency: Traditional pathological image analysis and recognition require manual work, which is time-consuming and labor-intensive. However, this embodiment can automatically extract and identify target information in pathological images, greatly improving the efficiency of analysis and recognition. Improve accuracy: Manual analysis often has subjectivity and subjective errors, while automated analysis and recognition solutions can reduce the interference of subjective factors and improve accuracy. Promote research and application: Automated pathological image analysis and recognition solutions provide more data and information for pathological research and clinical applications, and promote research and application in related fields. Assist diagnosis and treatment: Through automated analysis and recognition, doctors are assisted in making diagnosis and treatment decisions for pathological images, improving medical quality and effectiveness. This embodiment realizes automated pathological image analysis and recognition, improves efficiency and accuracy, promotes research and application, and assists diagnosis and treatment.

[0071] Example 4: Figure 4As shown, based on Example 3, the spatial relationship establishment submodule provided in this embodiment of the present invention includes:

[0072] The description vector splicing unit is responsible for extracting feature points related to the target information from the pathological image, clustering the obtained feature points using the mean clustering algorithm to obtain the cluster center, using the difference vector to generate the local aggregation description vector of each pathological image, dividing each pathological image into regions, obtaining the number of each feature point and the pixel coordinates, and using the local aggregation description vector to splice and generate the spatial local aggregation description vector of each pathological image;

[0073] The information model building unit is responsible for calculating the main direction of the feature point, matching the main direction with the spatial local aggregation description vector according to the nearest neighbor vector matching method, obtaining the local feature matching result of the pathological image, and building a global information model based on the local feature matching result. The global information model is used to describe the distribution and mutual relationship of different tissues in the virtual pathology scene;

[0074] The spatial relationship display unit is responsible for establishing the visualization simulation and display of the spatial relationship embodied in different pathological images by expressing tissue structure, cell morphology, abnormal structure, staining effect and context information based on the description results of the global information model;

[0075] The working principle and beneficial effects of the above technical scheme are as follows: the description vector splicing unit of this embodiment extracts feature points related to the target information from the pathological image, clusters the obtained feature points using the mean clustering algorithm to obtain the cluster center, generates a local aggregation description vector for each pathological image using the difference vector, divides each pathological image into regions, obtains the number of each feature point and the pixel coordinates, and splices and generates a spatial local aggregation description vector for each pathological image using the local aggregation description vector; the information model establishment unit calculates the main direction of the feature point, matches the main direction with the spatial local aggregation description vector according to the nearest vector matching method, obtains the local feature matching result of the pathological image, and establishes a global information model based on the local feature matching result, which is used to describe the distribution and mutual relationship of different tissues in the virtual pathological scene; the spatial relationship display unit establishes a visual simulation and display of the spatial relationship embodied in different pathological images by tissue structure, cell morphology, abnormal structure, staining effect and context information expression according to the description result of the global information model; the above scheme adopts a comprehensive method to process pathological images and extract useful information therefrom; specifically: feature extraction and clustering: by extracting feature points related to the target information from the pathological image, And use the mean clustering algorithm to cluster the feature points to obtain the cluster center, which helps to extract key features from complex image data, so as to better understand and study pathological images; local aggregation description vector: by using the difference vector and the number of feature points and pixel coordinates, the local aggregation description vector of each pathological image is generated, and the local information of the image is encoded and represented to facilitate subsequent analysis and matching; main direction calculation and matching: by calculating the main direction of the feature point, and using the nearest vector matching method to match the main direction with the local aggregation description vector, the local feature matching result of the pathological image is obtained, which is helpful for different pathological Find similar local structures or features in the image, thereby helping doctors to diagnose diseases and conduct pathological analysis; Global information model: Based on the local feature matching results, a global information model is established to describe the distribution and mutual relationship of different tissues in the virtual pathological scene, providing more information for the overall analysis and understanding of pathological images, and helping to reveal the development and change of pathology; Spatial relationship display: Based on the description results of the global information model, a visual simulation and display of tissue structure, cell morphology, abnormal structure, staining effect and contextual information is established to help doctors observe and analyze pathological images more intuitively and provide more accurate diagnosis and treatment suggestions. This embodiment provides a comprehensive and effective method to process pathological images and extract useful information from them, providing support for pathological research and clinical practice.

[0076] Example 5: Figure 5 As shown, based on Example 2, the spatial relationship division module provided by the embodiment of the present invention includes:

[0077] The database construction submodule is responsible for receiving the content of the first constraint relationship and the second constraint relationship, converting the content into a feature string, and constructing a spatial relationship database according to the feature string;

[0078] The data storage submodule is responsible for traversing each content of the spatial relationship database, extracting the relative position relationship, pathological changes and index position information of the spatial relationship database from the tissue structure and cells, and storing them in the spatial relationship database;

[0079] The scene generation submodule is responsible for inputting instructions in Chinese text format, extracting virtual pathology scene information and spatial relationships within the virtual pathology scene from the spatial relationship database, completing the model placement, realizing the full placement of the model of the target information of the virtual pathology scene, generating a scene description file and visualizing it;

[0080] The working principle and beneficial effects of the above technical solution are as follows: the database construction submodule of this embodiment receives the content of the first constraint relationship and the second constraint relationship, converts the content into a feature string, and constructs a spatial relationship database according to the feature string; the data storage submodule traverses each content of the spatial relationship database, extracts the relative position relationship, pathological changes and index position information of the spatial relationship database from the tissue structure and cells, and stores them in the spatial relationship database; the scene generation submodule inputs instructions in Chinese text form, extracts virtual pathological scene information and spatial relationships within the virtual pathological scene from the spatial relationship database, completes the model position placement, realizes the full placement of the model of the target information of the virtual pathological scene, generates a scene description file and visualizes it; this embodiment establishes a system that can construct a spatial relationship database according to the constraint relationship and extract virtual pathological scene information from the database, which can help pathology researchers quickly build virtual pathology scenes and visualize pathological changes and spatial relationships; it is of great significance for pathology research, disease diagnosis and treatment, and by using this system, researchers can better understand pathological changes and spatial relationships, and further improve the diagnosis and treatment effects of diseases; in addition, this system can also be used for teaching and training to help medical students and doctors better understand and learn pathology knowledge.

[0081] Example 6: Figure 6 As shown, based on Example 5, the scene generation submodule provided in the embodiment of the present invention includes:

[0082] A vertex data determination unit is responsible for obtaining the shape and structure information of the virtual pathology model, parsing the virtual pathology model file, and obtaining vertex data in the local space, where the vertex data represents the shape and structure of the virtual pathology model;

[0083] A vertex data processing unit is responsible for generating target vertex data by processing vertex data in local space based on vertex data, calculating the convex hull of the virtual pathology model in local space, and calculating the convex hull of the virtual pathology model in world space based on the convex hull in local space;

[0084] The position coordinate determination unit is responsible for determining the placement position of the virtual pathology model in the world space by converting vertex data into coordinates in the world space, extracting the virtual pathology scene information and placing it at the placement position; this is achieved by determining the world space matrix of the virtual pathology model in the world space and multiplying the coordinate value of the target vertex with the world space matrix;

[0085] The working principle and beneficial effects of the above technical scheme are as follows: the vertex data determination unit of this embodiment obtains the shape and structural information of the virtual pathology model, parses the virtual pathology model file, and obtains the vertex data in the local space, where the vertex data represents the shape and structure of the virtual pathology model; the vertex data processing unit generates target vertex data based on the vertex data by processing the vertex data in the local space, calculates the convex hull of the virtual pathology model in the local space, and calculates the convex hull of the virtual pathology model in the world space based on the convex hull in the local space; the position coordinate determination unit determines the placement position of the virtual pathology model in the world space by converting the vertex data into coordinates in the world space, extracts the virtual pathology scene information and places it at the placement position; this is achieved by determining the world space matrix of the virtual pathology model in the world space and multiplying the coordinate value of the target vertex with the world space matrix; the above scheme uses the vertex data and the position coordinate determination unit to obtain the shape, structure and placement position of the virtual pathology model The virtual pathology model is a virtual pathology model that is used for training and education of pathology images. ...

[0086] Example 7: Figure 7 As shown, based on Example 1, the virtual operation interaction subsystem provided by the embodiment of the present invention includes:

[0087] The function permission setting module is responsible for setting the function permissions of each virtual reality device. The virtual reality device includes a teaching end and a student end. Both the teaching end and the student end are operators. The teaching end is responsible for setting the information of the virtual pathology scene, generating virtual pathology scene control information, and sending the virtual pathology scene control information to the server; the student end is responsible for collecting the student's action and voice signals, generating virtual operation role control information, and sending the virtual operation role control information to the server;

[0088] The control information synchronization module is responsible for controlling the information of the virtual pathology scene and the control information of the virtual operation role, controlling the display scene of the student end, displaying the virtual role, receiving action and voice signals, and synchronizing the display information of the virtual pathology scene to each virtual reality device;

[0089] The control information change module is responsible for receiving the movement and attribute information of the elements in the virtual pathology scene model, and changing the position and attributes of the pathology image in the student-side display scene; the movement and attribute information includes selecting a specific area, zooming in or out of the image, marking and annotating, etc.;

[0090] The working principle and beneficial effects of the above technical solution are as follows: the function permission setting module of this embodiment sets the function permission of each virtual reality device. The virtual reality device includes a teaching end and a student end. Both the teaching end and the student end are operators. The teaching end is responsible for setting the information of the virtual pathology scene, generating virtual pathology scene control information, and sending the virtual pathology scene control information to the server; the student end is responsible for collecting the student's action and voice signals, generating virtual operation role control information, and sending the virtual operation role control information to the server; the control information synchronization module controls the information of the virtual pathology scene and the virtual operation role control information, controls the display scene of the student end, displays the virtual character, receives action and voice signals, and synchronizes the display information of the virtual pathology scene to each virtual reality device; the control information change module receives the movement and attribute information of the elements in the virtual pathology scene model, and changes the position and attributes of the pathological image in the student end display scene; the movement and attribute information includes selecting a specific area, zooming in or out of the image, marking and annotating, etc.; the above solution realizes virtual The functional authority setting of the real device enables the teaching end and the student end to assume different roles and tasks respectively. The teaching end sets the information of the virtual pathology scene and generates control information to send to the server, thereby realizing the control of the virtual pathology scene. The student end is responsible for collecting the student's movements and voice signals, and generating virtual operation role control information to send to the server, so as to realize the student's interactive operation of the virtual pathology scene; the control information synchronization module can control the information of the virtual pathology scene and the control information of the virtual operation role, control the display scene of the student end and the display of the virtual role, and can also receive the student's movements and voice signals, and synchronize the display information of the virtual pathology scene to each virtual reality device, so as to realize the sharing and collaborative operation of the virtual pathology scene; the control information change module can receive the movement and attribute information of the elements in the virtual pathology scene model, and change the position and attributes of the pathology image in the student end display scene. The movement and attribute information includes operations such as selecting specific areas, zooming in or out of images, marking and annotating, etc., to help students better understand and analyze pathology images. This embodiment provides a teaching platform based on virtual reality technology, which can realize the control, interaction and sharing of virtual pathology scenes, improve students' practical ability and learning effect in pathology learning; at the same time, it can also provide teachers with more intuitive and vivid teaching tools to improve teaching effects.

[0091] Example 8: Figure 8 As shown, based on Example 1, the training effect evaluation subsystem provided by the embodiment of the present invention includes:

[0092] The difficulty level setting module is responsible for setting multiple levels according to the difficulty of the training courses and teaching contents. From easy to difficult, they are set as level 1, level 2, level 3, level 4 and level 5. Each level is further divided into three sub-levels. The difficulty is set by the operator according to the actual situation.

[0093] The exercise effect evaluation module is responsible for presenting the effects of multiple interactive evaluation exercises, where each interactive evaluation exercise displays the operation process and judgment results, receives responses from multiple interactive evaluation exercises, and obtains the operator's skills and knowledge level;

[0094] The learning suggestion provision module is responsible for searching for operator deficiencies based on responses and providing personalized learning suggestions based on training courses and teaching contents;

[0095] The working principle and beneficial effects of the above technical solution are as follows: the difficulty level setting module of this embodiment sets multiple levels according to the difficulty of the training course and teaching content, and is set from easy to difficult to level one, level two, level three, level four and level five, and three sub-levels are set under each level; wherein the difficulty is set by the operator according to the actual situation; the exercise effect evaluation module presents the effects of multiple interactive evaluation exercises, wherein each interactive evaluation exercise displays the operation process and judgment results, receives responses from multiple interactive evaluation exercises, and obtains the operator's skills and knowledge level; the learning suggestion providing module searches for the operator's defects according to the responses, and provides personalized learning suggestions according to the training course and teaching content; the above scheme provides a complete set of difficulty level setting, exercise effect evaluation and learning suggestion modules for the training course and teaching content, which can help the operator better understand his own skills and knowledge level, and set the difficulty level suitable for him according to the actual situation; at the same time, through the interactive evaluation exercises, the operator can have a deeper understanding of his own operation process and judgment results, thereby improving his skills and knowledge level. The learning suggestion module can provide personalized learning suggestions according to the operator's responses and defects, and help the operator to carry out targeted learning and improvement for his own problems. This embodiment can improve the teaching effect of the training course, enable learners to participate in the learning process more actively, and get better guidance and help in the learning process.

[0096] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention belong to the scope of the present invention and its equivalent technologies, the present invention is also intended to include these changes and variations.

Claims

1. A pathological image training and education control system based on virtual reality, characterized in that: Include: The simulation scene establishment subsystem is responsible for obtaining pathological images from the pathological image database, establishing a virtual pathological scene, simulating and displaying the pathological images through the virtual pathological scene, establishing a connection with the virtual pathological scene through virtual reality equipment, and observing and analyzing the pathological images in the virtual pathological scene; The virtual operation interaction subsystem is responsible for operating the virtual pathology scene through virtual reality equipment, realizing the interaction between the operator and the pathology image, and simulating the actual experimental operation and pathology diagnosis process; the operation includes selecting a specific area, zooming in or out of the image, marking and annotating; The training effectiveness evaluation subsystem is responsible for providing training courses and teaching contents at different levels. It evaluates the operator's skills and knowledge level through interactive evaluation tools, provides real-time feedback and personalized learning suggestions; the training courses and teaching contents include the learning of basic knowledge, the cultivation of pathological image diagnosis skills and case analysis.

2. The pathological image training and education control system based on virtual reality as claimed in claim 1, characterized in that: The virtual reality device includes a head-mounted display, a handle and a tracker, which enables immersive entry into a virtual pathology scene; the virtual pathology scene includes a pathology laboratory or an operating room; the pathology image database includes healthy pathology images and diseased pathology images.

3. The pathological image training and education control system based on virtual reality as claimed in claim 1, characterized in that: The simulation scenario establishment subsystem includes: The target information extraction module is responsible for extracting target information from pathological images. By extracting the information, the tissue structure, cell morphology, abnormal structure, staining effect and context information related to the virtual pathological scene are obtained. The spatial relationship embodied in different pathological images is expressed using the tissue structure, cell morphology, abnormal structure, staining effect and context information. The spatial relationship division module is responsible for dividing the spatial relationship into the first constraint relationship and the second constraint relationship. The first constraint relationship is used to extract the relative position relationship of the tissue structure and cells from the pathological image. The second constraint relationship is used to infer the existing pathological changes when analyzing the pathological image. The placement of the tissue structure and cells is determined according to the first constraint relationship. The second constraint relationship in the virtual pathological scene is determined according to prior knowledge. The entire placement of the model of the target information of the virtual pathological scene is completed. Instructions are input in Chinese text format to extract the virtual pathological scene information and the spatial relationship within the virtual pathological scene, complete the model placement, generate a scene description file and visualize it. The communication connection establishment module is responsible for establishing a connection with the operating system of the virtual pathology scene through the virtual reality device, and observing and analyzing pathological images in the virtual pathology scene.

4. The pathological image training and education control system based on virtual reality as claimed in claim 3, characterized in that: The tissue structure of the target information extraction module includes: cells, cell nuclei and cell arrangement, reflecting the normal or abnormal state of the tissue; cell morphology includes cell size, shape, staining properties and distribution of organelles, reflecting the physiological state and pathological changes of cells; abnormal structure includes: abnormal changes in tissue structure caused by tumor formation and inflammation; staining effects include nuclear staining and immunohistochemical staining; context information includes the location of the tissue, surrounding structures and clinical information of the patient.

5. The pathological image training and education control system based on virtual reality as claimed in claim 3, characterized in that: The first constraint relationship of the spatial relationship division module: there is a specific relative position relationship between the tissue structure and cells in the pathological image. The cell arrangement presents glandular structure, duct structure and cluster structure. The relative position relationship provides important information about the tissue structure and cell arrangement. The second constraint relationship: based on reasoning and judgment based on medical knowledge and experience, doctors will infer the existing pathological changes based on common sense spatial relationships when analyzing pathological images. Tumors cause disorder of tissue structure and abnormal arrangement of cells.

6. The pathological image training and education control system based on virtual reality as claimed in claim 3, characterized in that: Target information extraction module, including: The extraction strategy generation submodule is responsible for generating the target information extraction strategy, which is used to identify the tissue structure, cell morphology, abnormal structure, staining effect and context information from the pathological image, so as to realize the search and positioning of the target information; the identification of the pathological image containing all the target information is selected from the target information, and the identification is a sign that can express the identity information; The target information matching submodule is responsible for matching the target information of each pathological image in turn. For all selected target information, according to the positional relationship of the selected target information, the following matching is performed in turn: matching of the first type of target information with the second type of target information, matching of the second type of target information or the first type of target information, the second type of target information with the third type of target information, matching of the third type of target information, or the second type of target information, the third type of target information, or the first type of target information, the second type of target information, the third type of target information with the fourth type of target information, and so on, until the fifth type of target information is matched; the first type of target information, the second type of target information, the third type of target information, the fourth type of target information and the fifth type of target information correspond to tissue structure, cell morphology, abnormal structure, staining effect and context information respectively; The spatial relationship establishment submodule is responsible for extracting description vectors related to establishing virtual pathology scenes from pathology images according to the target information extraction strategy and matching results, generating global information related to establishing virtual pathology scenes based on the description vectors, and using tissue structure, cell morphology, abnormal structure, staining effect and contextual information to express the spatial relationships embodied in different pathology images.

7. The pathological image training and education control system based on virtual reality as claimed in claim 6, characterized in that: The spatial relationship establishment submodule includes: The description vector splicing unit is responsible for extracting feature points related to the target information from the pathological image, clustering the obtained feature points using the mean clustering algorithm to obtain the cluster center, using the difference vector to generate the local aggregation description vector of each pathological image, dividing each pathological image into regions, obtaining the number of each feature point and the pixel coordinates, and using the local aggregation description vector to splice and generate the spatial local aggregation description vector of each pathological image; The information model building unit is responsible for calculating the main direction of the feature point, matching the main direction with the spatial local aggregation description vector according to the nearest neighbor vector matching method, obtaining the local feature matching result of the pathological image, and building a global information model based on the local feature matching result. The global information model is used to describe the distribution and mutual relationship of different tissues in the virtual pathology scene; The spatial relationship display unit is responsible for establishing a visual simulation and display of the spatial relationship embodied in different pathological images based on the description results of the global information model, including tissue structure, cell morphology, abnormal structure, staining effect and contextual information.

8. The pathological image training and education control system based on virtual reality as claimed in claim 3, characterized in that: Spatial relationship division module, including: The database construction submodule is responsible for receiving the content of the first constraint relationship and the second constraint relationship, converting the content into a feature string, and constructing a spatial relationship database according to the feature string; The data storage submodule is responsible for traversing each content of the spatial relationship database, extracting the relative position relationship, pathological changes and index position information of the spatial relationship database from the tissue structure and cells, and storing them in the spatial relationship database; The scene generation submodule is responsible for inputting instructions in Chinese text format, extracting virtual pathology scene information and spatial relationships within the virtual pathology scene from the spatial relationship database, completing the model placement, realizing the overall placement of the model of the target information of the virtual pathology scene, generating a scene description file and visualizing it.

9. The pathological image training and education control system based on virtual reality as claimed in claim 8, characterized in that: The scene generation submodule includes: A vertex data determination unit is responsible for obtaining the shape and structure information of the virtual pathology model, parsing the virtual pathology model file, and obtaining vertex data in the local space, where the vertex data represents the shape and structure of the virtual pathology model; A vertex data processing unit is responsible for generating target vertex data by processing vertex data in local space based on vertex data, calculating the convex hull of the virtual pathology model in local space, and calculating the convex hull of the virtual pathology model in world space based on the convex hull in local space; The position coordinate determination unit is responsible for determining the placement position of the virtual pathology model in the world space by converting vertex data into coordinates in the world space, extracting the virtual pathology scene information and placing it at the placement position; this is achieved by determining the world space matrix of the virtual pathology model in the world space and multiplying the coordinate value of the target vertex with the world space matrix.

10. The pathological image training and education control system based on virtual reality as claimed in claim 3, characterized in that: Virtual operation interaction subsystem, including: The function permission setting module is responsible for setting the function permissions of each virtual reality device. The virtual reality device includes a teaching end and a student end. Both the teaching end and the student end are operators. The teaching end is responsible for setting the information of the virtual pathology scene, generating virtual pathology scene control information, and sending the virtual pathology scene control information to the server; the student end is responsible for collecting the student's action and voice signals, generating virtual operation role control information, and sending the virtual operation role control information to the server; The control information synchronization module is responsible for controlling the information of the virtual pathology scene and the control information of the virtual operation role, controlling the display scene of the student end, displaying the virtual role, receiving action and voice signals, and synchronizing the display information of the virtual pathology scene to each virtual reality device; The control information change module is responsible for receiving the movement and attribute information of the elements in the virtual pathology scene model, and changing the position and attributes of the pathology image in the student-side display scene; the movement and attribute information includes selecting a specific area, zooming in or out of the image, marking and annotation.

Images