Three-dimensional visualization method, device, equipment and medium applicable to medical teaching

Abstract

The present application discloses a three-dimensional visualization method, device, equipment and medium applicable to medical teaching, which relates to the field of computer technology, and includes: performing data fusion on multi-modal medical data sets to construct three-dimensional digital models of various organs for three-dimensional dynamic visualization; constructing an initial knowledge graph for the organ system dimension by using a graph neural network based on the three-dimensional digital models of various organs, and the nodes are the anatomical structures, disease entities and molecular mechanisms of the three-dimensional digital models of various organs; drawing dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes to obtain a target knowledge graph; displaying an initial state of the target knowledge graph on a preset display carrier; locking a target display object corresponding to a graph display adjustment instruction in the target knowledge graph, and visualizing the target display object on the preset display carrier. It breaks the limitations of three-dimensional visualization centered on disciplines.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a three-dimensional visualization method, device, equipment and medium applicable to medical teaching. Background Art

[0002] The current mainstream teaching mode in the field of medical education is still subject-centered, with independent teaching according to traditional subject classifications such as anatomy, physiology, and pathology, and the supporting teaching visualization technologies are also mostly designed for single subjects or single scales. Although this mode is convenient for modular teaching of knowledge, there are many drawbacks in actual teaching and application.

[0003] First, the subject knowledge is presented in a fragmented state, and students need to piece together the knowledge points of different subjects by themselves. Second, the existing technologies usually only support the visualization presentation of single-scale data and cannot achieve cross-scale knowledge integration, resulting in the disconnection between basic theory and clinical practice. Third, different subject visualization software adopts independent interaction logics, and students need to adapt to different operation interfaces repeatedly, greatly increasing the learning cost.

[0004] With the development of precision medicine, this subject-centered teaching mode is more limited. It can neither integrate emerging multi-modal medical data nor adapt to the rapidly updated clinical needs. This technical fragmentation and limitation ultimately lead to the situation that the cultivated medical students may be proficient in single-subject knowledge but lack the comprehensive ability to solve actual clinical problems, seriously restricting the development of modern medical education.

[0005] In summary, how to break the limitations of subject-centered three-dimensional visualization is a problem to be solved in this field. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a three-dimensional visualization method, device, equipment and medium applicable to medical teaching, so as to break the limitations of subject-centered three-dimensional visualization. The specific solutions are as follows:

[0007] In the first aspect, the present application discloses a three-dimensional visualization method applicable to medical teaching, including:

[0008] Performing data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of each organ for three-dimensional dynamic visualization;

[0009] Based on each of the three-dimensional digital models of the organs, using a graph neural network to construct an initial knowledge graph for the organ system dimension; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of each of the three-dimensional digital models of the organs;

[0010] Draw dynamic connection lines between nodes in the initial knowledge graph according to the association relationships between nodes to obtain a target knowledge graph;

[0011] Display the initial state of the target knowledge graph on a preset display carrier;

[0012] Obtain a graph display adjustment instruction through a preset interaction interface, lock a target display object corresponding to the graph display adjustment instruction in the target knowledge graph, and visualize the target display object on the preset display carrier; wherein, the target display object is a target display node and a target display dynamic connection line corresponding to the graph display adjustment instruction.

[0013] Optionally, the data fusion of the preprocessed multi-modal medical data set to construct three-dimensional digital models of each organ for three-dimensional dynamic visualization includes:

[0014] Preprocess the original multi-modal medical data set to obtain a preprocessed multi-modal medical data set; wherein, the multi-modal medical data set includes anatomical structure data, pathological section data, and molecular mechanism data;

[0015] Use a transformer model to perform weighted data fusion on the multi-modal medical data set to construct three-dimensional digital models of each organ for three-dimensional dynamic visualization.

[0016] Optionally, the drawing of the dynamic connection lines between nodes in the initial knowledge graph according to the association relationships between nodes to obtain a target knowledge graph includes:

[0017] Obtain the initial node feature matrix and the initial edge set in the initial knowledge graph; wherein, the initial edge set includes the initial association relationship between the anatomical structure and the disease entity, the initial association relationship between the disease entity and the molecular mechanism, and the initial association relationship between the anatomical structure and the molecular mechanism;

[0018] Multiply the initial edge set by the initial node feature matrix, and update the initial node feature matrix according to the obtained multiplied features to obtain a target node feature matrix;

[0019] Construct dynamic connection lines between each of the nodes based on the target node feature matrix to obtain a target knowledge graph.

[0020] Optionally, the constructing of the dynamic connection lines between each of the nodes based on the target node feature matrix includes:

[0021] Determine the spatial distances between each of the nodes based on the Voronoi diagram and the target node feature matrix, and obtain the weight coefficients between each of the nodes by using the spatial distances and a preset attenuation coefficient;

[0022] Alternatively, based on the target node feature matrix, obtain the weight coefficients between each of the nodes by using a target edge weight formula constructed based on a target activation function and an orthogonal transformation matrix;

[0023] Determine the weight coefficients between each of the nodes as the weight coefficients of the dynamic connection lines between each of the nodes, so as to construct the dynamic connection lines between each of the nodes.

[0024] Optionally, the determining the weight coefficients between each of the nodes as the weight coefficients of the dynamic connection lines between each of the nodes, so as to construct the dynamic connection lines between each of the nodes, includes:

[0025] Screen out each target associated node that meets the preset dynamic connection line construction condition from each of the nodes according to the weight coefficients between each of the nodes;

[0026] Determine the attribute values of the dynamic connection lines between each of the target associated nodes based on the weight coefficients between each of the nodes, so as to construct the dynamic connection lines between each of the nodes based on the attribute values; wherein, the attribute values include color values, width values, and transparency values.

[0027] Optionally, the obtaining the graph display adjustment instruction through a preset interaction interface includes:

[0028] Obtain the eye gaze information of a target object through a preset interaction interface, then generate a gaze point heat map based on the eye gaze information, so as to determine a graph display adjustment instruction according to the gaze point heat map;

[0029] Or, identify the gesture information of a target object through a preset interaction interface, and match the gesture information with a preset gesture instruction posture, so as to determine a graph display adjustment instruction according to the obtained matching result;

[0030] Wherein, the graph display adjustment instruction includes any one or several of a transparency adjustment instruction, a display size adjustment instruction, and a display perspective adjustment instruction.

[0031] Optionally, the preset display carrier is a naked-eye 3D screen and includes each terminal adaptation interface, and the terminal adaptation interface is used to connect an AR device and an MR device;

[0032] The visualizing the target display object on the preset display carrier includes:

[0033] Visualize the target display object on the naked-eye 3D screen, and synchronize the target display object to the AR device and / or the MR device through the terminal adaptation interface.

[0034] In a second aspect, the present application discloses a three-dimensional visualization device suitable for medical teaching, including:

[0035] A digital model construction module for fusing the preprocessed multi-modal medical data sets to construct three-dimensional digital models of various organs for three-dimensional dynamic visualization;

[0036] An initial knowledge graph construction module for constructing an initial knowledge graph for the organ system dimension based on the three-dimensional digital models of the various organs and using a graph neural network; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of the three-dimensional digital models of the various organs;

[0037] A target knowledge graph construction module for drawing dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes to obtain a target knowledge graph;

[0038] An initial state display module for displaying the initial state of the target knowledge graph on a preset display carrier;

[0039] A knowledge graph adjustment visualization module for obtaining a knowledge graph display adjustment instruction through a preset interaction interface, locking a target display object corresponding to the knowledge graph display adjustment instruction in the target knowledge graph, and visualizing the target display object on the preset display carrier; wherein, the target display object is a target display node and a target display dynamic connection line corresponding to the knowledge graph display adjustment instruction.

[0040] In a third aspect, the present application discloses an electronic device, including:

[0041] A memory for storing a computer program;

[0042] A processor for executing the computer program to implement the steps of the three-dimensional visualization method for medical teaching disclosed above.

[0043] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the three-dimensional visualization method for medical teaching disclosed above are implemented.

[0044] The beneficial effects of this application are as follows: This application performs data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of various organs with three-dimensional dynamic visualization; based on the three-dimensional digital models of the organs, a graph neural network is used to construct an initial knowledge graph for the organ system dimension; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of the three-dimensional digital models of the organs; dynamic connection lines between the nodes in the initial knowledge graph are drawn according to the association relationships between the nodes to obtain the target knowledge graph; the initial state of the target knowledge graph is displayed on a preset display carrier; a graph display adjustment instruction is obtained through a preset interaction interface, and the target display object corresponding to the graph display adjustment instruction is locked in the target knowledge graph, and the target display object is visualized on the preset display carrier; wherein, the target display object is the target display node and the target display dynamic connection line corresponding to the graph display adjustment instruction. It can be seen that this application performs data fusion on the multi-modal medical data set to construct three-dimensional digital models of various organs with three-dimensional dynamic visualization, thereby constructing an initial knowledge graph centered on the organ system, that is, based on the three-dimensional digital models of the organs, a graph neural network is used to construct an initial knowledge graph for the organ system dimension, and the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of the three-dimensional digital models of the organs, which can enable users to understand medical macro, meso, and micro knowledge. Further, this application also draws dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes, that is, the dynamic connection lines can reflect the association relationships between different nodes. Then, when the target knowledge graph is displayed on the preset display carrier, on the one hand, users can intuitively understand the three-dimensional digital models of the organs and the anatomical structures, disease entities, and molecular mechanisms of the organs, and on the other hand, they can also intuitively understand the association relationships between the nodes, realizing multi-scale data visualization presentation, facilitating users to achieve cross-scale knowledge penetration. And because the target knowledge graph integrates macro, meso, and micro multi-modal medical data, it breaks through the limitations of three-dimensional visualization centered on disciplines. By obtaining a graph display adjustment instruction through a preset interaction interface and locking the target display object corresponding to the graph display adjustment instruction in the target knowledge graph, that is, the nodes and dynamic connection lines can be adjusted for display, so there is no need for users to adapt to different display carriers, significantly improving the user experience. Description of the Drawings

[0045] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0046] Figure 1 Flow chart of a 3D visualization method applicable to medical teaching disclosed in this application;

[0047] Figure 2 Flow chart of line width color mapping of a dynamic connection line disclosed in this application;

[0048] Figure 3 Specific display schematic diagram disclosed in this application;

[0049] Figure 4 Specific display schematic diagram based on a naked-eye 3D device disclosed in this application;

[0050] Figure 5 Schematic diagram of the structure of a 3D visualization device applicable to medical teaching disclosed in this application;

[0051] Figure 6 Schematic diagram of the structure of an electronic device disclosed in this application. Specific implementation manners

[0052] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0053] The mainstream teaching mode in the current medical education field is still centered around disciplines, and independent teaching is carried out according to traditional discipline classifications such as anatomy, physiology, and pathology. The supporting teaching visualization technologies are also mostly designed for single disciplines or single scales. Although this mode is convenient for modular teaching of knowledge, there are many drawbacks in actual teaching and application.

[0054] First of all, disciplinary knowledge is presented in a fragmented state, and students need to piece together knowledge points from different disciplines by themselves. Secondly, the existing technologies usually only support the visualization presentation of single-scale data and cannot achieve cross-scale knowledge integration, resulting in the disconnection between basic theory and clinical practice. Moreover, different disciplinary visualization software adopts independent interaction logics, and students need to repeatedly adapt to different operation interfaces, greatly increasing the learning cost.

[0055] With the development of precision medicine, this discipline-centered teaching mode is more limited. It can neither integrate emerging multi-modal medical data nor adapt to the rapidly updated clinical needs. This technical fragmentation and limitation ultimately lead to the situation that the trained medical students may be proficient in single-discipline knowledge but lack the comprehensive ability to solve actual clinical problems, severely restricting the development of modern medical education.

[0056] Therefore, the present application correspondingly provides a three-dimensional visualization solution applicable to medical teaching, breaking the limitations of discipline-centered three-dimensional visualization.

[0057] See Figure 1 As shown, an embodiment of the present application discloses a three-dimensional visualization method applicable to medical teaching, including:

[0058] Step S11: Perform data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of various organs for three-dimensional dynamic visualization.

[0059] In this embodiment, the performing data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of various organs for three-dimensional dynamic visualization includes: preprocessing the original multi-modal medical data set to obtain the preprocessed multi-modal medical data set; wherein, the multi-modal medical data set includes anatomical structure data, pathological section data, and molecular mechanism data; using a transformer model to perform weighted data fusion on the multi-modal medical data set to construct three-dimensional digital models of various organs for three-dimensional dynamic visualization.

[0060] Preprocess the original multi-modal medical data set, and the preprocessing includes abnormal data removal, redundant data removal, and missing data supplementation to obtain the preprocessed multi-modal medical data set. The multi-modal medical data set includes data from various disciplines such as histology, anatomy, physiology, pathology, and pathophysiology in the discipline dimension, and includes anatomical structure data, pathological section data, and molecular mechanism data in the scale dimension, that is, it includes macro, meso, and micro multi-modal medical data. Use a Transformer model to perform weighted data fusion on the multi-modal medical data set to construct three-dimensional digital models of various organs for three-dimensional dynamic visualization.

[0061] Taking the heart organ as an example, the macroscopic data of multimodal medical data is guided by coronary heart disease, including 3D specimens of normal hearts, heart specimens of coronary heart disease, which can be magnified, reduced, split 360 degrees - 720 degrees, coronary arteries, veins, conduction systems, CT images (resolution 0.5mm) of the heart, MRI images, and the directions of the left coronary artery, right coronary artery and their branches, as well as various parts of the heart such as the left ventricle, right ventricle, left atrium, right atrium, heart valves, and conduction system of the heart are labeled through OpenPose, and 3D models of each stage of heart development. Specifically, for example, it is a heart CT image (resolution 0.5mm), and the directions of the aorta and pulmonary artery are labeled through OpenPose; the microscopic data of multimodal medical data includes digital slices of myocardial tissue, digital slices of myocardial infarction, molecular structures of dynamic myocardial proteins, actin, troponin, sliding of myofilaments, electron microscopy images of the heart (resolution 5nm), and keyword-related molecular mechanism data is extracted through NLP. Specifically, for example, it is an electron microscopy image of cardiomyocytes (resolution 5nm), and keyword-related molecular mechanism data of "cardiomyocyte apoptosis" is extracted through NLP (Natural Language Processing, that is, Natural Language Processing).

[0062] Step S12: Based on each of the three-dimensional digital models of the organs, use a graph neural network to construct an initial knowledge graph for the organ system dimension; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of each of the three-dimensional digital models of the organs.

[0063] Based on each three-dimensional digital model of the organs and using a graph neural network to construct an initial knowledge graph for the organ system dimension, where each node is the anatomical structure, disease entity, and molecular mechanism of each three-dimensional digital model of the organs. It can be understood that each node in the initial knowledge graph is independent and does not represent the relationships between them. Therefore, in order for users to understand the associations between each node in subsequent medical teaching, it is necessary to establish the associations between the nodes later.

[0064] Step S13: Draw dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes to obtain the target knowledge graph.

[0065] In this embodiment, drawing the dynamic connection lines between the nodes in the initial knowledge graph according to the inter-node association relationship to obtain the target knowledge graph includes: obtaining the initial node feature matrix and the initial edge set in the initial knowledge graph; wherein, the initial edge set includes the initial association relationship between the anatomical structure and the disease entity, the initial association relationship between the disease entity and the molecular mechanism, and the initial association relationship between the anatomical structure and the molecular mechanism; multiplying the initial edge set by the initial node feature matrix, and updating the initial node feature matrix according to the obtained multiplied features to obtain the target node feature matrix; constructing the dynamic connection lines between the nodes based on the target node feature matrix to obtain the target knowledge graph.

[0066] Obtain the initial node feature matrix and the initial edge set in the initial knowledge graph. The initial edge set includes the initial association relationships between the nodes, specifically including the initial association relationship between the anatomical structure and the disease entity, the association relationship between the disease entity and the molecular mechanism, and the initial association relationship between the anatomical structure and the molecular mechanism. The message passing mechanism can be used to iteratively update the features of the nodes and edges, as follows:

[0067] python# Initialize node features and edge features;

[0068] nodes = organ_system_nodes # Organ system node feature matrix (N×D);

[0069] edges = anatomical_edges # Anatomical association edge set (M×D_e);

[0070] # The first round of message passing (anatomical association degree calculation);

[0071] for _ in range(h_layers): node_features = torch.matmul(edges, node_features) # Adjacency matrix multiplication;

[0072] node_features = F.relu(node_features) # Nonlinear activation;

[0073] # The second round of message passing (pathological association degree calculation);

[0074] path_edges = pathological_edges # Pathological association edge set (M'×D_e);

[0075] for_in range(h_layers): node_features = torch.matmul(path_edges, node_features) node_features = F.relu(node_features);

[0076] # Edge weight calculation (anatomy + pathology);

[0077] edge_weights = torch.sigmoid(torch.matmul(node_features[edge[:, 0]], node_features[edge[:, 1]].T));

[0078] Multiply the initial edge set with the initial node feature matrix, and update the initial node feature matrix according to the obtained multiplied features to obtain the target node feature matrix; construct dynamic connection lines between each node based on the target node feature matrix to obtain the target knowledge graph. Further, a knowledge system directory can also be generated based on each node and the dynamic connection lines between each node in the target knowledge graph.

[0079] In this embodiment, constructing the dynamic connection lines between each node based on the target node feature matrix includes: determining the spatial distance between each node based on the Voronoi diagram and the target node feature matrix, and obtaining the weight coefficient between each node by using the spatial distance and a preset attenuation coefficient; or, based on the target node feature matrix, obtaining the weight coefficient between each node by using a target edge weight formula constructed based on a target activation function and an orthogonal transformation matrix; determining the weight coefficient between each node as the weight coefficient of the dynamic connection line between each node to construct the dynamic connection line between each node.

[0080] It can be understood that the association degrees between different nodes are also different. In this embodiment, the weights between each node are used to represent the magnitude of the association degree, and the dynamic connection line visualizes this association degree, so that when teaching medicine, users can more intuitively view the association degree between each node.

[0081] In a specific embodiment, determining the spatial distance between each node based on the Voronoi diagram and the target node feature matrix, and obtaining the weight coefficient between each node by using the spatial distance and a preset attenuation coefficient, the specific implementation method is as follows:

[0082] class OrganGraphGNN(nn.Module):

[0083] def __init__(self, node_dim, edge_dim):

[0084] super().__init__();

[0085] self.conv1 = GCNConv(node_dim + edge_dim, 64) self.conv2 = GCNConv(64, 32);

[0086] self.conv3 = GCNConv(32, 16) # Output edge weights;

[0087] def forward(self, nodes, edges):

[0088] x = torch.cat([nodes, edges], dim=1) # Concatenate node and edge features x = self.conv1(x);

[0089] x = F.relu(x);

[0090] x = self.conv2(x);

[0091] x = self.conv3(x);

[0092] return x;

[0093] Obtain the weight coefficients between each node based on the reciprocal of the spatial distance between organ structures calculated by the Voronoi diagram and a preset attenuation coefficient , and the specific formula is as follows:

[0094] ;

[0095] In the formula, d ij is the Euclidean distance between nodes i and j, is the attenuation coefficient. The correlation degree between diseases and pathology and its occurrence mechanism is to extract the association strength of the disease - symptom - treatment path from the knowledge graph and normalize it to 0~1.

[0096] In the second specific embodiment, based on the target node feature matrix, obtain the weight coefficients between each of the nodes by using the target edge weight formula, and the target edge weight formula is constructed based on the target activation function and the orthogonal transformation matrix. The target edge weight formula is specifically as follows:

[0097] ;

[0098] In the formula, is the Sigmoid activation function, and the output range is [0, 1]. and are the weight coefficients, and + = 1. For example, the proportion of anatomy is 70% and that of pathology is 30%. and represent the feature vector of node i, where specifically refers to anatomical features such as CT / MRI coordinates and blood vessel directions. is the pathological feature such as pathological section annotation and molecular mechanism score. A and B are orthogonal transformation matrices used to reduce the feature dimension.

[0099] In this embodiment, determining the weight coefficients between the nodes as the weight coefficients of the dynamic connection lines between the nodes to construct the dynamic connection lines between the nodes includes: screening out each target associated node that meets the preset dynamic connection line construction condition from each node according to the weight coefficients between the nodes; determining the attribute values of the dynamic connection lines between the target associated nodes based on the weight coefficients between the nodes, so as to construct the dynamic connection lines between the nodes based on the attribute values; where the attribute values include color value, width value, and transparency value.

[0100] The existence of a dynamic connection line between nodes indicates an association between the nodes, and information such as the strength of the association can be represented by the color, width, transparency value, etc. of the dynamic connection line. First, each target associated node that meets the preset dynamic connection line construction condition is screened out from each node according to the weight coefficients between the nodes. For example, there are N nodes, but the weight coefficient between node A and node B meets the preset dynamic connection line construction condition. The preset dynamic connection line construction condition is, for example, that the weight coefficient between nodes is greater than the preset weight threshold. The attribute values of the dynamic connection lines between the target associated nodes can be specifically determined based on the weight coefficients between the nodes, where the attribute values include color value, width value, and transparency value, that is, the rendering rule of the dynamic connection line. Among them, the specific implementation method of the rendering rule of the dynamic connection line is as follows:

[0101] $ C = \begin{cases};

[0102] Markdown:

[0103] \text{Red} & \text{if} w_{ij} \geq 0.7 \\;

[0104] \text{Blue} & \text{if} 0.4 \leq w_{ij} < 0.7 \\;

[0105] \text{Green} & \text{otherwise};

[0106] \end{cases} $;

[0107] For example Figure 2 As shown in the line width - color mapping flowchart of a dynamic connection line, a more specific dynamic effect implementation is to dynamically adjust the line width and color of the dynamic connection line. Among them, the line width is linearly mapped to 1 - 5 px according to the weight coefficient. Red (#FF0000) represents the blood flow path, blue (#0000FF) represents nerve innervation, and green (#00FF00) represents immune regulation.

[0108] Step S14: Display the initial state of the target knowledge graph on a preset display carrier.

[0109] After obtaining the target knowledge graph, it is displayed in its initial state on a preset display carrier. It can be understood that each node in the target knowledge graph is the anatomical structure, disease entity, and molecular mechanism of the three - dimensional digital model of each organ. The initial state of the target knowledge graph includes the initial transparency. For example Figure 3 As shown in a specific display schematic diagram, the knowledge system directory and the target knowledge graph are displayed on a preset display carrier.

[0110] Furthermore, the preset display carrier can specifically be a carrier including a science - and - technology - sense colorful transparent glass cover, enabling the target knowledge graph containing the three - dimensional digital models of each organ suitable for medical teaching to achieve three - dimensional visualization on the colorful transparent glass cover, that is, users can see the three - dimensional digital models of each organ in three - dimensional through the colorful transparent glass cover. Among them, the material of the colorful transparent glass cover is a multi - layer composite material. The outer layer adopts a superposition structure of electrochromic glass (EC Glass) and nano - imprinted grating. The electrochromic layer selects AgI / TiO2 composite material, with a response time ≤ 50 ms and an adjustable light transmittance range of 30% - 80% (voltage drive 0 - 10 V). The grating layer is a microlens array with a period of 50 μm, and RGB three - color dynamic mixing is achieved through liquid - crystal phase modulation. The inner layer is a high - refractive - index PMMA (polymethyl methacrylate) optical resin with a thickness of 2 mm and a refractive index of 1.59. In the dynamic color generation mechanism of the colorful transparent glass cover, the electric - control color adjustment is to control the color of the electrochromic layer (CIE 1931 color space) through an ITO (indium tin oxide) electrode array, supporting gradient transition. Its RGB value dynamic mapping formula is:

[0111] ;

[0112] ;

[0113] ;

[0114] wherein is the voltage phase angle, is the voltage change. In the grating color mixing of the dynamic color generation mechanism of the colorful transparent glass cover, the polarization direction of the microlens array is adjusted by the liquid crystal controller to selectively reflect light of a specific wavelength (such as red wavelength 620 - 750nm).

[0115] Step S15: Obtain a graph display adjustment instruction through a preset interaction interface, lock a target display object corresponding to the graph display adjustment instruction in the target knowledge graph, and visualize the target display object on the preset display carrier; wherein, the target display object is a target display node and a target display dynamic connection line corresponding to the graph display adjustment instruction.

[0116] In this embodiment, obtaining the graph display adjustment instruction through the preset interaction interface includes: obtaining the eye gaze information of the target object through the preset interaction interface, then generating a gaze point heat map based on the eye gaze information to determine the graph display adjustment instruction according to the gaze point heat map; or, identifying the gesture information of the target object through the preset interaction interface and matching the gesture information with the preset gesture instruction postures to determine the graph display adjustment instruction according to the obtained matching result; wherein, the graph display adjustment instruction includes any one or several of a transparency adjustment instruction, a display size adjustment instruction, and a display angle adjustment instruction. The preset interaction interface can obtain the respective instructions or information conveyed by the user in real time. Specifically, a gaze point heat map can be generated based on the eye gaze information to determine the graph display adjustment instruction according to the gaze point heat map. For example, eye movement data is collected, and the pupil center coordinates are obtained through the Pupil Labs eye tracker of the MR glasses (sampling rate 120Hz). During the generation of the gaze point heat map, first, the gaze time distribution information of the user is statistically analyzed, and then the average gaze duration of the local area is calculated according to the gaze time distribution information of the user, so as to generate a gaze point heat map according to the average gaze duration of the local area to determine the graph display adjustment instruction. If the graph display adjustment instruction is, for example, a transparency dynamic adjustment instruction, then the specific formula is:

[0117] ;

[0118] In the formula, = 70% is the initial transparency, k = 0.5% is the adjustment coefficient, , is a time-varying function related to the interaction intensity of the user (such as eye gaze intensity or gesture operation amplitude), is the cumulative amount of interaction intensity from the initial moment to the moment t (reflecting the continuous attention of the user). When the user continuously gazes at or operates on a certain area ( ), the integral term increases with time, and the transparency linearly decreases from 70% to a minimum of 30%, and the minimum transparency is not less than 30% to avoid visual interference caused by complete exposure of the microstructure, and the maximum is not higher than 70% to retain the basic masking effect.

[0119] Identify the gesture information of the target object through the preset interaction interface, and match the gesture information with the preset gesture instruction posture to determine the atlas display adjustment instruction according to the obtained matching result. For example, the gesture information controls the transparency. That is, first perform gesture recognition, detect the user's gesture information through MediaPipe OpenPose, such as grabbing / scaling, map the gesture information to the transparency adjustment instruction, and the transparency adjustment instruction corresponding to the grabbing action is used to reduce the transparency by 10%, and the transparency adjustment instruction corresponding to the scaling action is used to restore the transparency to the historical average value. It should be noted that the transparency of the target knowledge atlas has a maximum value and a minimum value, which can be set according to the specific actual situation.

[0120] It can be understood that whether the atlas display adjustment instruction is determined through the fixation point heat map or through the gesture information, the atlas display adjustment instruction can include any one or several of the transparency adjustment instruction, the display size adjustment instruction, and the display perspective adjustment instruction. Specifically, different determination methods of the atlas display adjustment instruction can be set according to the actual situation.

[0121] In this embodiment, the preset display carrier is a naked-eye 3D screen and includes various terminal adaptation interfaces, and the terminal adaptation interfaces are used to connect AR devices and MR devices. The preset display carrier is a naked-eye 3D screen and includes various terminal adaptation interfaces, which proposes a unified rendering pipeline to be compatible with naked-eye 3D / AR / VR / MR devices. The naked-eye 3D device generates a parallax image through the split-screen rendering technology. The left and right screens of the double-screen folding structure are a 4K naked-eye 3D display screen (resolution 3840×2160) and a touch operation area respectively. The screen frame is made of square-corner metal material (thickness ≤ 8mm). The optical module is based on the integrated Waves2 stereoscopic grating technology, the parallax angle ≥ 45°, and the brightness ≥ 500 nit; the AR / MR device uses spatial anchor points to locate the organ system and the real environment. For example Figure 4 As shown in a specific display schematic diagram based on a naked-eye 3D device, the three-dimensional digital model of the heart can be presented to the user stereoscopically.

[0122] In this embodiment, visualizing the target display object on the preset display carrier includes: visualizing the target display object on the autostereoscopic screen and synchronizing the target display object to the AR device and / or the MR device through the terminal adaptation interface. In this way, for the lesion area marked by the teacher on the autostereoscopic screen, the student can immediately see the holographic projection annotation through the MR glasses, and all interaction states are synchronized in real time. This seamless connection not only greatly reduces the development cost, but more importantly, ensures the consistency of the teaching experience. This distributed data synchronization mechanism is established based on a blockchain-based data flow architecture. The blockchain-based data flow architecture includes a data sharding and encryption part and a synchronization protocol part. In the data sharding and encryption part, the organ model is divided into a 1024×1024×1024 voxel grid, and each voxel data is hashed and distributedly stored in the consortium chain nodes. The synchronization protocol part uses the QUIC protocol to implement data broadcasting among multiple devices, and the confirmation mechanism ensures a delay of ≤5ms.

[0123] Since the preset display carrier is adapted to various devices, it is necessary to obtain the atlas display adjustment instruction according to the device type. The specific implementation process is as follows:

[0124] # Input: device type (VR / AR / MR / autostereoscopic 3D), user gesture sequence, eye movement data;

[0125] # Output: standardized operation instruction;

[0126] def unified_input_mapping(device_type, gestures, gaze_data):

[0127] # Device characteristic weight matrix;

[0128] device_weights = {

[0129] 'VR': [0.4, 0.3, 0.2, 0.1], # Gesture: 40%, eye movement: 30%, touch: 20%, others: 10%;

[0130] 'AR': [0.2, 0.5, 0.2, 0.1];

[0131] 'MR': [0.3, 0.6, 0.1, 0.0];

[0132] 'Autostereoscopic 3D': [0.1, 0.2, 0.6, 0.1];

[0133] }

[0134] # Dynamic weight calculation;

[0135] gesture_score = calculate_gesture_score(gestures);

[0136] # Gesture recognition confidence;

[0137] gaze_score = calculate_gaze_focus(gaze_data);

[0138] # Eye movement focus intensity;

[0139] touch_score = calculate_touch_accuracy();

[0140] # Touch accuracy;

[0141] total_score = (

[0142] gesture_score×device_weights[device_type][0]+gaze_score×device_weights[device_type][1]+touch_score×device_weights[device_type]; );

[0144] # Return the standardized command;

[0145] return generate_standard_command(total_score).

[0146] Furthermore, this embodiment also includes an adaptive input engine for self-detection of device performance, and the specific implementation is as follows:

[0147] / / Device computing power grading detection;

[0148] enum DevicePerformanceLevel {

[0149] LEVEL_LOW, / / CPU ≤ 4 cores, GPU ≤ 2GB;

[0150] LEVEL_MEDIUM, / / CPU ≥ 6 cores, GPU ≥ 4GB;

[0151] LEVEL_HIGH / / CPU ≥ 8 cores, GPU ≥ 8GB;

[0152] };

[0153] DevicePerformanceLevel DetectPerformance(){

[0154] int cpu_cores = GetCPUCoreCount();

[0155] int gpu_memory = GetGPUMemory();

[0156] if (cpu_cores >= 8 && gpu_memory >= 8) {

[0157] return LEVEL_HIGH;

[0158] } else if (cpu_cores >= 6 && gpu_memory >= 4) {

[0159] return LEVEL_MEDIUM;

[0160] } else {

[0161] return LEVEL_LOW;

[0162] }

[0163] };

[0164] In this way, it is possible to realize a multi-device collaborative teaching scenario. For example, when switching devices, if the user switches from an MR glasses to a naked-eye 3D terminal, the system automatically inherits the current annotation state of the heart model (such as the marked coronary artery lesion area). For another example, to achieve interaction consistency, after selecting a lesion area using gaze control in the MR glasses, the naked-eye 3D terminal synchronously highlights the same area and pops up a pathological data window. Also, to achieve performance adaptability, after the naked-eye 3D terminal detects a computing power limit, it automatically downgrades the model to a resolution of 512×512×512 (the frame rate increases from 45 FPS to 62 FPS).

[0165] Obtain any one or several of the transparency adjustment instruction, display size adjustment instruction, and display perspective adjustment instruction through a preset interaction interface, and lock the target display object corresponding to the graph display adjustment instruction in the target knowledge graph, such as locking the target display node and / or the target display dynamic connection line, and then visualize the target display object on a preset display carrier. For example, if the graph display adjustment instruction is a display size adjustment instruction, expand the display size of the target display node A from the original 30% to 50%. For another example, if the graph display adjustment instruction is a display perspective adjustment instruction, adjust the display angle of the target display node B from the original upward view to a downward view, etc.

[0166] Furthermore, a graph modification instruction can also be obtained through the preset interaction interface, that is, modify the target display node and / or the target display dynamic connection line. For example, delete the dynamic connection line between node A and node B, and add a dynamic connection line between node A and node C. Another example is that node D can also be deleted or added, and information such as annotations can be added to the node and the dynamic connection line.

[0167] The beneficial effects of this application are as follows: This application performs data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of various organs with three-dimensional dynamic visualization; based on each of the three-dimensional organ digital models, a graph neural network is used to construct an initial knowledge graph for the organ system dimension; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of each of the three-dimensional organ digital models; dynamic connection lines between the nodes in the initial knowledge graph are drawn according to the association relationships between the nodes to obtain a target knowledge graph; the initial state of the target knowledge graph is displayed on a preset display carrier; a graph display adjustment instruction is obtained through a preset interaction interface, and a target display object corresponding to the graph display adjustment instruction is locked in the target knowledge graph, and the target display object is visualized on the preset display carrier; wherein, the target display object is a target display node and a target display dynamic connection line corresponding to the graph display adjustment instruction. It can be seen that this application performs data fusion on the multi-modal medical data set to construct three-dimensional digital models of various organs with three-dimensional dynamic visualization, thereby constructing an initial knowledge graph centered on the organ system, that is, based on each three-dimensional organ digital model, a graph neural network is used to construct an initial knowledge graph for the organ system dimension, and the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of each organ three-dimensional digital model, which can enable users to understand medical macro, meso, and micro knowledge. Further, this application also draws dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes, that is, the dynamic connection lines can reflect the association relationships between different nodes. Then, when the target knowledge graph is displayed on the preset display carrier, on the one hand, users can intuitively understand each three-dimensional organ digital model and the anatomical structures, disease entities, and molecular mechanisms of each organ, and on the other hand, they can also intuitively understand the association relationships between the nodes, realizing multi-scale data visualization presentation, facilitating users to achieve cross-scale knowledge penetration, and because the target knowledge graph integrates macro, meso, and micro multi-modal medical data, it breaks through the limitations of three-dimensional visualization centered on disciplines. By obtaining a graph display adjustment instruction through a preset interaction interface and locking a target display object corresponding to the graph display adjustment instruction in the target knowledge graph, that is, the display of nodes and dynamic connection lines can be adjusted, so there is no need for users to adapt to different display carriers, significantly improving the user experience.

[0168] See Figure 5 As shown, an embodiment of this application discloses a three-dimensional visualization device applicable to medical teaching, including:

[0169] A digital model construction module 11, configured to perform data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of various organs with three-dimensional dynamic visualization;

[0170] An initial graph construction module 12, configured to construct an initial knowledge graph for the organ system dimension based on each of the three-dimensional digital models of the organs and by using a graph neural network; wherein the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of each of the three-dimensional digital models of the organs.

[0171] A target graph construction module 13, configured to draw dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes, so as to obtain a target knowledge graph.

[0172] An initial state display module 14, configured to display the initial state of the target knowledge graph on a preset display carrier.

[0173] A graph adjustment visualization module 15, configured to obtain a graph display adjustment instruction through a preset interaction interface, lock a target display object corresponding to the graph display adjustment instruction in the target knowledge graph, and visualize the target display object on the preset display carrier; wherein the target display object is a target display node and a target display dynamic connection line corresponding to the graph display adjustment instruction.

[0174] The beneficial effects of this application are as follows: This application performs data fusion on the preprocessed multi-modal medical dataset to construct three-dimensional digital models of each organ for three-dimensional dynamic visualization; based on each of the three-dimensional organ digital models, a graph neural network is used to construct an initial knowledge graph for the organ system dimension; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of each of the three-dimensional organ digital models; dynamic connection lines between the nodes in the initial knowledge graph are drawn according to the association relationships between the nodes to obtain a target knowledge graph; the initial state of the target knowledge graph is displayed on a preset display carrier; a graph display adjustment instruction is obtained through a preset interaction interface, and a target display object corresponding to the graph display adjustment instruction is locked in the target knowledge graph, and the target display object is visualized on the preset display carrier; wherein, the target display object is a target display node and a target display dynamic connection line corresponding to the graph display adjustment instruction. It can be seen that this application performs data fusion on the multi-modal medical dataset to construct three-dimensional digital models of each organ for three-dimensional dynamic visualization, thereby constructing an initial knowledge graph centered on the organ system, that is, based on each three-dimensional organ digital model, a graph neural network is used to construct an initial knowledge graph for the organ system dimension, and the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of each organ three-dimensional digital model, which can enable users to understand medical macro, meso, and micro knowledge. Further, this application also draws dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes, that is, the dynamic connection lines can reflect the association relationships between different nodes. Then, when the target knowledge graph is displayed on a preset display carrier, on the one hand, users can intuitively understand each three-dimensional organ digital model and the anatomical structures, disease entities, and molecular mechanisms of each organ, and on the other hand, they can also intuitively understand the association relationships between the nodes, realizing multi-scale data visualization presentation, facilitating users to achieve cross-scale knowledge penetration, and because the target knowledge graph integrates macro, meso, and micro multi-modal medical data, it breaks the limitation of three-dimensional visualization centered on disciplines. By obtaining a graph display adjustment instruction through a preset interaction interface and locking a target display object corresponding to the graph display adjustment instruction in the target knowledge graph, that is, the nodes and dynamic connection lines can be adjusted for display, so there is no need for users to adapt to different display carriers, significantly improving the user experience.

[0175] Furthermore, an embodiment of this application also provides an electronic device. Figure 6 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure should not be considered as any limitation on the scope of use of this application.

[0176] Figure 6A schematic structural diagram of an electronic device provided by an embodiment of the present application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the three-dimensional visualization method applicable to medical teaching executed by the electronic device disclosed in any of the foregoing embodiments.

[0177] In this embodiment, the power supply 23 is used to provide working voltages for the various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and specific limitations are not imposed on it here; the input / output interface 25 is used to obtain external input data or output data to the outside, and the specific interface type thereof can be selected according to specific application requirements, and no specific limitations are made here.

[0178] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor used to process data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for the rendering and drawing of the content to be displayed on the display screen. In some embodiments, the processor 21 may further include an AI (Artificial Intelligence) processor, and the AI processor is used to process computational operations related to machine learning.

[0179] In addition, the memory 22, as a carrier for resource storage, may be a read-only memory, a random access memory, a disk, or an optical disc, etc. The resources stored thereon include an operating system 221, a computer program 222, and data 223, etc., and the storage method may be temporary storage or permanent storage.

[0180] Among them, the operating system 221 is used to manage and control each hardware device and computer program 222 on the electronic device, so as to implement the operation and processing of the massive data 223 in the memory 22 by the processor 21. It can be Windows, Unix, Linux, etc. In addition to the computer program that can be used to complete the three-dimensional visualization method applicable to medical teaching executed by the electronic device disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs that can be used to complete other specific tasks. The data 223 may include not only the data transmitted by the external device received by the electronic device, but also the data collected by its own input / output interface 25, etc.

[0181] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the three-dimensional visualization method applicable to medical teaching disclosed above. For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated here.

[0182] In this specification, each embodiment is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and reference can be made to the description of the method part for the relevant parts.

[0183] Those skilled in the art may further realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of function in the above description. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application. The steps of the methods or algorithms described in connection with the embodiments disclosed herein can be directly implemented by hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable EPROM (Erasable Programmable Read Only Memory), electrically erasable programmable EEPROM (Electrically Erasable Programmable read only memory), registers, hard disk, removable disk, CD-ROM (Compact Disc Read-Only Memory), or any other form of storage medium well known in the technical field.

[0184] Finally, it should also be noted that in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0185] The above has introduced in detail a three-dimensional visualization method, device, equipment and medium applicable to medical teaching. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A three-dimensional visualization method applicable to medical teaching, characterized in that, Including: Performing data fusion on the preprocessed multi-modal medical dataset to construct three-dimensional digital models of each organ for three-dimensional dynamic visualization; Based on each of the three-dimensional digital models of the organs, and using a graph neural network to construct an initial knowledge graph for the organ system dimension; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of each of the three-dimensional digital models of the organs; Drawing dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes to obtain a target knowledge graph; Displaying the initial state of the target knowledge graph on a preset display carrier; Obtaining a graph display adjustment instruction through a preset interaction interface, locking a target display object corresponding to the graph display adjustment instruction in the target knowledge graph, and visualizing the target display object on the preset display carrier; wherein, the target display object is a target display node and a target display dynamic connection line corresponding to the graph display adjustment instruction; The step of drawing dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes to obtain a target knowledge graph includes: Obtaining an initial node feature matrix and an initial edge set in the initial knowledge graph; wherein, the initial edge set includes the initial association relationships between the anatomical structure and the disease entity, the initial association relationships between the disease entity and the molecular mechanism, and the initial association relationships between the anatomical structure and the molecular mechanism; multiplying the initial edge set by the initial node feature matrix, and updating the initial node feature matrix according to the obtained multiplied features to obtain a target node feature matrix; constructing dynamic connection lines between each of the nodes based on the target node feature matrix to obtain a target knowledge graph.

2. The three-dimensional visualization method applicable to medical teaching according to claim 1, characterized in that The step of performing data fusion on the preprocessed multi-modal medical dataset to construct three-dimensional digital models of each organ for three-dimensional dynamic visualization includes: Preprocessing the original multi-modal medical dataset to obtain a preprocessed multi-modal medical dataset; wherein, the multi-modal medical dataset includes anatomical structure data, pathological section data, and molecular mechanism data; Using a transformer model to perform weighted data fusion on the multi-modal medical dataset to construct three-dimensional digital models of each organ for three-dimensional dynamic visualization.

3. The three-dimensional visualization method applicable to medical teaching according to claim 1, wherein The step of constructing dynamic connection lines between each of the nodes based on the target node feature matrix includes: Determining the spatial distances between each of the nodes based on the Voronoi diagram and the target node feature matrix, and obtaining the weight coefficients between each of the nodes using the spatial distances and a preset attenuation coefficient; Or, based on the target node feature matrix, and obtaining the weight coefficients between each of the nodes using a target edge weight formula constructed based on a target activation function and an orthogonal transformation matrix; Determining the weight coefficients between each of the nodes as the weight coefficients of the dynamic connection lines between each of the nodes to construct the dynamic connection lines between each of the nodes.

4. The three-dimensional visualization method applicable to medical teaching according to claim 3, characterized in that, The step of determining the weight coefficients between each of the nodes as the weight coefficients of the dynamic connection lines between each of the nodes to construct the dynamic connection lines between each of the nodes includes: Screen out each target associated node that meets the preset dynamic connection line construction conditions from each of the nodes according to the weight coefficients between the nodes; Determine the attribute values of the dynamic connection lines between the target associated nodes based on the weight coefficients between the nodes, so as to construct the dynamic connection lines between the nodes based on the attribute values; wherein, the attribute values include color values, width values, and transparency values.

5. The three-dimensional visualization method applicable to medical teaching according to any one of claims 1 to 4, characterized in that, The obtaining of the graph display adjustment instruction through the preset interaction interface includes: Obtain the eye gaze information of the target object through the preset interaction interface, then generate a gaze point heat map based on the eye gaze information, so as to determine the graph display adjustment instruction according to the gaze point heat map; Or, identify the gesture information of the target object through the preset interaction interface, and match the gesture information with the preset gesture instruction postures, so as to determine the graph display adjustment instruction according to the obtained matching result; Wherein, the graph display adjustment instruction includes any one or several of a transparency adjustment instruction, a display size adjustment instruction, and a display perspective adjustment instruction.

6. The three-dimensional visualization method applicable to medical teaching according to claim 1, characterized in that, The preset display carrier is a naked-eye 3D screen and includes each terminal adaptation interface, and the terminal adaptation interface is used to connect an AR device and an MR device; The visualizing of the target display object on the preset display carrier includes: Visualize the target display object on the naked-eye 3D screen, and synchronize the target display object to the AR device and / or the MR device through the terminal adaptation interface.

7. A three-dimensional visualization device applicable to medical teaching, characterized in that, Including: A digital model construction module, configured to perform data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of each organ for three-dimensional dynamic visualization; An initial graph construction module, configured to construct an initial knowledge graph for the organ system dimension based on each of the three-dimensional digital models of the organs and using a graph neural network; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of each of the three-dimensional digital models of the organs; A target graph construction module, configured to draw dynamic connection lines between the nodes in the initial knowledge graph according to the association relationship between the nodes to obtain a target knowledge graph; An initial state display module, configured to display the initial state of the target knowledge graph on a preset display carrier; A graph adjustment visualization module, configured to obtain a graph display adjustment instruction through a preset interaction interface, lock a target display object corresponding to the graph display adjustment instruction in the target knowledge graph, and visualize the target display object on the preset display carrier; wherein, the target display object is a target display node and a target display dynamic connection line corresponding to the graph display adjustment instruction; The target graph construction module is specifically configured to: Obtain the initial node feature matrix and the initial edge set in the initial knowledge graph; wherein, the initial edge set includes the initial association relationship between the anatomical structure and the disease entity, the initial association relationship between the disease entity and the molecular mechanism, and the initial association relationship between the anatomical structure and the molecular mechanism; multiply the initial edge set by the initial node feature matrix, and update the initial node feature matrix according to the obtained multiplied features to obtain the target node feature matrix; based on the target node feature matrix, construct dynamic connection lines between the nodes to obtain the target knowledge graph.

8. An electronic device, characterized in that, Comprising: A memory for storing a computer program; A processor for executing the computer program to implement the steps of the three-dimensional visualization method applicable to medical teaching according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, For storing a computer program; wherein, when the computer program is executed by a processor, the steps of the three-dimensional visualization method applicable to medical teaching according to any one of claims 1 to 6 are implemented.

Images