Three-dimensional visualization method and device suitable for medical teaching, equipment and medium

Abstract

The invention discloses a three-dimensional visualization method and device suitable for medical teaching, equipment and a medium, and relates to the technical field of computers, and the method comprises the steps: carrying out the data fusion of a multi-modal medical data set, so as to construct a three-dimensional digital model of each organ of three-dimensional dynamic visualization; based on the three-dimensional digital model of each organ, utilizing a graph neural network to construct an initial knowledge graph aiming at organ system dimensions, wherein nodes are an anatomical structure, a disease entity and a molecular mechanism of the three-dimensional digital model of each organ; drawing a dynamic connecting line between the nodes in the initial knowledge graph according to the association relationship between the nodes to obtain a target knowledge graph; displaying the initial state of the target knowledge graph on a preset display carrier; and locking a target display object corresponding to the map display adjustment instruction in the target knowledge map, and visualizing the target display object in a preset display carrier. And the limitation of three-dimensional visualization taking subjects as the center is broken.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a three-dimensional visualization method, device, equipment and medium applicable to medical teaching. Background Art

[0002] The current mainstream teaching mode in the field of medical education is still subject-centered, with independent teaching according to traditional subject classifications such as anatomy, physiology, and pathology. The supporting teaching visualization technologies are also mostly designed for single subjects or single scales. Although this mode is convenient for modular teaching of knowledge, there are many drawbacks in actual teaching and application.

[0003] First, subject knowledge is presented in a fragmented state, and students need to piece together knowledge points from different subjects by themselves. Second, the existing technologies usually only support the visualization presentation of single-scale data and cannot achieve cross-scale knowledge integration, resulting in the disconnection between basic theory and clinical practice. Moreover, different subject visualization software adopts independent interaction logics, and students need to adapt to different operation interfaces repeatedly, greatly increasing the learning cost.

[0004] With the development of precision medicine, this subject-centered teaching mode is more limited. It can neither integrate emerging multi-modal medical data nor adapt to the rapidly updated clinical needs. This technical fragmentation and limitation ultimately lead to the situation that the trained medical students may be proficient in single-subject knowledge but lack the comprehensive ability to solve actual clinical problems, seriously restricting the development of modern medical education.

[0005] In summary, how to break the limitations of subject-centered three-dimensional visualization is a problem to be solved in this field. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a three-dimensional visualization method, device, equipment and medium applicable to medical teaching to break the limitations of subject-centered three-dimensional visualization. The specific solutions are as follows:

[0007] In the first aspect, the present application discloses a three-dimensional visualization method applicable to medical teaching, including:

[0008] Performing data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of each organ for three-dimensional dynamic visualization;

[0009] Based on each of the three-dimensional digital models of the organs, using a graph neural network to construct an initial knowledge graph for the organ system dimension; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of each of the three-dimensional digital models of the organs.

[0010] Draw the dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes to obtain the target knowledge graph;

[0011] Display the initial state of the target knowledge graph on a preset display carrier;

[0012] Obtain a graph display adjustment instruction through a preset interaction interface, lock the target display object corresponding to the graph display adjustment instruction in the target knowledge graph, and visualize the target display object on the preset display carrier; wherein, the target display object is a target display node and a target display dynamic connection line corresponding to the graph display adjustment instruction.

[0013] Optionally, the data fusion of the preprocessed multi-modal medical data set to construct three-dimensional digital models of each organ for three-dimensional dynamic visualization includes:

[0014] Preprocess the original multi-modal medical data set to obtain the preprocessed multi-modal medical data set; wherein, the multi-modal medical data set includes anatomical structure data, pathological section data, and molecular mechanism data;

[0015] Use a transformer model to perform weighted data fusion on the multi-modal medical data set to construct three-dimensional digital models of each organ for three-dimensional dynamic visualization.

[0016] Optionally, the drawing of the dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes to obtain the target knowledge graph includes:

[0017] Obtain the initial node feature matrix and the initial edge set in the initial knowledge graph; wherein, the initial edge set includes the initial association relationship between the anatomical structure and the disease entity, the initial association relationship between the disease entity and the molecular mechanism, and the initial association relationship between the anatomical structure and the molecular mechanism;

[0018] Multiply the initial edge set by the initial node feature matrix, and update the initial node feature matrix according to the obtained multiplied features to obtain the target node feature matrix;

[0019] Construct dynamic connection lines between the nodes based on the target node feature matrix to obtain the target knowledge graph.

[0020] Optionally, the constructing of the dynamic connection lines between the nodes based on the target node feature matrix includes:

[0021] Determine the spatial distances between the nodes based on the Voronoi diagram and the target node feature matrix, and obtain the weight coefficients between the nodes by using the spatial distances and a preset attenuation coefficient;

[0022] Alternatively, based on the target node feature matrix, obtain the weight coefficients between each of the nodes by using a target edge weight formula constructed based on a target activation function and an orthogonal transformation matrix;

[0023] Determine the weight coefficients between each of the nodes as the weight coefficients of the dynamic connection lines between each of the nodes, so as to construct the dynamic connection lines between each of the nodes.

[0024] Optionally, the determining the weight coefficients between each of the nodes as the weight coefficients of the dynamic connection lines between each of the nodes, so as to construct the dynamic connection lines between each of the nodes, includes:

[0025] Screen out each target associated node that meets the preset dynamic connection line construction condition from each of the nodes according to the weight coefficients between each of the nodes;

[0026] Determine the attribute values of the dynamic connection lines between each of the target associated nodes based on the weight coefficients between each of the nodes, so as to construct the dynamic connection lines between each of the nodes based on the attribute values; wherein, the attribute values include color values, width values, and transparency values.

[0027] Optionally, the obtaining the map display adjustment instruction through a preset interaction interface includes:

[0028] Obtain the eye gaze information of a target object through a preset interaction interface, then generate a gaze point heat map based on the eye gaze information, so as to determine a map display adjustment instruction according to the gaze point heat map;

[0029] Or, identify the gesture information of a target object through a preset interaction interface, and match the gesture information with a preset gesture instruction posture, so as to determine a map display adjustment instruction according to the obtained matching result;

[0030] Wherein, the map display adjustment instruction includes any one or several of a transparency adjustment instruction, a display size adjustment instruction, and a display perspective adjustment instruction.

[0031] Optionally, the preset display carrier is a naked-eye 3D screen and includes each terminal adaptation interface, and the terminal adaptation interface is used to connect an AR device and an MR device;

[0032] The visualizing the target display object on the preset display carrier includes:

[0033] Visualize the target display object on the naked-eye 3D screen, and synchronize the target display object to the AR device and / or the MR device through the terminal adaptation interface.

[0034] In a second aspect, the present application discloses a three-dimensional visualization device applicable to medical teaching, including:

[0035] A digital model construction module, configured to perform data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of various organs for three-dimensional dynamic visualization;

[0036] An initial knowledge graph construction module, configured to construct an initial knowledge graph for the organ system dimension based on the three-dimensional digital models of the various organs and using a graph neural network; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of the three-dimensional digital models of the various organs;

[0037] A target knowledge graph construction module, configured to draw dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes to obtain a target knowledge graph;

[0038] An initial state display module, configured to display the initial state of the target knowledge graph on a preset display carrier;

[0039] A knowledge graph adjustment visualization module, configured to obtain a knowledge graph display adjustment instruction through a preset interaction interface, lock a target display object corresponding to the knowledge graph display adjustment instruction in the target knowledge graph, and visualize the target display object on the preset display carrier; wherein, the target display object is a target display node and a target display dynamic connection line corresponding to the knowledge graph display adjustment instruction.

[0040] In a third aspect, the present application discloses an electronic device, including:

[0041] A memory, configured to store a computer program;

[0042] A processor, configured to execute the computer program to implement the steps of the three-dimensional visualization method applicable to medical teaching disclosed above.

[0043] In a fourth aspect, the present application discloses a computer-readable storage medium, configured to store a computer program; wherein, when the computer program is executed by a processor, the steps of the three-dimensional visualization method applicable to medical teaching disclosed above are implemented.

[0044] The beneficial effects of this application are as follows: This application performs data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of various organs with three-dimensional dynamic visualization; based on the three-dimensional digital models of the organs, a graph neural network is used to construct an initial knowledge graph for the organ system dimension; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of the three-dimensional digital models of the organs; dynamic connection lines between the nodes in the initial knowledge graph are drawn according to the association relationships between the nodes to obtain the target knowledge graph; the initial state of the target knowledge graph is displayed on a preset display carrier; a graph display adjustment instruction is obtained through a preset interaction interface, and the target display object corresponding to the graph display adjustment instruction is locked in the target knowledge graph, and the target display object is visualized on the preset display carrier; wherein, the target display object is the target display node and the target display dynamic connection line corresponding to the graph display adjustment instruction. It can be seen that this application performs data fusion on the multi-modal medical data set to construct three-dimensional digital models of various organs with three-dimensional dynamic visualization, thereby constructing an initial knowledge graph centered on the organ system, that is, based on the three-dimensional digital models of the organs, a graph neural network is used to construct an initial knowledge graph for the organ system dimension, and the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of the three-dimensional digital models of the organs, which can enable users to understand medical macro, meso, and micro knowledge. Further, this application also draws dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes, that is, the dynamic connection lines can reflect the association relationships between different nodes. Then, when the target knowledge graph is displayed on the preset display carrier, on the one hand, users can intuitively understand the three-dimensional digital models of each organ and the anatomical structures, disease entities, and molecular mechanisms of each organ, and on the other hand, they can also intuitively understand the association relationships between the nodes, realizing multi-scale data visualization presentation, facilitating users to achieve cross-scale knowledge penetration. And because the target knowledge graph integrates macro, meso, and micro multi-modal medical data, it breaks through the limitations of three-dimensional visualization centered on disciplines. By obtaining a graph display adjustment instruction through a preset interaction interface and locking the target display object corresponding to the graph display adjustment instruction in the target knowledge graph, that is, the display of nodes and dynamic connection lines can be adjusted, so there is no need for users to adapt to different display carriers, significantly improving the user experience. Description of the Drawings

[0045] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0046] Figure 1 Flowchart of a 3D visualization method applicable to medical teaching disclosed in this application;

[0047] Figure 2 Flowchart of line width color mapping of a dynamic connection line disclosed in this application;

[0048] Figure 3 Specific display schematic diagram disclosed in this application;

[0049] Figure 4 Specific display schematic diagram based on a naked-eye 3D device disclosed in this application;

[0050] Figure 5 Schematic diagram of the structure of a 3D visualization device applicable to medical teaching disclosed in this application;

[0051] Figure 6 Schematic diagram of the structure of an electronic device disclosed in this application. Specific implementation manners

[0052] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0053] The current mainstream teaching mode in the field of medical education still focuses on disciplines, and independent teaching is carried out according to traditional subject classifications such as anatomy, physiology, and pathology. The supporting teaching visualization technologies are also mostly designed for single disciplines or single scales. Although this mode is convenient for modular teaching of knowledge, there are many drawbacks in actual teaching and application.

[0054] First, the subject knowledge is presented in a fragmented state, and students need to piece together the knowledge points of different disciplines by themselves. Second, the existing technologies usually only support the visualization presentation of single-scale data and cannot achieve cross-scale knowledge penetration, resulting in the disconnection between basic theory and clinical practice. Moreover, different discipline visualization software adopts independent interaction logics, and students need to repeatedly adapt to different operation interfaces, greatly increasing the learning cost.

[0055] With the development of precision medicine, this discipline-centered teaching mode is more limited. It can neither integrate emerging multi-modal medical data nor adapt to the rapidly updated clinical needs. This technical fragmentation and limitation ultimately lead to the situation that the trained medical students may be proficient in single-discipline knowledge but lack the comprehensive ability to solve actual clinical problems, seriously restricting the development of modern medical education.

[0056] Therefore, the present application correspondingly provides a three-dimensional visualization solution applicable to medical teaching, breaking the limitations of three-dimensional visualization centered on disciplines.

[0057] See Figure 1 As shown, an embodiment of the present application discloses a three-dimensional visualization method applicable to medical teaching, including:

[0058] Step S11: Perform data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of various organs for three-dimensional dynamic visualization.

[0059] In this embodiment, the performing data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of various organs for three-dimensional dynamic visualization includes: preprocessing the original multi-modal medical data set to obtain the preprocessed multi-modal medical data set; wherein, the multi-modal medical data set includes anatomical structure data, pathological section data, and molecular mechanism data; using a Transformer model to perform weighted data fusion on the multi-modal medical data set to construct three-dimensional digital models of various organs for three-dimensional dynamic visualization.

[0060] Preprocess the original multi-modal medical data set, and the preprocessing includes abnormal data elimination, redundant data elimination, and missing data supplementation to obtain the preprocessed multi-modal medical data set. The multi-modal medical data set includes data from various disciplines such as histology, anatomy, physiology, pathology, and pathophysiology in the discipline dimension, and includes anatomical structure data, pathological section data, and molecular mechanism data in the scale dimension, that is, it includes macro, meso, and micro multi-modal medical data. Use a Transformer model to perform weighted data fusion on the multi-modal medical data set to construct three-dimensional digital models of various organs for three-dimensional dynamic visualization.

[0061] Taking the heart organ as an example, the macroscopic data of multimodal medical data is guided by coronary heart disease, including 3D specimens of normal hearts, heart specimens of coronary heart disease, which can be magnified, reduced, split 360 degrees to 720 degrees, coronary arteries, veins, conduction systems, CT images (resolution 0.5mm) and MRI images of the heart, and the OpenPose is used to label the walking directions of the left coronary artery, right coronary artery and their branches, as well as various parts of the heart such as the left ventricle, right ventricle, left atrium, right atrium, heart valves, and conduction system of the heart, and 3D models of each stage of the development of the heart. Specifically, for example, it is a heart CT image (resolution 0.5mm), and the OpenPose is used to label the walking directions of the aorta and pulmonary artery; the microscopic data of multimodal medical data includes digital slices of myocardial tissue, digital slices of myocardial infarction, molecular structures of dynamic myocardial proteins, actin, troponin, sliding of myofilaments, and electron microscope images of the heart (resolution 5nm), and NLP is used to extract keyword-related molecular mechanism data. Specifically, for example, it is an electron microscope image of cardiomyocytes (resolution 5nm), and the keyword "cardiomyocyte apoptosis" is extracted through NLP (Natural Language Processing) to obtain keyword-related molecular mechanism data.

[0062] Step S12: Based on each of the three-dimensional digital models of the organs, a graph neural network is used to construct an initial knowledge graph for the organ system dimension; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of each of the three-dimensional digital models of the organs.

[0063] Based on each three-dimensional digital model of the organs, a graph neural network is used to construct an initial knowledge graph for the organ system dimension, and each node of the initial knowledge graph is the anatomical structure, disease entity, and molecular mechanism of each three-dimensional digital model of the organs. It can be understood that each node in the initial knowledge graph is independent and does not represent the relationships between them. Therefore, in order for users to understand the associations between each node in subsequent medical teaching, it is necessary to establish the associations between the nodes in the follow-up.

[0064] Step S13: Draw dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes to obtain the target knowledge graph.

[0065] In this embodiment, drawing the dynamic connection lines between the nodes in the initial knowledge graph according to the inter-node association relationship to obtain the target knowledge graph includes: obtaining the initial node feature matrix and the initial edge set in the initial knowledge graph; wherein, the initial edge set includes the initial association relationship between the anatomical structure and the disease entity, the initial association relationship between the disease entity and the molecular mechanism, and the initial association relationship between the anatomical structure and the molecular mechanism; multiplying the initial edge set by the initial node feature matrix, and updating the initial node feature matrix according to the obtained multiplied features to obtain the target node feature matrix; constructing the dynamic connection lines between the nodes based on the target node feature matrix to obtain the target knowledge graph.

[0066] Obtain the initial node feature matrix and the initial edge set in the initial knowledge graph. The initial edge set includes the initial association relationships between various nodes, specifically including the initial association relationship between the anatomical structure and the disease entity, the association relationship between the disease entity and the molecular mechanism, and the initial association relationship between the anatomical structure and the molecular mechanism. The message passing mechanism can be used to iteratively update the features of nodes and edges, as follows:

[0067] python# Initialize node features and edge features;

[0068] nodes = organ_system_nodes # Organ system node feature matrix (N×D);

[0069] edges = anatomical_edges # Anatomical association edge set (M×D_e);

[0070] # The first round of message passing (anatomical association degree calculation);

[0071] for _ in range(h_layers): node_features = torch.matmul(edges, node_features) # Adjacency matrix multiplication;

[0072] node_features = F.relu(node_features) # Nonlinear activation;

[0073] # The second round of message passing (pathological association degree calculation);

[0074] path_edges = pathological_edges # Pathological association edge set (M'×D_e);

[0075] for_in range(h_layers): node_features = torch.matmul(path_edges, node_features) node_features = F.relu(node_features);

[0076] # Edge weight calculation (anatomy + pathology);

[0077] edge_weights = torch.sigmoid(torch.matmul(node_features[edge[:, 0]], node_features[edge[:, 1]].t()));

[0078] Multiply the initial edge set by the initial node feature matrix, and update the initial node feature matrix according to the obtained multiplied features to obtain the target node feature matrix; construct dynamic connection lines between each node based on the target node feature matrix to obtain the target knowledge graph. Further, a knowledge system directory can also be generated based on each node and the dynamic connection lines between each node in the target knowledge graph.

[0079] In this embodiment, constructing the dynamic connection lines between each node based on the target node feature matrix includes: determining the spatial distance between each node based on the Voronoi diagram and the target node feature matrix, and obtaining the weight coefficient between each node by using the spatial distance and a preset attenuation coefficient; or, based on the target node feature matrix, obtaining the weight coefficient between each node by using a target edge weight formula constructed based on a target activation function and an orthogonal transformation matrix; determining the weight coefficient between each node as the weight coefficient of the dynamic connection line between each node to construct the dynamic connection line between each node.

[0080] It can be understood that the association degrees between different nodes are also different. In this embodiment, the weights between each node are used to represent the magnitude of the association degree, and the dynamic connection line visualizes this association degree, so that when teaching medicine, users can more intuitively view the association degree between each node.

[0081] In a specific embodiment, determining the spatial distance between each node based on the Voronoi diagram and the target node feature matrix, and obtaining the weight coefficient between each node by using the spatial distance and a preset attenuation coefficient, the specific implementation method is as follows:

[0082] class OrganGraphGNN(nn.Module):

[0083] def __init__(self, node_dim, edge_dim):

[0084] super().__init__();

[0085] self.conv1 = GCNConv(node_dim + edge_dim, 64) self.conv2 = GCNConv(64, 32);

[0086] self.conv3 = GCNConv(32, 16) # Output edge weights;

[0087] def forward(self, nodes, edges):

[0088] x = torch.cat([nodes, edges], dim=1) # Concatenate node and edge features x = self.conv1(x);

[0089] x = F.relu(x);

[0090] x = self.conv2(x);

[0091] x = self.conv3(x);

[0092] return x;

[0093] Obtain the weight coefficients between each node based on the reciprocal of the spatial distance between organ structures calculated by the Voronoi diagram and a preset attenuation coefficient , and the specific formula is as follows:

[0094] ;

[0095] In the formula, d ij is the Euclidean distance between nodes i and j, is the attenuation coefficient. The degree of association between diseases and pathology and its occurrence mechanism is to extract the association strength of the disease - symptom - treatment path from the knowledge graph and normalize it to 0 - 1.

[0096] In the second specific embodiment, based on the target node feature matrix, and using the target edge weight formula to obtain the weight coefficients between each of the nodes, and the target edge weight formula is constructed based on a target activation function and an orthogonal transformation matrix, and the target edge weight formula is specifically as follows:

[0097] ;

[0098] In the formula, is the Sigmoid activation function, and the output range is [0,1]. and are the weight coefficients, and + = 1. For example, the anatomical proportion is 70% and the pathological proportion is 30%. and represent the feature vector of node i, where specifically refers to anatomical features, such as CT / MRI coordinates and blood vessel directions. are pathological features, such as pathological section annotations and molecular mechanism scores. A and B are orthogonal transformation matrices used to reduce the feature dimension.

[0099] In this embodiment, determining the weight coefficients between the nodes as the weight coefficients of the dynamic connection lines between the nodes to construct the dynamic connection lines between the nodes includes: screening out each target associated node that meets the preset dynamic connection line construction condition from each node according to the weight coefficients between the nodes; determining the attribute values of the dynamic connection lines between the target associated nodes based on the weight coefficients between the nodes, so as to construct the dynamic connection lines between the nodes based on the attribute values; where the attribute values include color values, width values, and transparency values.

[0100] The existence of a dynamic connection line between nodes indicates an association between the nodes, and the color, width, transparency value, etc. of the dynamic connection line can represent information such as the strength of the association. First, screen out each target associated node that meets the preset dynamic connection line construction condition from each node according to the weight coefficients between the nodes. For example, there are N nodes, but the weight coefficient between node A and node B meets the preset dynamic connection line construction condition. The preset dynamic connection line construction condition is, for example, that the weight coefficient between the nodes is greater than the preset weight threshold. The attribute values of the dynamic connection lines between the target associated nodes can be specifically determined based on the weight coefficients between the nodes, where the attribute values include color values, width values, and transparency values, that is, the rendering rules of the dynamic connection lines. Among them, the specific implementation method of the rendering rules of the dynamic connection lines is as follows:

[0101] $ C = \begin{cases};

[0102] Markdown:

[0103] \text{Red} & \text{if} w_{ij} \geq 0.7 \\;

[0104] \text{Blue} & \text{if} 0.4 \leq w_{ij} < 0.7 \\;

[0105] \text{Green} & \text{otherwise};

[0106] \end{cases} $;

[0107] For example Figure 2 As shown in the flowchart of the line width - color mapping of a dynamic connection line, a more specific implementation of the dynamic effect is to dynamically adjust the line width and color of the dynamic connection line. Among them, the line width is linearly mapped to 1 - 5 px according to the weight coefficient. Red (#FF0000) represents the blood flow path, blue (#0000FF) represents nerve innervation, and green (#00FF00) represents immune regulation.

[0108] Step S14: Display the initial state of the target knowledge graph on a preset display carrier.

[0109] After obtaining the target knowledge graph, it is displayed in its initial state on a preset display carrier. It can be understood that each node in the target knowledge graph is the anatomical structure, disease entity, and molecular mechanism of the three - dimensional digital model of each organ. The initial state of the target knowledge graph includes the initial transparency. For example Figure 3 As shown in a specific display schematic diagram, the knowledge system directory and the target knowledge graph are displayed on a preset display carrier.

[0110] Furthermore, the preset display carrier can specifically be a carrier including a technologically - inspired colorful transparent glass cover, enabling the three - dimensional visualization of the target knowledge graph containing the three - dimensional digital models of each organ suitable for medical teaching on the colorful transparent glass cover, that is, users can see the three - dimensional digital models of each organ in three - dimensional through the colorful transparent glass cover. Among them, the material of the colorful transparent glass cover is a multi - layer composite material. The outer layer adopts a superposition structure of electrochromic glass (EC Glass) and nano - imprinted grating. The electrochromic layer selects an AgI / TiO 2 composite material, with a response time ≤ 50 ms, an adjustable light transmittance range of 30% - 80% (voltage drive 0 - 10 V). The grating layer is a microlens array with a period of 50 μm, and RGB three - color dynamic mixing is achieved through liquid - crystal phase modulation. The inner layer is a high - refractive - index PMMA (polymethyl methacrylate) optical resin with a thickness of 2 mm and a refractive index of 1.59. In the electro - controlled color mixing in the dynamic color generation mechanism of the colorful transparent glass cover, the color of the electrochromic layer (CIE 1931 color space) is controlled by an ITO (indium tin oxide) electrode array, supporting gradient transition. Its RGB value dynamic mapping formula is:

[0111] ;

[0112] ;

[0113] ;

[0114] where is the voltage phase angle, is the voltage change. In the grating color mixing of the dynamic color generation mechanism of the colorful transparent glass cover, the polarization direction of the microlens array is adjusted by a liquid crystal controller to selectively reflect light of a specific wavelength (such as a red wavelength of 620 - 750 nm).

[0115] Step S15: Obtain a graph display adjustment instruction through a preset interaction interface, lock a target display object corresponding to the graph display adjustment instruction in the target knowledge graph, and visualize the target display object on the preset display carrier; wherein, the target display object is a target display node and a target display dynamic connection line corresponding to the graph display adjustment instruction.

[0116] In this embodiment, obtaining the graph display adjustment instruction through the preset interaction interface includes: obtaining the eye gaze information of the target object through the preset interaction interface, then generating a gaze point heat map based on the eye gaze information to determine the graph display adjustment instruction according to the gaze point heat map; or, identifying the gesture information of the target object through the preset interaction interface and matching the gesture information with a preset gesture instruction posture to determine the graph display adjustment instruction according to the obtained matching result; wherein, the graph display adjustment instruction includes any one or several of a transparency adjustment instruction, a display size adjustment instruction, and a display perspective adjustment instruction. The preset interaction interface can obtain the respective instructions or information conveyed by the user in real time. Specifically, a gaze point heat map can be generated based on the eye gaze information to determine the graph display adjustment instruction according to the gaze point heat map. For example, eye movement data is collected, and the pupil center coordinates (sampling rate 120 Hz) are obtained through the Pupil Labs eye tracker of the MR glasses. During the generation of the gaze point heat map, first, the gaze time distribution information of the user is statistically analyzed, and then the average gaze duration of the local area is calculated according to the gaze time distribution information of the user, so as to generate a gaze point heat map according to the average gaze duration of the local area to determine the graph display adjustment instruction. If the graph display adjustment instruction is, for example, a transparency dynamic adjustment instruction, then the specific formula is:

[0117] ;

[0118] In the formula, = 70% is the initial transparency, k = 0.5% is the adjustment coefficient, , is a time-varying function related to the interaction intensity of the user (such as the eye gaze intensity or the gesture operation amplitude), is the cumulative amount of interaction intensity from the initial moment to time t (reflecting the user's continuous attention). When the user continuously gazes at or operates on a certain area ( ), the integral term increases with time, and the transparency linearly decreases from 70% to a minimum of 30%, and the minimum transparency is not lower than 30% to avoid visual interference caused by completely exposing the microstructure, and the maximum is not higher than 70% to retain the basic masking effect.

[0119] Identify the gesture information of the target object through the preset interaction interface, and match the gesture information with the preset gesture instruction posture to determine the atlas display adjustment instruction according to the obtained matching result. For example, the gesture information controls the transparency. First, gesture recognition is performed, and the user's gesture information is detected through MediaPipe OpenPose, such as grasping / zooming. The gesture information is mapped to the transparency adjustment instruction. The transparency adjustment instruction corresponding to the grasping action is used to reduce the transparency by 10%, and the transparency adjustment instruction corresponding to the zooming action is used to restore the transparency to the historical average value. It should be noted that the transparency of the target knowledge atlas has a maximum value and a minimum value, which can be set according to the specific actual situation.

[0120] It can be understood that whether the atlas display adjustment instruction is determined through the fixation point heat map or through the gesture information, the atlas display adjustment instruction can include any one or several of the transparency adjustment instruction, the display size adjustment instruction, and the display perspective adjustment instruction. Specifically, different determination methods of the atlas display adjustment instruction can be set according to the actual situation.

[0121] In this embodiment, the preset display carrier is a naked-eye 3D screen and includes various terminal adaptation interfaces. The terminal adaptation interfaces are used to connect AR devices and MR devices. The preset display carrier is a naked-eye 3D screen and includes various terminal adaptation interfaces, and a unified rendering pipeline is proposed to be compatible with naked-eye 3D / AR / VR / MR devices. The naked-eye 3D device generates a parallax image through the split-screen rendering technology. The left and right screens of the double-screen folding structure are a 4K naked-eye 3D display screen (resolution 3840×2160) and a touch operation area respectively. The screen frame is made of square-corner metal material (thickness ≤ 8mm). The optical module is based on the integrated Waves2 stereoscopic grating technology, the parallax angle ≥ 45°, and the brightness ≥ 500 nit; the AR / MR device uses spatial anchor points to locate the organ system and the real environment. For example Figure 4 As shown in a specific display schematic diagram based on the naked-eye 3D device, the three-dimensional digital model of the heart can be presented to the user stereoscopically.

[0122] In this embodiment, the visualization of the target display object on the preset display carrier includes: visualizing the target display object on the autostereoscopic screen and synchronizing the target display object to the AR device and / or the MR device through the terminal adaptation interface. In this way, the lesion area marked by the teacher on the autostereoscopic screen can be immediately seen by the students through the MR glasses as a holographic projection annotation, and all interaction states are synchronized in real time. This seamless connection not only significantly reduces the development cost, but more importantly, ensures the consistency of the teaching experience. This distributed data synchronization mechanism is established based on a blockchain-based data stream architecture, where the blockchain-based data stream architecture includes a data sharding and encryption part and a synchronization protocol part. In the data sharding and encryption part, the organ model is divided into a 1024×1024×1024 voxel grid, and each voxel data is hashed and distributedly stored in the consortium chain nodes. The synchronization protocol part uses the QUIC protocol to implement data broadcasting among multiple devices, and the confirmation mechanism ensures a delay of ≤5ms.

[0123] Since the preset display carrier is adapted to various devices, it is necessary to obtain the atlas display adjustment instruction according to the device type. The specific implementation process is as follows:

[0124] # Input: device type (VR / AR / MR / autostereoscopic 3D), user gesture sequence, eye movement data;

[0125] # Output: standardized operation instruction;

[0126] def unified_input_mapping(device_type, gestures, gaze_data):

[0127] # Device characteristic weight matrix;

[0128] device_weights = {

[0129] 'VR': [0.4, 0.3, 0.2, 0.1], # Gesture: 40%, eye movement: 30%, touch: 20%, others: 10%;

[0130] 'AR': [0.2, 0.5, 0.2, 0.1];

[0131] 'MR': [0.3, 0.6, 0.1, 0.0];

[0132] 'Autostereoscopic 3D': [0.1, 0.2, 0.6, 0.1];

[0133] }

[0134] # Dynamic weight calculation;

[0135] gesture_score = calculate_gesture_score(gestures);

[0136] # Gesture recognition confidence;

[0137] gaze_score = calculate_gaze_focus(gaze_data);

[0138] # Gaze focus intensity;

[0139] touch_score = calculate_touch_accuracy();

[0140] # Touch accuracy;

[0141] total_score = (

[0142] gesture_score×device_weights[device_type][0]+gaze_score×device_weights[device_type][1]+touch_score×device_weights[device_type]; );

[0144] # Return the standardized command;

[0145] return generate_standard_command(total_score).

[0146] Furthermore, this embodiment also includes an adaptive input engine for self-detection of device performance, and the specific implementation is as follows:

[0147] / / Device computing power classification detection;

[0148] enum DevicePerformanceLevel {

[0149] LEVEL_LOW, / / CPU ≤ 4 cores, GPU ≤ 2GB;

[0150] LEVEL_MEDIUM, / / CPU ≥ 6 cores, GPU ≥ 4GB;

[0151] LEVEL_HIGH / / CPU ≥ 8 cores, GPU ≥ 8GB;

[0152] };

[0153] DevicePerformanceLevel DetectPerformance(){

[0154] int cpu_cores = GetCPUCoreCount();

[0155] int gpu_memory = GetGPUMemory();

[0156] if (cpu_cores >= 8 && gpu_memory >= 8) {

[0157] return LEVEL_HIGH;

[0158] } else if (cpu_cores >= 6 && gpu_memory >= 4) {

[0159] return LEVEL_MEDIUM;

[0160] } else {

[0161] return LEVEL_LOW;

[0162] }

[0163] };

[0164] In this way, it is possible to realize a multi-device collaborative teaching scenario. For example, when switching devices, if the user switches from MR glasses to a naked-eye 3D terminal, the system automatically inherits the current annotation status of the heart model (such as the marked coronary artery lesion area). For example, to achieve interaction consistency, after selecting a lesion area using gaze control in the MR glasses, the same area is highlighted synchronously on the naked-eye 3D terminal and a pathological data pop-up window appears. Another example is to achieve performance adaptation. After the naked-eye 3D terminal detects a computing power limit, the model is automatically downgraded to a resolution of 512×512×512 (the frame rate increases from 45 FPS to 62 FPS).

[0165] Obtain any one or several of the transparency adjustment instruction, display size adjustment instruction, and display perspective adjustment instruction through a preset interaction interface, and lock the target display object corresponding to the graph display adjustment instruction in the target knowledge graph, such as locking the target display node and / or the target display dynamic connection line, and then visualize the target display object on a preset display carrier. For example, if the graph display adjustment instruction is the display size adjustment instruction, expand the display size of the target display node A from the original 30% to 50%. For another example, if the graph display adjustment instruction is the display perspective adjustment instruction, adjust the display angle of the target display node B from the original upward view to the downward view, and so on.

[0166] Furthermore, a graph modification instruction can also be obtained through the preset interaction interface, that is, modify the target display node and / or the target display dynamic connection line. For example, delete the dynamic connection line between node A and node B, and add a dynamic connection line between node A and node C. Another example is that node D can also be deleted or added, and information such as annotations can be added to the node and the dynamic connection line.

[0167] The beneficial effects of the present application are as follows: The present application performs data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of various organs with three-dimensional dynamic visualization; based on the three-dimensional digital models of the organs, an initial knowledge graph for the organ system dimension is constructed using a graph neural network; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of the three-dimensional digital models of the organs; dynamic connection lines between the nodes in the initial knowledge graph are drawn according to the association relationships between the nodes to obtain a target knowledge graph; the initial state of the target knowledge graph is displayed on a preset display carrier; a graph display adjustment instruction is obtained through a preset interaction interface, and a target display object corresponding to the graph display adjustment instruction is locked in the target knowledge graph, and the target display object is visualized on the preset display carrier; wherein, the target display object is a target display node and a target display dynamic connection line corresponding to the graph display adjustment instruction. It can be seen that the present application performs data fusion on the multi-modal medical data set to construct three-dimensional digital models of various organs with three-dimensional dynamic visualization, thereby constructing an initial knowledge graph centered on the organ system, that is, based on the three-dimensional digital models of the organs, an initial knowledge graph for the organ system dimension is constructed using a graph neural network, and the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of the three-dimensional digital models of the organs, which can enable users to understand medical macro, meso, and micro knowledge. Further, the present application also draws dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes, that is, the dynamic connection lines can reflect the association relationships between different nodes. Then, when the target knowledge graph is displayed on the preset display carrier, on the one hand, users can intuitively understand the three-dimensional digital models of each organ and the anatomical structures, disease entities, and molecular mechanisms of each organ, and on the other hand, they can also intuitively understand the association relationships between the nodes, realizing multi-scale data visualization presentation, facilitating users to achieve cross-scale knowledge penetration, and because the target knowledge graph integrates macro, meso, and micro multi-modal medical data, it breaks the limitation of three-dimensional visualization centered on disciplines. By obtaining a graph display adjustment instruction through a preset interaction interface and locking a target display object corresponding to the graph display adjustment instruction in the target knowledge graph, that is, the nodes and dynamic connection lines can be adjusted for display, so there is no need for users to adapt to different display carriers, significantly improving the user experience.

[0168] See Figure 5 As shown, an embodiment of the present application discloses a three-dimensional visualization device applicable to medical teaching, including:

[0169] A digital model construction module 11, configured to perform data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of various organs with three-dimensional dynamic visualization;

[0170] The initial knowledge graph construction module 12 is used to construct an initial knowledge graph for the organ system dimension based on each of the three-dimensional digital models of the organs and by using a graph neural network; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of each of the three-dimensional digital models of the organs.

[0171] The target knowledge graph construction module 13 is used to draw dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes to obtain a target knowledge graph.

[0172] The initial state display module 14 is used to display the initial state of the target knowledge graph on a preset display carrier.

[0173] The knowledge graph adjustment visualization module 15 is used to obtain a knowledge graph display adjustment instruction through a preset interaction interface, lock a target display object corresponding to the knowledge graph display adjustment instruction in the target knowledge graph, and visualize the target display object on the preset display carrier; wherein, the target display object is a target display node and a target display dynamic connection line corresponding to the knowledge graph display adjustment instruction.

[0174] The beneficial effects of this application are as follows: This application performs data fusion on the preprocessed multi-modal medical data set to construct three-dimensional digital models of various organs with three-dimensional dynamic visualization; based on the three-dimensional digital models of each organ, a graph neural network is used to construct an initial knowledge graph for the organ system dimension; wherein, the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of the three-dimensional digital models of each organ; dynamic connection lines between the nodes in the initial knowledge graph are drawn according to the association relationships between the nodes to obtain a target knowledge graph; the initial state of the target knowledge graph is displayed on a preset display carrier; a graph display adjustment instruction is obtained through a preset interaction interface, and the target display object corresponding to the graph display adjustment instruction is locked in the target knowledge graph, and the target display object is visualized on the preset display carrier; wherein, the target display object is the target display node and the target display dynamic connection line corresponding to the graph display adjustment instruction. It can be seen that this application performs data fusion on the multi-modal medical data set to construct three-dimensional digital models of various organs with three-dimensional dynamic visualization, thereby constructing an initial knowledge graph centered on the organ system, that is, based on the three-dimensional digital models of each organ, a graph neural network is used to construct an initial knowledge graph for the organ system dimension, and the nodes of the initial knowledge graph are the anatomical structures, disease entities, and molecular mechanisms of the three-dimensional digital models of each organ, which can enable users to understand medical macroscopic, mesoscopic, and microscopic knowledge. Further, this application also draws dynamic connection lines between the nodes in the initial knowledge graph according to the association relationships between the nodes, that is, the dynamic connection lines can reflect the association relationships between different nodes. Then, when the target knowledge graph is displayed on the preset display carrier, on the one hand, users can intuitively understand the three-dimensional digital models of each organ and the anatomical structures, disease entities, and molecular mechanisms of each organ, and on the other hand, they can also intuitively understand the association relationships between the nodes, realizing multi-scale data visualization presentation, facilitating users to achieve cross-scale knowledge penetration, and because the target knowledge graph integrates macroscopic, mesoscopic, and microscopic multi-modal medical data, it breaks through the limitations of three-dimensional visualization centered on disciplines. By obtaining a graph display adjustment instruction through a preset interaction interface and locking the target display object corresponding to the graph display adjustment instruction in the target knowledge graph, that is, the display of nodes and dynamic connection lines can be adjusted, so there is no need for users to adapt to different display carriers, significantly improving the user experience.

[0175] Furthermore, an embodiment of this application also provides an electronic device. Figure 6 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure should not be regarded as any limitation on the scope of use of this application.

[0176] Figure 6A schematic structural diagram of an electronic device provided by an embodiment of the present application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the three-dimensional visualization method applicable to medical teaching performed by the electronic device disclosed in any of the foregoing embodiments.

[0177] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and specific limitations are not imposed on it here; the input / output interface 25 is used to obtain external input data or output data to the outside, and the specific interface type can be selected according to specific application needs, and no specific limitations are made here.

[0178] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor used to process data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for the rendering and drawing of the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, and the AI processor is used to process computing operations related to machine learning.

[0179] In addition, as a carrier for resource storage, the memory 22 may be a read-only memory, a random access memory, a disk, or an optical disc, etc. The resources stored thereon include an operating system 221, a computer program 222, and data 223, etc., and the storage method may be short-term storage or permanent storage.

[0180] Among them, the operating system 221 is used to manage and control each hardware device and computer program 222 on the electronic device, so as to implement the operation and processing of the massive data 223 in the memory 22 by the processor 21. It can be Windows, Unix, Linux, etc. In addition to the computer program that can be used to complete the three-dimensional visualization method applicable to medical teaching executed by the electronic device disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs that can be used to complete other specific tasks. The data 223 may include not only the data transmitted by the external device received by the electronic device, but also the data collected by its own input / output interface 25, etc.

[0181] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the three-dimensional visualization method applicable to medical teaching disclosed above. For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated here.

[0182] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts between the various embodiments, reference can be made to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and reference can be made to the description in the method part for the relevant parts.

[0183] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application. The steps of the methods or algorithms described in combination with the embodiments disclosed herein can be directly implemented by hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable EPROM (Erasable Programmable Read Only Memory), electrically erasable programmable EEPROM (Electrically Erasable Programmable read only memory), registers, hard disks, removable disks, CD-ROM (Compact Disc Read-Only Memory), or any other form of storage medium known in the technical field.

[0184] Finally, it should also be noted that in this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0185] The above has introduced in detail a three-dimensional visualization method, device, equipment and medium applicable to medical teaching. In this article, specific examples are used to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A three-dimensional visualization method suitable for medical teaching, characterized in that: include: Perform data fusion on the preprocessed multimodal medical data sets to construct three-dimensional digital models of various organs with three-dimensional dynamic visualization; Based on the three-dimensional digital models of each organ, an initial knowledge graph for the organ system dimension is constructed using a graph neural network; wherein the nodes of the initial knowledge graph are the anatomical structure, disease entity, and molecular mechanism of the three-dimensional digital model of each organ; Draw dynamic connection lines between nodes in the initial knowledge graph according to the association relationship between nodes to obtain a target knowledge graph; Displaying the initial state of the target knowledge graph on a preset display carrier; A graph display adjustment instruction is obtained through a preset interactive interface, and a target display object corresponding to the graph display adjustment instruction is locked in the target knowledge graph, and the target display object is visualized on the preset display carrier; wherein the target display object is a target display node and a target display dynamic connection line corresponding to the graph display adjustment instruction.

2. The three-dimensional visualization method suitable for medical teaching according to claim 1, characterized in that: The method of fusing the preprocessed multimodal medical data set to construct a three-dimensional digital model of each organ with three-dimensional dynamic visualization includes: Preprocessing the original multimodal medical data set to obtain a preprocessed multimodal medical data set; wherein the multimodal medical data set includes anatomical structure data, pathological section data and molecular mechanism data; The converter model is used to perform weighted data fusion on the multimodal medical data set to construct a three-dimensional digital model of each organ with three-dimensional dynamic visualization.

3. The three-dimensional visualization method suitable for medical teaching according to claim 1, characterized in that: Drawing dynamic connection lines between nodes in the initial knowledge graph according to the association relationship between nodes to obtain the target knowledge graph includes: Acquire an initial node feature matrix and an initial edge set in the initial knowledge graph; wherein the initial edge set includes an initial association relationship between the anatomical structure and the disease entity, an initial association relationship between the disease entity and the molecular mechanism, and an initial association relationship between the anatomical structure and the molecular mechanism; Multiplying the initial edge set by the initial node feature matrix, and updating the initial node feature matrix according to the obtained multiplied features to obtain a target node feature matrix; Based on the target node feature matrix, dynamic connection lines are constructed between the nodes to obtain the target knowledge graph.

4. The three-dimensional visualization method suitable for medical teaching according to claim 3, characterized in that: The constructing of dynamic connection lines between the nodes based on the target node feature matrix includes: Determine the spatial distance between each of the nodes based on the Voronoi diagram and the target node feature matrix, and obtain the weight coefficient between each of the nodes using the spatial distance and a preset attenuation coefficient; Or, based on the target node feature matrix, a target edge weight formula constructed based on a target activation function and an orthogonal transformation matrix is ​​used to obtain a weight coefficient between each of the nodes; The weight coefficients between the nodes are determined as the weight coefficients of the dynamic connection lines between the nodes, so as to construct the dynamic connection lines between the nodes.

5. The three-dimensional visualization method suitable for medical teaching according to claim 4, characterized in that: The step of determining the weight coefficients between the nodes as the weight coefficients of the dynamic connection lines between the nodes to construct the dynamic connection lines between the nodes includes: Filtering target associated nodes that meet preset dynamic link construction conditions from the nodes according to weight coefficients between the nodes; The attribute value of the dynamic connection line between each of the target associated nodes is determined based on the weight coefficient between each of the nodes, so as to construct the dynamic connection line between each of the nodes based on the attribute value; wherein the attribute value includes a color value, a width value, and a transparency value.

6. The three-dimensional visualization method suitable for medical teaching according to any one of claims 1 to 5, characterized in that: The obtaining of the graph display adjustment instruction through the preset interactive interface includes: Obtaining eye gaze information of the target object through a preset interactive interface, generating a gaze point heat map based on the eye gaze information, and determining a graph display adjustment instruction according to the gaze point heat map; Or, identifying the gesture information of the target object through a preset interactive interface, and matching the gesture information with a preset gesture instruction posture, so as to determine the atlas display adjustment instruction according to the obtained matching result; The atlas display adjustment instructions include any one or more of transparency adjustment instructions, display size adjustment instructions, and display viewing angle adjustment instructions.

7. The three-dimensional visualization method suitable for medical teaching according to claim 1, characterized in that: The preset display carrier is a naked-eye three-dimensional screen and includes various terminal adaptation interfaces, and the terminal adaptation interfaces are used to connect AR devices and MR devices; The visualizing the target display object on the preset display carrier includes: The target display object is visualized on the naked-eye 3D screen, and the target display object is synchronized to the AR device and / or the MR device through the terminal adaptation interface.

8. A three-dimensional visualization device suitable for medical teaching, characterized in that: include: A digital model building module is used to fuse the pre-processed multimodal medical data set to build a three-dimensional digital model of each organ with three-dimensional dynamic visualization; An initial graph construction module is used to construct an initial knowledge graph for the organ system dimension based on the three-dimensional digital model of each organ and using a graph neural network; wherein the nodes of the initial knowledge graph are the anatomical structure, disease entity, and molecular mechanism of the three-dimensional digital model of each organ; A target graph construction module is used to draw dynamic connection lines between nodes in the initial knowledge graph according to the association relationship between nodes to obtain a target knowledge graph; An initial state display module, used to display the initial state of the target knowledge graph on a preset display carrier; A graph adjustment visualization module is used to obtain a graph display adjustment instruction through a preset interactive interface, and lock a target display object corresponding to the graph display adjustment instruction in the target knowledge graph, and visualize the target display object on the preset display carrier; wherein the target display object is a target display node and a target display dynamic connection line corresponding to the graph display adjustment instruction.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the three-dimensional visualization method suitable for medical teaching as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the steps of the three-dimensional visualization method suitable for medical teaching as described in any one of claims 1 to 7 are implemented.

Images