A three-dimensional modeling device, surgical planning system and teaching system

By designing a three-dimensional modeling device for head and neck medical images, using feature extraction model and weight fusion technology, the problem of head and neck three-dimensional modeling that is difficult to achieve high accuracy in the existing technology is solved, and the head and neck surgical planning and High accuracy of navigation.

CN112950774BActive Publication Date: 2024-11-05EYE & ENT HOSPITAL SHANGHAI MEDICAL SCHOOL FUDAN UNIV +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110396314.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-04-13
Publication Date
2024-11-05
Estimated Expiration
2041-04-13

AI Technical Summary

Technical Problem

The existing three-dimensional modeling technology is difficult to directly apply to the three-dimensional modeling of the head and neck, and cannot meet the needs of minimally invasive and high accuracy of retaining tissue functions.

Method used

A three-dimensional modeling device for head and neck medical images is designed. Using a feature extraction model that matches the tissue type, the accuracy of segmentation results is improved through feature value fusion and weight configuration, and a high-precision three-dimensional model is generated.

Benefits of technology

High-precision three-dimensional modeling in head and neck surgical planning and surgical navigation is achieved to meet the needs of minimally invasive and functional retention.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112950774B_ABST
    Figure CN112950774B_ABST
Patent Text Reader

Abstract

The present invention provides a three-dimensional modeling device, a surgical planning system and a teaching system. The three-dimensional modeling device includes: a medical image acquisition module for acquiring head and neck medical images; a feature model acquisition module for acquiring a matching feature extraction model according to the tissue type in the head and neck medical images; a feature value acquisition module for acquiring multiple feature values ​​of the head and neck medical images; a weight value acquisition module for acquiring the weight corresponding to each feature value; a feature value fusion module for acquiring a combination of features; a segmentation result acquisition module for acquiring the segmentation result of the head and neck medical images; a file generation module for generating a three-dimensional model file; and a three-dimensional model construction module for acquiring a three-dimensional model of the head and neck medical images. The three-dimensional model generated by the three-dimensional modeling device has a high degree of accuracy and can be directly applied to surgical planning or surgical navigation of head and neck surgical operations.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a three-dimensional modeling device, and in particular to a three-dimensional modeling device, a surgery planning system and a teaching system. Background Art

[0002] Head and neck surgery is an extension of otolaryngology. It is a discipline that has gradually developed to meet the needs of clinical diagnosis and treatment and scientific research of head and neck and otolaryngology-related malignant tumors. Head and neck surgical diseases can be divided into many categories according to the time of onset, location and nature. They can be generally summarized as: congenital diseases and deformities of the neck, inflammatory diseases of the neck, vascular diseases of the neck, thyroid and parathyroid diseases, submandibular gland diseases, parotid gland diseases, neck tumors, neck masses, etc. Surgery is one of the most commonly used radical treatments for head and neck tumors. It requires that on the basis of complete removal of the tumor, the basic functions of the organs should be retained as much as possible, and the quality of life of patients should be improved while improving their survival rate.

[0003] In actual clinical practice, surgical planning or surgical navigation based on the three-dimensional model of the patient's head and neck can make head and neck surgery faster, more accurate and safer. The implementation of surgical planning or surgical navigation depends on the three-dimensional modeling of the patient's head and neck. However, the inventors found in actual applications that, unlike surgical operations on other parts of the body, patients usually have minimally invasive and tissue appearance and function requirements for head and neck surgery. Therefore, surgical planning or surgical navigation for head and neck surgery must have a high degree of accuracy. To meet this requirement, the three-dimensional modeling of the patient's head and neck must be sufficiently accurate, which makes it difficult to directly apply existing three-dimensional modeling technology to the three-dimensional modeling of the head and neck. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a three-dimensional modeling device, a surgical planning system and a teaching system to solve the problem that the existing three-dimensional modeling technology is difficult to directly apply to the three-dimensional modeling of the head and neck.

[0005] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present invention provides a three-dimensional modeling device for head and neck medical images, the three-dimensional modeling device comprising: a medical image acquisition module, used to acquire a patient's head and neck medical image; a feature model acquisition module, connected to the medical image acquisition module, used to acquire a matching feature extraction model according to the tissue type in the head and neck medical image; a feature value acquisition module, connected to the medical image acquisition module and the feature model acquisition module, used to process the head and neck medical image using the feature extraction model to acquire multiple feature values ​​of the head and neck medical image; a weight value acquisition module, connected to the feature value acquisition module, used to acquire each of the feature values. The invention relates to a method for obtaining a head and neck medical image by using a three-dimensional model file and a three-dimensional model construction module. The method comprises the following steps: a first step is to obtain a head and neck medical image by using a three-dimensional model file and a third step is to obtain a head and neck medical image by using a three-dimensional model construction module. The first step is to obtain a head and neck medical image by using a three-dimensional model file and a three-dimensional model construction module. The first step is to obtain a head and neck medical image by using a three-dimensional model file and a three-dimensional model construction module. The first step is to obtain a head and neck medical image by using a three-dimensional model file and a three-dimensional model construction module. The first step is to obtain a head and neck medical image by using a three-dimensional model file and a three-dimensional model construction module.

[0006] In an embodiment of the first aspect, the feature extraction model includes a convolution layer, an activation function layer and a pooling layer, and the feature map output by any intermediate layer in the feature extraction model serves as an input parameter of each subsequent layer.

[0007] In an embodiment of the first aspect, the feature model acquisition module obtains the feature extraction model from a model library according to the tissue type in the head and neck medical image, wherein the model library includes multiple pre-trained feature extraction models, and each feature extraction model in the model library matches a head and neck tissue.

[0008] In an embodiment of the first aspect, the three-dimensional modeling device also includes: an information amplification module, connected to the medical image acquisition module and the feature value acquisition module, for processing the head and neck medical image to obtain at least one intermediate medical image, and the feature value acquisition module is used to use the feature extraction model to process the head and neck medical image and the intermediate medical image to obtain multiple feature values ​​of the head and neck medical image.

[0009] In an embodiment of the first aspect, the feature value acquisition module is further used to screen the multiple feature values.

[0010] In an embodiment of the first aspect, the file generation module includes: an intermediate file generation unit, connected to the segmentation result acquisition module, used to replace all coordinates in the coordinate point set into the medical digital imaging and communication pixel matrix to obtain an intermediate file; a three-dimensional model file generation unit, connected to the intermediate file generation unit, used to convert the intermediate file into an MHD file, and the MHD file is the three-dimensional model file.

[0011] In an embodiment of the first aspect, the three-dimensional modeling device further includes: a display module, connected to the three-dimensional model building module, for displaying the three-dimensional model of the head and neck medical image.

[0012] In an embodiment of the first aspect, the three-dimensional modeling device further includes: an interactive control module, connected to the display module, for controlling the display content and / or display mode of the display module in response to a received control instruction.

[0013] A second aspect of the present invention provides a preoperative planning system, which includes the three-dimensional modeling device described in any one of the first aspects of the present invention.

[0014] The third aspect of the present invention provides a teaching system, which includes: a three-dimensional modeling device as described in any one of the first aspects of the present invention, used to obtain a three-dimensional model of the head and neck medical image; a virtual reality display module, connected to the three-dimensional modeling device, used to display the three-dimensional model of the head and neck medical image using virtual reality technology.

[0015] As described above, a technical solution of the three-dimensional modeling device, surgical planning system and teaching system of the present invention has the following beneficial effects:

[0016] The three-dimensional modeling device uses a feature extraction model that matches the tissue type in the head and neck medical image, and performs feature value fusion based on the weight value corresponding to each feature value and segments the head and neck medical image according to the combined features obtained by fusion, which can ensure that the obtained segmentation result has a high accuracy. The three-dimensional model of the head and neck medical image obtained according to the high-precision segmentation result also has a high accuracy, and can therefore be directly applied to surgical planning or surgical navigation of head and neck surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1A Shown is a schematic structural diagram of the three-dimensional modeling device of the present invention in a specific embodiment.

[0018] Figure 1B Shown is a schematic diagram of the network structure of a feature extraction model of the three-dimensional modeling device of the present invention in a specific embodiment.

[0019] Figure 2 Shown is a schematic structural diagram of the three-dimensional modeling device of the present invention in another specific embodiment.

[0020] Figure 3 Shown is a schematic diagram of the structure of a file generation module in a specific embodiment of the three-dimensional modeling device of the present invention.

[0021] Component number description

[0022] 1 3D modeling device

[0023] 11 Medical Image Acquisition Module

[0024] 12 Feature model acquisition module

[0025] 13. Feature value acquisition module

[0026] 14 Weight value acquisition module

[0027] 15 Eigenvalue Fusion Module

[0028] 16 Segmentation result acquisition module

[0029] 17 File Generation Module

[0030] 171 Intermediate file generation unit

[0031] 172 3D model file generation unit

[0032] 18 3D model building module

[0033] 19 Information Amplification Module

[0034] 110 Display module

[0035] 111 Interactive Control Module

[0036] 2 Feature Extraction Model

[0037] 21 Convolutional Layer

[0038] 22 Activation Function Layer

[0039] 23 Pooling Layer DETAILED DESCRIPTION

[0040] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0041] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. The illustrations only show the components related to the present invention rather than the number, shape and size of the components in actual implementation. The type, quantity and proportion of each component in actual implementation may be changed at will, and the component layout type may also be more complicated. In addition, in this article, relational terms such as "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0042] In actual clinical practice, surgical planning or surgical navigation based on the patient's head and neck 3D model can make head and neck surgery faster, more accurate and safer. The implementation of surgical planning or surgical navigation depends on the 3D modeling of the patient's head and neck. However, unlike surgical operations in other parts of the body, patients have requirements for minimally invasive head and neck surgery and for preserving the appearance and function of tissues as much as possible. Therefore, surgical planning or surgical navigation for head and neck surgery must have a high degree of accuracy. To meet this requirement, the 3D modeling of the patient's head and neck must be sufficiently accurate, which makes it difficult to directly apply existing 3D modeling technology to 3D modeling of the head and neck.

[0043] In view of the above problems, the present invention provides a three-dimensional modeling device. First, the three-dimensional modeling device uses a feature extraction model that matches the tissue type in the head and neck medical image. For example, for a medical image containing a carotid artery, the three-dimensional modeling device uses a feature extraction model that matches the carotid artery. For a medical image containing an external auditory canal, the three-dimensional modeling device uses a feature extraction model corresponding to the external auditory canal. This method can improve the accuracy of the obtained feature value. Secondly, for the multiple feature values ​​obtained by the feature extraction model, different feature values ​​have different effects and contributions to the segmentation results. Based on this, the three-dimensional modeling device configures corresponding weights for different feature values, and performs feature value fusion based on the weight values ​​corresponding to each feature value and segments the head and neck medical image according to the combined features obtained by the fusion. This method can improve the accuracy of the segmentation result. The three-dimensional model of the head and neck medical image obtained according to the high-precision segmentation result also has a high accuracy. Therefore, the three-dimensional model of the head and neck medical image obtained by the three-dimensional modeling device of the present invention can be directly applied to the surgical planning or surgical navigation of head and neck surgery.

[0044] See also Figure 1A In one embodiment of the present invention, the three-dimensional modeling device 1 includes a medical image acquisition module 11, a feature model acquisition module 12, a feature value acquisition module 13, a weight value acquisition module 14, a feature value fusion module 15, a segmentation result acquisition module 16, a file generation module 17 and a three-dimensional model construction module 18.

[0045] The medical image acquisition module 11 is used to acquire medical images of the patient's head and neck, and the medical images of the head and neck are, for example, CT (Computed Tomography) or MR (Magnetic Resonance) images of patients acquired from different hospitals, different campuses, and different machines. It should be noted that even medical images acquired by the same machine may contain differences in multiple parameters, such as differences in respiratory motion, tube voltage, tube current, signal-to-noise ratio, reconstruction method, etc.

[0046] The feature model acquisition module 12 is connected to the medical image acquisition module 11, and is used to acquire a matching feature extraction model according to the tissue types in the head and neck medical images. The tissue types are, for example, mastoid, zygomatic, cranial fossa, common crus, cochlea, tympanic sinus, stapes, round window, cochlear process, vestibule, tympanic membrane, styloid process, odontoid process, semicircular canal, Eustachian tube, sigmoid sinus, facial nerve, carotid artery, jugular vein, handle of malleus, incus body, facial recess, external auditory canal, mandible, hypoglossal nerve, occipital condyle, foramen ovale, internal auditory canal, digastric escarpment, temporomandibular joint, arcuate eminence, greater superficial petrosal nerve, trigeminal nerve, etc.

[0047] Optionally, the feature model acquisition module 12 acquires the feature extraction model from a model library according to the tissue type in the head and neck medical image, wherein the model library includes multiple pre-trained feature extraction models, and each feature extraction model in the model library matches a head and neck tissue.

[0048] Preferably, see Figure 1B , the feature extraction model 2 includes a convolution layer 21, an activation function layer 22 and a pooling layer 23, and the feature maps output by any intermediate layer in the feature extraction model are used as input parameters for each subsequent layer, wherein the intermediate layer refers to the layer between the input layer and the output layer. At this time, since the feature maps output by any intermediate layer in the feature extraction model can be accessed by all subsequent layers, the features captured by each intermediate layer can be fully reused, so the model is relatively compact and the number of its parameters is relatively small, which is conducive to simplifying the complexity of the model. In addition, since the feature extraction model contains more shortcut connections, each layer in the model can independently receive the gradient conduction of the loss function, thereby realizing deep supervision. In addition, within the feature extraction model, any intermediate layer has a direct connection with all subsequent layers, making it possible for network layers in two different models to be connected through a conversion layer, thereby realizing random depth and connection.

[0049] The feature value acquisition module 13 is connected to the medical image acquisition module 11 and the feature model acquisition module 12, and is used to process the head and neck medical image using the feature extraction model to obtain multiple feature values ​​of the head and neck medical image. Specifically, the feature extraction model has multiple output channels, wherein each channel corresponds to an feature value.

[0050] Optionally, the feature value acquisition module 13 is further used to screen the multiple feature values. For example, the regularization term L1 may be used to screen the multiple feature values ​​to prevent overfitting of the feature extraction model.

[0051] The weight value acquisition module 14 is connected to the feature value acquisition module 13, and is used to obtain the weight corresponding to each feature value. Specifically, since different feature values ​​play different roles in different tasks of image segmentation and three-dimensional model reconstruction, in order to further improve the accuracy of the segmentation results and three-dimensional models, corresponding weights can be configured for each feature value according to the size of the role it plays in the specific task. For example, for the image segmentation task, if a certain feature value plays a greater role in the task, the weight corresponding to the feature value is greater, and vice versa.

[0052] The feature value fusion module 15 is connected to the feature value acquisition module 13 and the weight value acquisition module 14, and is used to fuse the feature values ​​according to the weights corresponding to the feature values ​​to obtain a combined feature. Specifically, the combined feature is Among them, w i is the weight corresponding to the i-th eigenvalue, F i is the i-th eigenvalue, and M is the number of the eigenvalues.

[0053] The segmentation result acquisition module 16 is connected to the feature value fusion module 15 and is used to process the combined features to obtain the segmentation result of the head and neck medical image. The segmentation result is, for example, the outline of the lesion or tissue boundary, that is, a series of coordinate points on a matrix.

[0054] The file generating module 17 is connected to the segmentation result acquiring module 16 and is used to generate a three-dimensional model file according to the coordinate point set included in the segmentation result. The three-dimensional model file is, for example, an mhd file.

[0055] The three-dimensional model construction module 18 is connected to the file generation module 17, and is used to process the three-dimensional model file using three-dimensional visualization technology to obtain the three-dimensional model of the head and neck medical image. Specifically, the three-dimensional model construction module 18 reads the three-dimensional model file and renders the obtained model using three-dimensional visualization technology to obtain the three-dimensional model of the head and neck medical image. In addition, the three-dimensional model construction module 18 can also reproduce the three-dimensional structure of individualized anatomy, and facilitate users to observe and understand the spatial three-dimensional structure of organs and tissues through operations such as transparency, rotation, scaling, and coloring. In addition, the three-dimensional model construction module 18 can also quantify the biological characteristic information of individual anatomy in three-dimensional space through measurement methods such as length, angle, area, and volume.

[0056] According to the above description, the three-dimensional modeling device of this embodiment uses a feature extraction model that matches the tissue type in the head and neck medical image, and performs feature value fusion based on the weight value corresponding to each feature value and segments the head and neck medical image according to the combined features obtained by fusion, which can ensure that the obtained segmentation result has a high accuracy. The three-dimensional model of the head and neck medical image obtained according to the high-precision segmentation result also has a high accuracy, and can therefore be directly applied to surgical planning or surgical navigation of head and neck surgery.

[0057] See also Figure 2In one embodiment of the present invention, considering that the head and neck medical images are mostly black and white two-dimensional images with limited information, in order to increase the information volume and data diversity, in this embodiment, the three-dimensional modeling device 1 further includes an information amplification module 19. The information amplification module 19 is connected to the medical image acquisition module 11 and the feature value acquisition module 13, and is used to process the head and neck medical images to obtain at least one intermediate medical image. The feature value acquisition module 13 uses the feature extraction module to process the head medical image and the intermediate medical image to obtain multiple feature values ​​of the head and neck medical image.

[0058] Optionally, the information augmentation module 19 processes the head and neck medical image using a generative adversarial network to generate the intermediate medical image.

[0059] Optionally, the information augmentation module 19 processes the head and neck medical image using Gaussian Laplace transform to generate the intermediate medical image. The specific transformation formula is: Among them, x, y and z are the coordinate values ​​of the pixel point on the x-axis, y-axis and z-axis respectively, σ is the standard deviation, G(x, y, z, σ) is the transformed value, and the amplification module 19 uses this formula to transform each pixel point in the head and neck medical image to obtain the intermediate medical image.

[0060] Optionally, the information augmentation module 19 processes the head and neck medical image using wavelet transform to generate the intermediate medical image. The specific transform formula is: in, is the basic wavelet function, a>0 is the scale factor, τ reflects the displacement, x(t) is the intensity of each pixel, WT x (a,τ) is the value after transformation.

[0061] Optionally, the information augmentation module 19 processes the head and neck medical image using image intensity square filtering to obtain the intermediate medical image. The specific transformation formula is: f(x)=(c1×x) 2 ,in x and f(x) are the image intensities of the head and neck medical image and the intermediate medical image, respectively.

[0062] Optionally, the information augmentation module 19 processes the head and neck medical image using image intensity square root filtering to obtain the intermediate medical image. The specific transformation formula is: Wherein, c2=max(|x|), x and f(x) are the image intensities of the head and neck medical image and the intermediate medical image, respectively.

[0063] Optionally, the information augmentation module 19 processes the head and neck medical image using logarithmic transformation to obtain the intermediate medical image. The specific transformation formula is: in x and f(x) are the image intensities of the head and neck medical image and the intermediate medical image, respectively.

[0064] Optionally, the information amplification module 19 processes the head and neck medical image using exponential transformation to obtain the intermediate medical image. The specific transformation formula is: in, x and f(x) are the image intensities of the head and neck medical image and the intermediate medical image, respectively.

[0065] Optionally, the information augmentation module 19 processes the head and neck medical image using gradient transformation to obtain the intermediate medical image. The specific transformation formula is: Wherein, f is the pixel matrix of the head and neck medical image.

[0066] Optionally, the information augmentation matrix 19 processes the head and neck medical image using a local binary transformation to obtain the intermediate medical image. The specific transformation formula is: in, (x c ,y c ) is the central pixel, and its brightness is i c ,i p is the brightness of the adjacent pixels, and P is the number of adjacent pixels of the central pixel.

[0067] According to the above description, this embodiment can obtain at least one intermediate medical image through the information augmentation module 19, which can increase the amount of information and data diversity of the head and neck medical image. The feature values ​​obtained jointly from the head and neck medical image and the intermediate medical image carry more information, and the segmentation result obtained based on such feature values ​​has higher accuracy, which is conducive to further improving the accuracy of the three-dimensional model.

[0068] See also Figure 3 In one embodiment of the present invention, the file generation module 17 includes an intermediate file generation unit 171 and a three-dimensional model file generation unit 172 .

[0069] The intermediate file generation unit 171 is connected to the segmentation result acquisition module 16, and is used to obtain the pixel matrix in the DICOM (Digital Imaging and Communications in Medicine) file, and replace all coordinates in the coordinate point set into the pixel matrix of the DICOM file, and its value is pre-set according to the tissue type, thereby forming a new DICOM file, which is the intermediate file.

[0070] For example, if the coordinate points in the segmentation result are marked with 1 and the other points are marked with 0, then a segmentation result obtained in this embodiment is represented by a matrix as follows: Assume that an original pixel matrix in the acquired DICOM file is If the highest value is set to 9, the matrix obtained by replacing all coordinates in the coordinate point set with the original pixel matrix of the DICOM file is The new DICOM file can be generated according to the matrix.

[0071] The 3D model file generating unit 172 is connected to the intermediate file generating unit 171 and is used to convert the intermediate file into an mhd file. The mhd file is the 3D model file, that is, a preliminary draft of the 3D model.

[0072] In one embodiment of the present invention, the three-dimensional modeling device 1 further includes a display module 110. The display module 110 is connected to the three-dimensional model building module 18 and is used to display the three-dimensional model of the head and neck medical image.

[0073] Optionally, the three-dimensional modeling device 1 further includes an interactive control module 111. The interactive module 111 is connected to the display module 110, and is used to control the display content and / or display mode of the display module in response to a received control instruction.

[0074] Optionally, the control instructions include two-dimensional image display instructions, three-dimensional image display instructions, image segmentation instructions or model data processing instructions, and the interactive control module 111 controls the display module 110 in response to the control instructions to respectively implement a two-dimensional image viewing function, a three-dimensional image viewing function, an image segmentation function or a model data processing function.

[0075] Specifically, the two-dimensional image view function includes: realizing multi-image display, and enlarged display of three-dimensional, cross-sectional, sagittal and coronal views; allowing the user to input control instructions through input devices such as a mouse to realize window width and window position adjustment, and display the window width and window position information in the status bar; allowing the user to set the display and hiding of cross lines and prompt information through control instructions; allowing the user to set and adjust the area of ​​interest in a two-dimensional view or a three-dimensional view through control instructions; prompting the user to reset the default values ​​of the three-dimensional scene camera position and the image window width and window position in the form of a pop-up menu, etc.; allowing the user to perform distance, angle, cross line and area measurements in a two-dimensional view through control instructions; allowing the user to automatically scroll, take screenshots and record videos of different cross-sectional images through control instructions; allowing the user to display, hide, parameterize, rename, modify transparency, view properties and / or copy and paste segmented data through control instructions.

[0076] The 3D view function includes: allowing the user to preview the segmented data in 3D through control instructions, and to implement local addition, local deletion, polygon cutting and intelligent connection in the 3D view; allowing the user to render the 3D model through control instructions to correctly display the 3D depth position relationship; allowing the user to implement volume rendering of 3D image data through control instructions Rendering) and fusion display with surface model data; allows users to rotate, zoom in, zoom out, show, hide and make transparent interactive operations of 3D models through control commands; allows users to measure length, angle, area, maximum diameter and volume in 3D view through control commands; realize automatic rotation, acceleration, stop, screenshot and video recording of models; allows users to adjust and view any section in 3D view through control commands; allows users to set the background color of 3D rendering scene through control commands; allows users to display and hide 3D model data, rename parameters, modify transparency, view properties and copy and paste through control commands; allows users to adjust the 3D position of 3D model in stereo space through control commands; allows users to dynamically and intelligently connect blood vessels, dynamically add segmented blood vessels, delete, smooth, fill and expand local segmented data through control commands, and realize arbitrary shape cutting of segmented data in 3D mode; allows users to automatically remove backboard, automatically remove bones and cut in any shape through control commands, and realize image segmentation function in WYSIWYG mode with 3D film reading function.

[0077] The image segmentation function includes: allowing the user to create new segmentation data through control instructions and automatically select the segmentation color; allowing the user to input control instructions through intuitive methods such as drawing lines and dragging boundary lines to achieve parameter selection of threshold segmentation; segmenting the maximum connection result based on the starting point selected by the user and the adjusted threshold range; obtaining the segmentation result through single-step segmentation processing for objects with unclear threshold boundaries; realizing intelligent segmentation for images with uneven CT thresholds; supporting vascular enhancement and realizing vascular segmentation for data with poor angiography conditions; for confirmed segmentations, the conversion from segmentation data to Mesh data can be realized at any time through three-dimensional modeling; and automatically optimizing the segmentation results.

[0078] The model data processing function includes: selecting appropriate parameters for 3D modeling according to the segmented data to obtain a Mesh surface model for 3D printing and display; allowing the user to perform line cutting through control instructions, that is, allowing the user to select two points on the 3D model through the input device to generate a circular cutting line, and double-click the mouse in a blank space to execute cutting, and double-click the right button to cancel. In this process, points can be selected continuously, and the final selection shall prevail; allowing the user to perform columnar cutting through control instructions, that is, allowing the user to construct a column through the input device to surround the surface data to be cut, and cut based on the column; allowing the user to perform point-select cutting through control instructions, that is, allowing the user to set the starting point and use the mouse to scroll The connected area can be expanded by means of wheels, and the cutting can be performed based on the user's selection; the user can be allowed to drag and deform through control instructions, that is, for a specific model, the user can select a center point and realize the model deformation as the point moves; the user can perform frame selection optimization through control instructions, that is, the user can stretch a rectangular area by clicking the mouse, etc. to determine the surface of the selected surface model, and then optimize and smooth the surface of the local optimized model to reduce the three-piece surface data; the user can perform point selection optimization through control instructions, that is, the user can select a connected area by clicking the mouse, etc. to determine the surface of the selected surface model, and then optimize and smooth the surface of the local optimized model to reduce the three-piece surface data; realize the model global optimization, and allows users to select different optimization levels through control commands; allows users to make interconnected selections through control commands, that is, allows users to select interconnected models for copying and deleting, for example, after selecting the model, the user double-clicks the left button to copy, and double-clicks the right button to delete; automatically reconstructs the skin model without operating any segmentation data; reconverts the surface model into a binary image model; merges the surface models; allows users to roll back the model editing data through control commands; supports safe editing, that is, after selecting a tumor or a space-occupying area, realizes the simulation of the tumor boundary within the safe distance range of the tumor; allows users to measure the distance of blood vessels in three-dimensional mode through control commands; allows users to adjust the distance according to puncture The system can set the position of the puncture or specify the guide plate through control instructions; allow the user to perform surgical simulation through control instructions, that is, allow the user to use the surface cutting tool to cut the surface model in manual mode. For example, the user can implement the cutting operation through the three steps of "clicking the surface", "generating the surface" and "surface cutting" in sequence, and automatically calculate the volume and maximum diameter of the two cut parts after the cutting is completed; allow the user to perform manual alignment through control instructions, that is, allow the user to move the surface model through the mouse or other means to align the spatial position relationship of the corresponding model; allow the user to perform watershed analysis through control instructions, that is, analyze the organ segmentation of the corresponding branch and nutrition according to the pipeline structure such as blood vessels and trachea.

[0079] Based on the above description of the three-dimensional modeling device, the present invention also provides a preoperative planning system, the preoperative planning system includes Figure 1A or Figure 2 The three-dimensional modeling device shown.

[0080] Optionally, the preoperative planning system can use the HoloLens device to display individualized three-dimensional medical cases and bring a good sense of immersion to the user. The preoperative planning system can rotate the scene 360 ​​degrees, scale the distance and size, change the color and transparency of the image, and split the parts to facilitate the observation of the specific information of the medical image. In addition, the preoperative planning system can also support multi-person collaboration to achieve multi-person multi-perspective, collaborative operation (real-time synchronization of scene position, rotation, scaling, etc.), and joint viewing. In addition, the preoperative planning system can also expand the display output content through a large screen, and allow users to interact through voice, gestures or tablet control.

[0081] Based on the above description of the three-dimensional modeling device, the present invention also provides a teaching system. The teaching system includes Figure 1A or Figure 2 The three-dimensional modeling transposition and virtual display module shown, wherein the user of the three-dimensional modeling device obtains the three-dimensional model of the head and neck medical image, and the virtual reality display module is connected to the three-dimensional modeling device for displaying the three-dimensional model of the head and neck medical image using virtual reality technology.

[0082] The three-dimensional modeling device of the present invention utilizes a feature extraction model that matches the tissue type in the head and neck medical image, and performs feature value fusion based on the weight value corresponding to each feature value and segments the head and neck medical image according to the combined features obtained by fusion, which can ensure that the obtained segmentation result has a high accuracy. The three-dimensional model of the head and neck medical image obtained according to the high-precision segmentation result also has a high accuracy, and can therefore be directly applied to surgical planning or surgical navigation of head and neck surgery.

[0083] In summary, the present invention effectively overcomes various shortcomings of the prior art and has high industrial utilization value.

[0084] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A three-dimensional modeling device for head and neck medical images, characterized in that: The three-dimensional modeling device comprises: A medical image acquisition module, used to acquire medical images of the patient's head and neck; a feature model acquisition module, connected to the medical image acquisition module, for acquiring a matching feature extraction model from a model library according to the tissue type in the head and neck medical image, wherein the model library includes a plurality of pre-trained feature extraction models, the feature map output by any intermediate layer in the feature extraction model is used as the input parameter of each subsequent layer, and each feature extraction model in the model library is matched with a head and neck tissue, and the tissue types in the head and neck medical image include mastoid, zygomatic, cranial fossa, common crus, cochlea, tympanic sinus, stapes, round window, cochlear process, vestibule, tympanic membrane, styloid process, odontoid process, semicircular canal, Eustachian tube, sigmoid sinus, facial nerve, carotid artery, jugular vein, handle of malleus, incus body, facial recess, external auditory canal, mandible, hypoglossal nerve, occipital condyle, foramen ovale, internal auditory canal, digastric escarpment, temporomandibular joint, arcuate eminence, greater superficial petrosal nerve and trigeminal nerve; A feature value acquisition module, connected to the medical image acquisition module and the feature model acquisition module, for processing the head and neck medical image using the matched feature extraction model to obtain a plurality of feature values ​​of the head and neck medical image; A weight value acquisition module, connected to the feature value acquisition module, for acquiring the weight corresponding to each feature value; A feature value fusion module, connected to the feature value acquisition module and the weight value acquisition module, for fusing the feature values ​​according to the weights corresponding to the feature values ​​to obtain a combined feature; A segmentation result acquisition module, connected to the feature value fusion module, for processing the combined features to obtain the segmentation result of the head and neck medical image; A file generation module, connected to the segmentation result acquisition module, for generating a three-dimensional model file according to a set of coordinate points included in the segmentation result; The three-dimensional model construction module is connected to the file generation module and is used to process the three-dimensional model file using three-dimensional visualization technology to obtain the three-dimensional model of the head and neck medical image.

2. The three-dimensional modeling device according to claim 1, characterized in that: The feature extraction model includes a convolution layer, an activation function layer and a pooling layer.

3. The three-dimensional modeling device according to any one of claims 1 to 2, characterized in that: The three-dimensional modeling device also includes: An information augmentation module is connected to the medical image acquisition module and the feature value acquisition module, and is used to process the head and neck medical image to obtain at least one intermediate medical image. The feature value acquisition module is used to process the head and neck medical image and the intermediate medical image using the matched feature extraction model to obtain multiple feature values ​​of the head and neck medical image.

4. The three-dimensional modeling device according to claim 1, characterized in that: The feature value acquisition module is also used to screen the multiple feature values.

5. The three-dimensional modeling device according to claim 1, characterized in that: The file generation module includes: An intermediate file generating unit, connected to the segmentation result acquiring module, is used to replace all coordinates in the coordinate point set into the medical digital imaging and communication pixel matrix to acquire an intermediate file; The three-dimensional model file generating unit is connected to the intermediate file generating unit and is used to convert the intermediate file into an MHD file, where the MHD file is the three-dimensional model file.

6. The three-dimensional modeling device according to claim 1, characterized in that: The three-dimensional modeling device also includes: The display module is connected to the three-dimensional model building module and is used to display the three-dimensional model of the head and neck medical image.

7. The three-dimensional modeling device according to claim 6, characterized in that: The three-dimensional modeling device also includes: The interactive control module is connected to the display module and is used to control the display content and / or display mode of the display module in response to a received control instruction.

8. A preoperative planning system, characterized in that: The preoperative planning system comprises the three-dimensional modeling device according to any one of claims 1-7.

9. A teaching system, characterized in that: The teaching system comprises: The three-dimensional modeling device according to any one of claims 1 to 7, used to obtain the three-dimensional model of the head and neck medical image; The virtual reality display module is connected to the three-dimensional modeling device and is used to display the three-dimensional model of the head and neck medical image using virtual reality technology.