Lateral lingual canal segmentation method and device based on deep learning
Through the deep learning-based lingual canal segmentation method, the multi-receptive field and multi-scale feature extraction module are used to solve the problem of difficult segmentation of lingual canals in oral and maxillofacial surgery, and efficient automatic segmentation of lingual canals is achieved, improving diagnosis and treatment efficiency and accuracy.
Patent Information
- Application Number
- CN202411940878.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-04-15
AI Technical Summary
In oral and maxillofacial surgery, the lingual canal is difficult to accurately observe and segment due to its complex structure and poor imaging quality, resulting in low diagnosis and treatment efficiency and a risk of bleeding and tumor spread.
The lingual tube segmentation method based on deep learning is adopted, and the segmentation network model of the multi-receptive field feature extraction module and the multi-scale feature fusion module is constructed to automatically segment the lingual tube, thereby improving the accuracy of small-objective segmentation.
It realizes automatic acquisition of the segmentation results of the lingual canal in three-dimensional CT images, improves diagnosis and treatment efficiency, assists doctors to more accurately judge the position and morphology of the lingual canal, and reduces the risk of bleeding and tumor spread during surgery.
Smart Images
Figure CN119832243A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a lingual duct segmentation method and device based on deep learning. Background Art
[0002] The lingual side of the mandible has a rich blood supply, involving the submental artery, sublingual artery, and incisive artery. These arterial branches enter or leave the mandible through the lingual canal. According to their location in the mandible, the lingual canals can be divided into median and lateral. The median lingual canal is located perpendicular to the mental spine, and the number is usually 1 to 3, with an average diameter of about 0.6 mm; the lateral lingual canals are mostly located in the incisor and premolar areas, and the number is usually 0 to 6, with an average diameter of about 0.5 mm. In surgical procedures such as implant treatment and mandibular osteotomy, once the arterial branches entering the mandible are damaged, there will be a serious risk of bleeding. Some reports have recorded floor of mouth bleeding and lip abnormalities related to mandibular implants, all of which are related to damage to the lingual canal. In addition, the lingual canal is also a potential pathway for tumor spread to the mandible, so it is crucial to determine the location of the mandibular lingual canal and its detailed anatomical characteristics.
[0003] Clinically, the observation of lingual canals is easily overlooked due to their complex structure and poor imaging quality. Since 3D CT has the advantages of providing 3D stereoscopic views and high-quality bone anatomical imaging, it is widely used in the diagnosis and treatment of oral and maxillofacial diseases. Due to the complex morphology of lingual canals and the lack of professional knowledge of some doctors, accurately judging the impact of lingual canals during surgery has become a major difficulty in diagnosis and treatment. Therefore, how to achieve the automated segmentation of lingual canals has become an urgent problem to be solved.
[0004] With the continuous development of artificial intelligence technology, image segmentation algorithms based on deep learning have been widely used in the field of medical images. The algorithm automatically segments the tissue structure that the doctor is concerned about so that the doctor can conduct further analysis. However, the current research on algorithms for lingual canal segmentation is still blank. There are the following difficulties in the design of lingual canal segmentation algorithms. First, the lingual canal is located in the incisor and premolar area of the mandible, and the three-dimensional CT image of the oral and maxillofacial region not only contains the complete mandible, but also involves a larger area. It is challenging to accurately locate the target area of the lingual canal. Second, the segmentation of small and complex structures such as lingual canals is also a difficulty in the field of image segmentation. For example, in the structure of the encoder-decoder, as the downsampling ratio increases, the features of small targets will be lost, resulting in the inability to guarantee the accuracy of the segmentation results. Summary of the invention
[0005] In order to solve the above technical problems, the present invention provides a lingual tube segmentation method and device based on deep learning, designs an efficient small target segmentation algorithm, solves the problem of poor small target segmentation ability; assists doctors in making judgments on the position and shape of the lingual tube, and improves the efficiency of diagnosis and treatment.
[0006] In order to achieve the above object, the present invention adopts the following technical solution:
[0007] A lingual duct segmentation method based on deep learning comprises the following steps:
[0008] Step 1, obtaining a three-dimensional oral CT image, wherein the three-dimensional oral CT image contains a complete mandibular structure;
[0009] Step 2: Perform cropping processing based on the three-dimensional oral CT image to obtain the lingual canal target area;
[0010] Step 3: Construct a lingual duct segmentation network model using a multi-receptive field feature extraction module and a multi-scale feature fusion module to perform lingual duct segmentation on the lingual duct target image.
[0011] Furthermore, the step 2 comprises:
[0012] The target area of the lingual tube is defined by a spatial cube frame. The coordinates of the eight vertices of the frame are calculated to determine the spatial cube frame. The eight vertices are recorded as vertices A, B, C, D, E, F, G, and H respectively.
[0013] Furthermore, the coordinate points are calculated as follows:
[0014] According to the relative position relationship between the mandibular nerve canal and the lingual canal, a spatial cube frame is formed, and the area within the frame is cut out from the original three-dimensional CT image as the lingual canal target image.
[0015] Further, step 3 comprises:
[0016] Step 3.1: Collect three-dimensional CT image data containing the complete mandible according to the three-dimensional oral CT image, and mark the median lingual canal and the lateral lingual canal areas as labels on the three-dimensional CT image; intercept the lingual canal target image and target label according to the above-mentioned spatial cube frame from the original three-dimensional CT image and label as training data, thereby forming a training data set;
[0017] Step 3.2: Build a lingual canal segmentation network model;
[0018] Step 3.3: Training the lingual canal segmentation network model;
[0019] Step 3.4: Use the trained segmentation model for lingual duct segmentation.
[0020] Furthermore, the step 3.2 includes:
[0021] The encoder-decoder network architecture is used as the basic architecture of the segmentation algorithm, and the multi-receptive field feature extraction module and multi-scale feature fusion module are used to improve the algorithm's ability to retain small target features;
[0022] A multi-view feature extraction module is used in the encoder. The multi-view feature extraction module consists of three parallel extraction branches. The first branch is a residual structure. The second branch is a convolution connected in series with a dilated convolution. The third branch is a convolution connected in series with two convolution operations with a dilation factor of 2.
[0023] A multi-scale feature fusion module is used in the decoder to fuse the features of adjacent layers. The multi-scale feature fusion module is based on low-level semantic features. The low-level semantic features are up-sampled by linear interpolation to enlarge the feature size; the upper-level spatial features are down-sampled by interpolation to reduce the feature size; the low-level semantic features, the upper-level spatial features and the current-level features are concatenated according to the dimension, and finally the multi-scale feature fusion is completed through a double-layer convolution operation.
[0024] Furthermore, the step 3.3 includes:
[0025] First, the images and labels of the training dataset in step 3.1 are resampled to a uniform pixel size and the images are normalized; fixed-size voxel blocks are intercepted in the dataset as training data and input into the encoder-decoder network architecture; the lingual duct segmentation network model is trained for 2-category segmentation.
[0026] Further, the step 3.4 comprises:
[0027] First, the overlapping lingual duct target image is divided into multiple fixed-size images as the images to be segmented; after the images to be segmented are input into the segmentation model in sequence to obtain all the segmentation results, voting and merging are performed to form a complete segmentation result; the segmentation noise is removed by volume thresholding to generate segmentation marks for the median lingual duct and the lateral lingual duct.
[0028] The present invention also provides a lingual duct segmentation device based on deep learning, comprising the following modules:
[0029] A three-dimensional image acquisition module is used to acquire a three-dimensional oral CT image, wherein the three-dimensional oral CT image includes a complete mandibular structure;
[0030] A target image acquisition module, which acquires a lingual tube target image according to a three-dimensional oral CT image;
[0031] The segmentation module constructs a lingual duct segmentation network model using a multi-receptive field feature extraction module and a multi-scale feature fusion module, and performs lingual duct segmentation according to the lingual duct target image.
[0032] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the above-mentioned deep learning-based lingual duct segmentation method are implemented.
[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned deep learning-based lingual duct segmentation method.
[0034] Beneficial effects:
[0035] The present invention uses deep learning technology in three-dimensional CT images to automatically obtain the segmentation results of the lingual duct. Through the multi-view feature extraction module and the multi-scale feature fusion module, the encoder-decoder structure's segmentation ability for small targets is improved, which assists doctors in making judgments on the position and shape of the lingual duct and improves the efficiency of diagnosis and treatment. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 Schematic diagram for obtaining target images of the lingual canal;
[0037] Figure 2 It is a flow chart of a lingual duct segmentation method based on deep learning of the present invention;
[0038] Figure 3 This is the network structure diagram of the lingual canal segmentation. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0040] In the description of the present invention, it should be understood that the terms "upper", "lower", "top", "bottom", "inner", "outer", etc. used in the present invention to indicate the orientation or position relationship is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.
[0041] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0042] As shown in 2, a lingual duct segmentation method based on deep learning of the present invention comprises the following steps:
[0043] Step 1: Acquire a three-dimensional oral CT image, wherein the three-dimensional oral CT image contains a complete mandibular structure.
[0044] Step 2: acquiring a lingual tube target image based on a three-dimensional oral CT image, including:
[0045] The target area of the lingual tube is defined by a space cube frame, and the coordinates of the eight vertices of the frame are calculated to determine the space cube frame. Figure 1 As shown, the eight vertices are respectively recorded as vertices A, B, C, D, E, F, G, and H;
[0046] The coordinate points are calculated as follows:
[0047] Obtain the center lines of the mandibular nerve canals on both sides. Take the coordinates of the middle points of the center lines on both sides, compare the Z coordinate values of the two middle points, and use the larger value as the Z coordinate value of vertex A, vertex B, vertex E, and vertex F; compare the Y coordinate values of the two middle points, and use the larger value as the Y axis coordinate value of vertex E, vertex F, vertex G, and vertex H;
[0048] The X coordinate value of the middle point on the center line of the right mandibular nerve canal is taken as the X coordinate value of vertex A, vertex C, vertex E, and vertex G; the X coordinate value of the middle point on the center line of the left mandibular nerve canal is taken as the X coordinate value of vertex B, vertex D, vertex F, and vertex H.
[0049] Starting from the origin of the coordinate system, the sum of the HU values of the bone (the unit of measurement for describing the density of human tissue in CT) is calculated layer by layer along the Y axis (coronal plane). After the layer with bone is detected, the Y axis layer number is recorded as the Y coordinate value of vertex A, vertex B, vertex C, and vertex D. Starting from the origin of the coordinate system, the sum of the HU values of the bone is calculated layer by layer along the Z axis (cross section). After the layer with bone is detected, the Z axis layer number is recorded as the Z coordinate value of vertex C, vertex D, vertex G, and vertex H.
[0050] A spatial cube frame is formed according to the above 8 vertices, and the area inside the cube frame is cut out from the original three-dimensional CT image as the lingual duct target image.
[0051] Step 3: Segmenting the lingual duct according to the lingual duct target image, including:
[0052] Step 3.1: Build a training dataset, including:
[0053] According to the three-dimensional oral CT images, three-dimensional CT image data containing the complete mandible is collected, and the number of samples of different age groups and different genders in the data is the same. The median lingual canal and the lateral lingual canal areas are marked as labels on the three-dimensional CT images. The original three-dimensional CT images and labels are intercepted according to the above-mentioned spatial cube frame to obtain the lingual canal target image and target label as training data.
[0054] Step 3.2: Build a lingual canal segmentation network model, including:
[0055] The encoder-decoder network architecture is used as the basic architecture of the segmentation algorithm, and the multi-receptive field feature extraction module and the multi-scale feature fusion module are used to improve the algorithm's ability to retain small target features. The specific network structure is as follows Figure 3 shown.
[0056] In the decoder, a multi-scale feature fusion module is used to fuse the features of adjacent layers. The multi-scale feature fusion module is based on low-level semantic features. The low-level semantic features use linear interpolation upsampling to enlarge the feature size; the upper-level spatial features use interpolation downsampling to reduce the feature size; the low-level semantic features, upper-level spatial features and the current-level features are spliced according to the dimension, and finally the multi-scale feature fusion is completed through a double-layer convolution operation. The splicing and fusion of features can be expressed by the formula:
[0057] ;
[0058] Among them, x is the feature output by the decoder, i is the number of decoder layers, μ is the feature output by the encoder, U(.) is the upsampling operation, D(.) is the downsampling operation, [.] means concatenation according to dimension, and F(.) is a double-layer convolution operation.
[0059] Step 3.3: Training the lingual duct segmentation network model, including:
[0060] First, the images and labels of the training dataset in step 3.1 were resampled to a uniform pixel size and image normalization was performed. Fixed-size voxel blocks were intercepted from the training dataset as training data and input into the encoder-decoder network architecture. The lingual canal segmentation network model was trained for 2-category segmentation. During the training process, data enhancement methods such as random angle rotation and random masking were used. The Dice similarity coefficient DSC was combined with multi-classification cross entropy CE, and the weights of the two were adjusted as the loss function.
[0061] Step 3.4: Use the trained segmentation model for lingual duct segmentation, including:
[0062] First, the overlapping lingual canal target images are divided into multiple fixed-size images as the images to be segmented. After the images to be segmented are input into the segmentation model in sequence to obtain all the segmentation results, voting is performed to merge and form a complete segmentation result. The segmentation noise is removed by volume thresholding to generate segmentation marks for the median lingual canal and the lateral lingual canal.
[0063] The present invention also provides a lingual duct segmentation device based on deep learning, comprising the following modules:
[0064] A three-dimensional image acquisition module is used to acquire a three-dimensional oral CT image, wherein the three-dimensional oral CT image includes a complete mandibular structure;
[0065] A target image acquisition module, which acquires a lingual tube target image according to a three-dimensional oral CT image;
[0066] The segmentation module constructs a lingual duct segmentation network model using a multi-receptive field feature extraction module and a multi-scale feature fusion module, and performs lingual duct segmentation according to the lingual duct target image.
[0067] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the above-mentioned deep learning-based lingual duct segmentation method are implemented.
[0068] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned deep learning-based lingual duct segmentation method.
[0069] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The solutions in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and interpreted scripting language JavaScript, etc.
[0070] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0071] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0073] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0074] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A lingual duct segmentation method based on deep learning, characterized in that: The steps include: Step 1, obtaining a three-dimensional oral CT image, wherein the three-dimensional oral CT image contains a complete mandibular structure; Step 2, acquiring a lingual tube target image based on a three-dimensional oral CT image; Step 3: Construct a lingual duct segmentation network model using a multi-receptive field feature extraction module and a multi-scale feature fusion module, and perform lingual duct segmentation based on the lingual duct target image.
2. The deep learning-based lingual duct segmentation method according to claim 1, characterized in that: The step 2 comprises: The target area of the lingual tube is defined by a spatial cube frame. The coordinates of the eight vertices of the frame are calculated to determine the spatial cube frame. The eight vertices are recorded as vertices A, B, C, D, E, F, G, and H respectively.
3. The deep learning-based lingual duct segmentation method according to claim 2, characterized in that: The coordinate points are calculated as follows: The center lines of the mandibular nerve canals on both sides were obtained based on the three-dimensional oral CT images; According to the relative position relationship between the mandibular nerve canal and the lingual canal, a spatial cube frame is formed, and the area within the frame is cut out from the original three-dimensional CT image as the lingual canal target image.
4. The deep learning-based lingual duct segmentation method according to claim 1, characterized in that: Step 3 comprises: Step 3.1: Collect three-dimensional CT image data containing the complete mandible according to the three-dimensional oral CT image, and mark the median lingual canal and the lateral lingual canal areas as labels on the three-dimensional CT image; intercept the lingual canal target image and target label according to the above-mentioned spatial cube frame from the original three-dimensional CT image and label as training data, thereby forming a training data set; Step 3.2: Build a lingual canal segmentation network model; Step 3.3: Training the lingual canal segmentation network model; Step 3.4: Use the trained segmentation model for lingual duct segmentation.
5. The deep learning-based lingual duct segmentation method according to claim 4, characterized in that: The step 3.2 comprises: The encoder-decoder network architecture is used as the basic architecture of the segmentation algorithm, and the multi-receptive field feature extraction module and multi-scale feature fusion module are used to improve the algorithm's ability to retain small target features; A multi-view feature extraction module is used in the encoder. The multi-view feature extraction module consists of three parallel extraction branches. The first branch is a residual structure. The second branch is a convolution connected in series with a dilated convolution. The third branch is a convolution connected in series with two convolution operations with a dilation factor of 2. A multi-scale feature fusion module is used in the decoder to fuse the features of adjacent layers. The multi-scale feature fusion module is based on low-level semantic features. The low-level semantic features are up-sampled by linear interpolation to enlarge the feature size; the upper-level spatial features are down-sampled by interpolation to reduce the feature size; the low-level semantic features, the upper-level spatial features and the current-level features are concatenated according to the dimension, and finally the multi-scale feature fusion is completed through a double-layer convolution operation.
6. The deep learning-based lingual duct segmentation method according to claim 5, characterized in that: The step 3.3 comprises: First, the images and labels of the training dataset in step 3.1 are resampled to a uniform pixel size and the images are normalized; fixed-size voxel blocks are intercepted in the dataset as training data and input into the encoder-decoder network architecture; the lingual duct segmentation network model is trained for 2-category segmentation.
7. The deep learning-based lingual duct segmentation method according to claim 6, characterized in that: The step 3.4 comprises: First, the overlapping lingual duct target image is divided into multiple fixed-size images as the images to be segmented; after the images to be segmented are input into the segmentation model in sequence to obtain all the segmentation results, voting and merging are performed to form a complete segmentation result; the segmentation noise is removed by volume thresholding to generate segmentation marks for the median lingual duct and the lateral lingual duct.
8. A lingual duct segmentation device based on deep learning, characterized in that: Includes the following modules: A three-dimensional image acquisition module is used to acquire a three-dimensional oral CT image, wherein the three-dimensional oral CT image includes a complete mandibular structure; A target image acquisition module, which acquires a lingual tube target image according to a three-dimensional oral CT image; The segmentation module constructs a lingual duct segmentation network model using a multi-receptive field feature extraction module and a multi-scale feature fusion module, and performs lingual duct segmentation according to the lingual duct target image.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of a lingual duct segmentation method based on deep learning as described in any one of claims 1 to 7 are implemented.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a lingual duct segmentation method based on deep learning as described in any one of claims 1 to 7 are implemented.