Method and system for three-dimensional modeling of temporal bone based on CT images

Abstract

The application discloses a CT image-based temporal bone three-dimensional modeling method and system, relates to the technical field of image data processing, and comprises the following steps: using a high-resolution computer to perform tomography on the temporal bone of a target user to obtain HRCT image data, constructing three-dimensional HRCT volume data after preprocessing, inputting the three-dimensional semantic segmentation network to generate a multi-class probability graph, then generating a semantic region mask, optimizing the geometric consistency of the three-dimensional HRCT volume data under the constraint of the mask, generating a temporal bone three-dimensional mesh model through three-dimensional surface reconstruction, and finally optimizing the model and outputting a temporal bone three-dimensional model. The application solves the technical problems that the complex anatomical structure of the temporal bone leads to great difficulty in small structure recognition in the modeling process and easy semantic segmentation blurring and geometric feature distortion, achieves the technical effect of accurately restoring the anatomical structure of the temporal bone, improves the stability and authenticity of modeling, and makes the model highly fit the actual anatomical state of the temporal bone of the human body.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a method and system for three-dimensional modeling of the temporal bone based on CT images. Background Technology

[0002] Three-dimensional modeling of the temporal bone is of great significance for clinical diagnosis and treatment in otolaryngology and for skull base anatomy research. Region segmentation, as a core technical aspect, is crucial for controlling modeling accuracy. Currently, for the region segmentation problem in temporal bone three-dimensional modeling, traditional image region segmentation combined with conventional 3D reconstruction methods is mainly used. These methods have played an important role in image modeling of routine anatomical structures, but as clinical demands for modeling accuracy continue to increase, traditional techniques have revealed many limitations in temporal bone modeling applications. Due to the complex anatomical structure of the temporal bone and the numerous substructures, traditional region segmentation cannot accurately divide different anatomical structural regions, resulting in incomplete and inaccurate modeling data extraction, insufficient model realism and stability, and difficulty in meeting the high-precision modeling needs of precise clinical diagnosis and treatment and anatomical research. Summary of the Invention

[0003] This application provides a method and system for three-dimensional modeling of the temporal bone based on CT images, which solves the technical problems of difficulty in recognizing small structures, semantic segmentation ambiguity, and geometric feature distortion during the modeling process due to the complex anatomical structure of the temporal bone.

[0004] The first aspect of this application provides a method for three-dimensional modeling of the temporal bone based on CT images. The method includes: performing a tomographic scan of the temporal bone of a target user using a high-resolution computer to obtain HRCT image data; performing preprocessing on the HRCT image data to construct three-dimensional HRCT volume data; inputting the three-dimensional HRCT volume data into a three-dimensional semantic segmentation network to generate a multi-class probability map, wherein the multi-class probability map describes the semantic probability distribution of each voxel under multiple structural categories; generating a semantic region mask based on the multi-class probability map, wherein the semantic region mask characterizes semantically reliable regions and semantically uncertain regions; performing geometric consistency optimization on the three-dimensional HRCT volume data under the constraints of the semantic region mask to generate geometrically consistent enhanced volume data; performing three-dimensional surface reconstruction based on the geometrically consistent enhanced volume data to generate a three-dimensional mesh model of the temporal bone; performing model optimization processing on the three-dimensional mesh model of the temporal bone, and outputting the three-dimensional model of the temporal bone.

[0005] A second aspect of this application provides a temporal bone 3D modeling system based on CT images. The system includes: a 3D data acquisition module, which uses a high-resolution computer to perform tomographic scanning of the temporal bone of a target user to obtain HRCT image data, and performs preprocessing on the HRCT image data to construct 3D HRCT volume data; a multi-class probability map generation module, which inputs the 3D HRCT volume data into a 3D semantic segmentation network to generate a multi-class probability map, wherein the multi-class probability map describes the semantic probability distribution of each voxel under multiple structural categories; a semantic region mask generation module, which generates a semantic region mask based on the multi-class probability map, wherein the semantic region mask characterizes semantically reliable and semantically uncertain regions; an augmentation volume data generation module, which performs geometric consistency optimization on the 3D HRCT volume data under the constraints of the semantic region mask to generate geometrically consistent augmentation volume data; a 3D mesh model construction module, which performs 3D surface reconstruction based on the geometrically consistent augmentation volume data to generate a 3D mesh model of the temporal bone; and a 3D temporal bone model acquisition module, which performs model optimization processing on the 3D mesh model of the temporal bone and outputs the 3D temporal bone model.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages: This application constructs three-dimensional volume data by performing high-resolution tomographic scanning of the temporal bone and preprocessing the image data. A multi-class probability map is generated through a three-dimensional semantic segmentation network to obtain mask information for reliable and uncertain semantic prediction regions. Geometric consistency optimization is performed on the volume data under mask constraints. Then, a three-dimensional surface reconstruction and mesh model optimization process is used to generate a three-dimensional model of the temporal bone. Combined with semantically guided geometric optimization and multi-stage refined verification and adjustment, the complex anatomical structure and microstructural features of the temporal bone are accurately reproduced. This makes the structural segmentation, realistic reproduction, and microstructural recognition of the three-dimensional temporal bone model more stable and reliable. The technical effect of accurately reproducing the anatomical structure of the temporal bone, improving the stability and realism of the model, and making the model highly consistent with the actual anatomical state of the human temporal bone is achieved. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 This is a flowchart illustrating the method for three-dimensional modeling of the temporal bone based on CT images provided in the embodiments of this application.

[0009] Figure 2This is a schematic diagram of the structure of the temporal bone three-dimensional modeling system based on CT images provided in the embodiments of this application.

[0010] Figure labeling: 1. 3D data acquisition module; 2. Multi-class probability map generation module; 3. Semantic region mask generation module; 4. Augmentation volume data generation module; 5. 3D mesh model construction module; 6. Temporal bone 3D model acquisition module. Detailed Implementation

[0011] This application provides a method and system for three-dimensional modeling of the temporal bone based on CT images, which solves the technical problems of difficulty in recognizing small structures, semantic segmentation ambiguity, and geometric feature distortion during the modeling process due to the complex anatomical structure of the temporal bone.

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0013] It should be noted that the terms "first," "second," etc., in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0014] Example 1, as Figure 1 As shown, a three-dimensional modeling method for the temporal bone based on CT images is described, wherein the method includes: High-resolution computer is used to perform tomographic scanning of the temporal bone of the target user to obtain HRCT image data, and the HRCT image data is preprocessed to construct three-dimensional HRCT volume data.

[0015] In this embodiment, a high-resolution computer refers to a medical HRCT (High Resolution Computed Tomography) device with thin-slice scanning and ultra-high spatial resolution imaging capabilities. It is a specialized medical image acquisition device capable of performing high-definition tomographic scanning of the human temporal bone and outputting corresponding image data. HRCT images, specifically high-resolution computed tomographic images, refer to digital tomographic medical images that clearly display the fine anatomical structures of the temporal bone, obtained by performing thin-slice scanning of the target object's temporal bone using an HRCT device and processing it with a high-resolution reconstruction algorithm.

[0016] Specifically, the first step is to acquire DICOM (Digital Imaging and Communications in Medicine) sequence data of HRCT image data, read its spatial location information to reconstruct three-dimensional volume data, and then perform grayscale normalization, noise reduction filtering and isotropic resampling processing on the three-dimensional volume data in sequence to construct three-dimensional HRCT volume data. This step will be explained in detail later.

[0017] The three-dimensional HRCT volume data is input into a three-dimensional semantic segmentation network to generate a multi-class probability map, wherein the multi-class probability map is used to describe the semantic probability distribution of each voxel under multiple structural categories.

[0018] Optionally, a three-dimensional semantic segmentation network is first constructed. The preprocessed three-dimensional HRCT volume data is input into the three-dimensional semantic segmentation network. The multi-scale spatial features of the three-dimensional HRCT volume data are extracted through the encoder and decoder structure of the network. Then, the network output is subjected to probability normalization processing through the Softmax function. Finally, a multi-class probability map is generated to describe the semantic probability distribution of each voxel under multiple structural categories. This step will be explained in detail in the following content.

[0019] A semantic region mask is generated based on the multi-class probability map, wherein the semantic region mask is used to characterize semantically reliable regions and semantically uncertain regions.

[0020] In one embodiment of this application, firstly, the maximum class probability value in the semantic probability distribution of each voxel is extracted based on the multi-class probability map output by the three-dimensional semantic segmentation network. The maximum class probability value corresponding to each voxel is compared with a preset probability threshold. According to the comparison result, the voxels are respectively assigned to the semantic prediction reliable region or the semantic uncertainty region. Then, adjacency verification is performed on the two types of regions. After the verification is passed, a semantic region mask is generated to represent the semantic prediction reliable region and the semantic uncertainty region. This step will be described in detail in the following content.

[0021] Under the constraints of the semantic region mask, geometric consistency optimization is performed on the three-dimensional HRCT volume data to generate geometrically consistent enhanced volume data.

[0022] Specifically, firstly, the voxel spatial gradient information is calculated based on the 3D HRCT volume data, and the local surface normal vector is calculated accordingly. Then, the position of the semantically reliable region in the semantic region mask is optimized by normal consistency constraint to reduce the normal difference between neighboring voxels. The position of the semantically uncertain region is not subject to geometric consistency constraint. Finally, geometric consistency enhanced volume data is generated based on the geometric consistency constraint optimization results. This step will be explained in detail in the following content.

[0023] Three-dimensional surface reconstruction is performed based on the geometrically consistent augmentation data to generate a three-dimensional mesh model of the temporal bone.

[0024] Specifically, firstly, isosurface extraction is performed on the generated geometric consistency enhancement body data to obtain the corresponding isosurface extraction results. Then, a three-dimensional triangular mesh is constructed based on the isosurface extraction results, and finally, a three-dimensional mesh model of the temporal bone is generated. This step will be explained in detail later.

[0025] The temporal bone 3D mesh model is optimized, and the temporal bone 3D model is output.

[0026] Specifically, the generated temporal bone 3D mesh model is subjected to surface smoothing, mesh simplification, and hole repair processes in sequence. After the entire process is optimized, the final temporal bone 3D model is output. This step will also be explained in detail later.

[0027] Furthermore, the method provided in this application embodiment includes: Acquire DICOM sequence data from HRCT image data; read spatial location information from the DICOM sequence data and reconstruct three-dimensional volume data; perform grayscale normalization, noise reduction filtering, and isotropic resampling on the three-dimensional volume data to construct three-dimensional HRCT volume data.

[0028] In this embodiment of the application, DICOM is a common international standard format and communication protocol in the field of medical imaging. DICOM sequence data is a dataset of continuous temporal bone tomographic images that conforms to the standard specifications and contains spatial location metadata.

[0029] Specifically, firstly, a high-resolution computed tomography (HRCT) device is used to perform continuous thin-slice tomographic scans of the target user's temporal bone region. During the scan, a dedicated high-resolution scanning sequence for the temporal bone is employed, with slice thickness set to sub-millimeter levels of 0.5 mm or less. The scan area completely covers the entire anatomical structure of the temporal bone, from the bottom of the external auditory canal to the apex of the petrous bone. After the scan is completed, the raw HRCT image data of the target user's temporal bone is output. Following the scan, DICOM format sequence data of the target user's temporal bone HRCT image is extracted from the image storage module of the high-resolution computed tomography device. The extracted DICOM sequence data contains multiple consecutive frames of two-dimensional tomographic images arranged in the scan order, as well as the scanning device parameters and image spatial metadata information corresponding to each frame of the two-dimensional tomographic image.

[0030] Next, the extracted DICOM sequence data is parsed to read the spatial coordinate information, scanning layer spacing information, and planar pixel spacing information corresponding to each frame of the two-dimensional tomographic image in the sequence. According to the scanning arrangement order of the DICOM sequence, multiple consecutive frames of two-dimensional tomographic images are spatially stacked along the scanning direction to complete the reconstruction of the three-dimensional volume data and obtain the initial three-dimensional volume data of the corresponding temporal bone scanning area.

[0031] Next, the initial 3D volume data undergoes grayscale normalization, noise reduction filtering, and isotropic resampling standardization preprocessing. The specific steps are as follows: Step a: Perform grayscale normalization processing on the initial three-dimensional volume data obtained from the reconstruction. First, based on the conventional parameters of the temporal bone CT bone window, determine the effective grayscale range of 3000~4000HU for the window width and 500~700HU for the window level. Statistically analyze the grayscale value distribution range of all pixels in the initial three-dimensional volume data within the effective range. Then, through linear mapping, uniformly map the grayscale values ​​within the effective range to a fixed value range of 0 to 255 to eliminate the differences in image grayscale values ​​generated under different scanning equipment and different scanning parameters, so that temporal bone image data from different sources have a unified grayscale benchmark.

[0032] Step b: Perform noise reduction filtering on the 3D volume data. Use a 3D anisotropic filtering algorithm to traverse the 3D volume data pixel by pixel. The preferred size of the filtering kernel is 3×3×3 voxels, and the preferred number of iterations is 2 to 5. During the processing, adjust the filtering weight according to the gray-level gradient value in the neighborhood of the pixel. Perform high-intensity filtering on smooth areas with small gray-level gradient values ​​and low-intensity filtering on structural boundary areas with large gray-level gradient values. While preserving the anatomical structure boundary features in the image, filter out quantum noise and random noise generated during the scanning process to reduce the interference of image noise on the subsequent identification of anatomical structure boundaries.

[0033] Step c: Perform isotropic resampling on the 3D volume data. Based on the planar pixel spacing and scan layer spacing information read from the DICOM sequence data, determine the uniform voxel size after resampling. The voxel size is preferably between 0.3mm×0.3mm×0.3mm and 0.5mm×0.5mm×0.5mm. Use a trilinear interpolation algorithm to spatially resample the 3D volume data. During the resampling process, the gray value of the target voxel is calculated by weighting the gray values ​​of the eight adjacent voxels around the target voxel. The interpolation weight is inversely proportional to the spatial distance from the target voxel to the adjacent voxels, so that the voxels of the resampled 3D volume data have the same physical size in the three spatial dimensions of the X-axis, Y-axis, and Z-axis. Finally, a standardized 3D HRCT volume data is constructed.

[0034] Furthermore, the method provided in this application embodiment includes: A three-dimensional semantic segmentation network is constructed, and the three-dimensional HRCT volume data is input into the three-dimensional semantic segmentation network; multi-scale spatial features of the three-dimensional HRCT volume data are extracted through an encoder and decoder structure; the network output results are subjected to probability normalization processing through the Softmax function to generate a multi-class probability map.

[0035] Optionally, a three-dimensional semantic segmentation network is first constructed. The network adopts the 3DU-Net architecture, which is a symmetrical encoder-decoder U-shaped structure. The specific steps are as follows: The encoder section consists of four consecutive downsampling stages. The first downsampling stage has one input channel, followed by two 3×3×3 convolutional layers with a stride of 1 and padding of 1, resulting in 16 output channels. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function. After convolution, a 2×2×2 max-pooling layer with a stride of 2 is applied, completing the first downsampling. The spatial size of the output feature map is reduced to half that of the input, and the number of feature channels is doubled. The second to fourth downsampling stages maintain the same structure as the first, with the number of input channels equal to the number of output channels in the previous stage (32, 64, and 128 respectively). Each stage uses two 3×3×3 convolutional layers for feature extraction, followed by batch normalization and ReLU activation, and then a max-pooling layer for downsampling. This progressively extracts deep semantic features and multi-scale spatial features from the 3D volumetric data.

[0036] The decoder section is configured with four consecutive upsampling stages symmetrical to the encoder. The first upsampling stage has the same number of input channels as the encoder's final output of 128 channels. It first connects to a 2×2×2 three-dimensional deconvolution layer with a stride of 2 and an output of 64 channels. The feature map space size is restored to twice that of the input. Then, the feature map output by the deconvolution is concatenated with the feature map of the same size output before the pooling operation in the fourth downsampling stage of the encoder in the channel dimension. The concatenated feature map is then connected to two 3×3×3 three-dimensional convolution layers with a stride of 1 and padding of 1, and an output of 64 channels. Each three-dimensional convolution layer is followed by a batch normalization layer and a ReLU activation function. The second to fourth upsampling stages maintain the same structure as the first upsampling stage. The number of input channels in each stage is the same as the number of output channels in the previous stage. The number of deconvolution output channels is 32, 16, and 16 respectively. Each stage first performs feature map upsampling through a 3D deconvolution layer, and then performs channel concatenation with the feature map of the same size in the corresponding stage of the encoder. The feature fusion and extraction are completed through two 3D convolution layers, gradually restoring the spatial resolution of the feature map. At the same time, the anatomical boundary details in the shallow features and the semantic information in the deep features are fused.

[0037] The feature map output from the fourth upsampling stage of the decoder is fed into a 1×1×1 three-dimensional convolutional layer with a stride of 1 and padding of 0. The number of output channels of this convolutional layer is consistent with the number of preset temporal bone anatomical structure categories. The preset categories include multiple structure categories, such as background, temporal bone structure, ossicles, facial nerve canal, semicircular canals, and cholesteatoma lesions, which correspond to a set of 6 output channels. The feature map output by the 1×1×1 three-dimensional convolutional layer is the original prediction result of the network, and the size of the feature map is consistent with the size of the three-dimensional volume data input to the network.

[0038] Next, the training operation of the 3D semantic segmentation network is performed. First, the training dataset is constructed. HRCT image data of clinically diagnosed temporal bones are collected, and the corresponding DICOM sequence data is extracted. Gray-level normalization, noise reduction filtering, and isotropic resampling are performed to construct standardized 3D HRCT volume data. Otolaryngologists and radiologists with more than five years of clinical experience jointly perform voxel-level annotation on the 3D HRCT volume data. The annotation categories correspond one-to-one with the preset categories of the network output layer. The annotated 3D HRCT volume data are paired with the corresponding annotation labels and divided into training, validation, and test sets in an 8:1:1 ratio to complete the construction of the training dataset.

[0039] The 3D semantic segmentation network is trained using a combined loss function for multi-class classification tasks. This combined loss function is obtained by weighting and summing the Dice loss function and the cross-entropy loss function in a 1:1 ratio. The Adam optimizer is used for iterative optimization of the network parameters during training. The initial learning rate is set to 1×10^-4, the weight decay coefficient is set to 1×10^-5, and the learning rate decays every 30 epochs with a decay coefficient of 0.5. The batch size is set to 2, with a selectable range of 1 to 4. The maximum number of training epochs is set to 150, with a selectable range of 100 to 200 epochs. After each training epoch, the segmentation accuracy of the network is validated using validation set data. Training is stopped when the Dice coefficient on the validation set does not improve for 10 consecutive epochs, and the current network weight parameters are saved, resulting in the initial model of the trained 3D semantic segmentation network.

[0040] Finally, the initial model was tested using test set data. The passing criteria were: an overall Dice coefficient of no less than 0.85, a segmentation precision of no less than 0.88 for each category, and a recall of no less than 0.86. Models that meet the passing criteria are the final 3D semantic segmentation network models that can be used for inference.

[0041] Next, the preprocessed standardized 3D HRCT volumetric data is input into the trained 3D semantic segmentation network. The 3D HRCT volumetric data is single-channel 3D grayscale volumetric data, and the network's preset input size is 128×128×128 voxels. If the original 3D HRCT volumetric data size does not match the network's preset input size, trilinear interpolation resampling is used to adjust the volumetric data size to the preset input size, or a combination of center clipping and zero-value padding is used to achieve size adaptation, ensuring that the input volumetric data size is consistent with the network's preset input size. After the 3D HRCT volumetric data is input into the network, it first enters the encoder's downsampling path. Through continuous convolution and pooling operations, deep semantic features and spatial structural features at different scales are extracted from the volumetric data level by level. After each downsampling operation, the spatial size of the feature map is halved, and the number of feature channels is doubled, completing the initial extraction of multi-scale spatial features. The multi-scale spatial features extracted by the encoder are input to the upsampling path of the decoder. Through continuous deconvolution and feature concatenation operations, the spatial resolution of the feature map is restored step by step. At the same time, the shallow boundary features of the corresponding layer of the encoder and the deep semantic features of the decoder are fused through skip connections. While preserving the boundary details of the small anatomical structures of the temporal bone, the semantic distinguishability of different anatomical structures is enhanced. The complete extraction and fusion of multi-scale spatial features of the 3D HRCT volume data is completed, and the output is the original predicted feature map of the same size as the input 3D HRCT volume data.

[0042] Finally, the raw predicted feature map output by the network is input into the Softmax activation function to perform probability normalization on the raw multi-class predicted values ​​corresponding to each voxel. During the processing, for each voxel, its probability value in each preset class is calculated, so that the sum of the probability values ​​of each voxel in all classes is 1, ultimately generating a multi-class probability map of the same size as the input 3D HRCT volumetric data. The value at each position in the multi-class probability map corresponds to the semantic probability distribution of the voxel at the same spatial position in the 3D HRCT volumetric data under each structural category: background, temporal bone structure, ossicles, facial nerve canal, semicircular canals, and cholesteatoma lesion.

[0043] Furthermore, the method provided in this application embodiment includes: Based on the multi-class probability map, the maximum class probability value in the semantic probability distribution of each voxel is extracted; the maximum class probability value in the semantic probability distribution of each voxel is compared with a preset probability threshold. When the maximum class probability value is greater than the preset probability threshold, the voxel is added to the semantic prediction reliable region; when the maximum class probability value is less than or equal to the preset probability threshold, the voxel is added to the semantic uncertainty region; adjacency verification is performed on the semantic prediction reliable region and the semantic uncertainty region. If the adjacency verification passes, the semantic region mask is obtained.

[0044] Specifically, the first step is to obtain the multi-class probability map output by the 3D semantic segmentation network. This multi-class probability map is a 3D matrix structure with spatial dimensions identical to the preprocessed 3D HRCT volumetric data. Each element in the matrix corresponds to the semantic probability value of the voxel at the same location in the 3D HRCT volumetric data under various preset anatomical structure categories. The multi-class probability map is traversed element-wise using the 3D matrix. For each voxel location, all corresponding category semantic probability values ​​are read. By comparing the numerical values, the maximum value in the set is selected as the maximum category probability value for that voxel. Simultaneously, the anatomical structure category label corresponding to this maximum value is recorded. After the full traversal is complete, a maximum category probability value matrix of the same size as the multi-class probability map is generated, where each element is the maximum category probability value for the corresponding voxel.

[0045] A probability threshold for determining the reliability of semantic prediction is preset. The value of the probability threshold ranges from 0.7 to 0.9, with a preferred value of 0.8. This threshold can be linearly adjusted within the range according to the accuracy requirements of the actual segmentation scenario. An element-wise comparison method is used to traverse each element in the maximum class probability value matrix. The maximum class probability value corresponding to each voxel is compared one-to-one with a preset probability threshold. When the maximum class probability value of a voxel is greater than the preset probability threshold, the semantic prediction result of that voxel is determined to have high reliability. The spatial location of that voxel is classified into the semantic prediction reliable region, and an anatomical structure category label corresponding to its maximum class probability value is assigned to that voxel. When the maximum class probability value of a voxel is less than or equal to the preset probability threshold, the semantic prediction result of that voxel is determined to have uncertainty. The spatial location of that voxel is classified into the semantic uncertainty region, and no fixed anatomical structure category label is assigned to that voxel. After the region division is completed, the spatial localization of the reliable region and the uncertainty region is initially completed. The semantic uncertainty region is mainly distributed at the boundary of the fine anatomical structure of the temporal bone, including the structural edges of the ossicles, facial nerve canal, and semicircular canals, as well as the boundary between cholesteatoma lesions and normal bone structures. This type of region has blurred structural boundaries and drastic gray-level gradient changes, and belongs to the difficult sample region of semantic segmentation.

[0046] Then, each voxel in the semantic prediction reliable region and semantic uncertain region is traversed and the neighboring voxel region consistency coefficient is calculated. The calculated neighboring voxel region consistency coefficient is compared with the preset coefficient threshold. If the coefficient is greater than or equal to the preset coefficient threshold, the adjacency verification is determined to be passed. If the coefficient is less than the preset coefficient threshold, the adjacency verification is determined to be failed. This step will be explained in detail in the following content.

[0047] Finally, after completing the full-range adjacency verification and region adjustment, two sets of binary mask matrices of the same size as the 3D HRCT volume data are generated, namely the semantically predictable reliable region mask and the semantically uncertain region mask. The two sets of mask matrices together form the semantic region mask. In the semantically predictable reliable region mask, the voxel positions corresponding to the semantically predictable reliable region are assigned a value of 1, and the other positions are assigned a value of 0. In the semantically uncertain region mask, the voxel positions corresponding to the semantically uncertain region are assigned a value of 1, and the other positions are assigned a value of 0.

[0048] By extracting voxel probability extrema from multi-class probability maps, performing initial thresholding of regions, and optimizing adjacency consistency verification, we achieved accurate segmentation and standardized labeling of reliable and uncertain semantic prediction regions in 3D images of the temporal bone, providing accurate spatial constraints for subsequent differentiated geometric consistency optimization.

[0049] Furthermore, the method provided in this application embodiment includes: Traverse each voxel in the semantically reliable region and the semantically uncertain region, and calculate the neighboring voxel region consistency coefficient; if the neighboring voxel region consistency coefficient is greater than or equal to a preset coefficient threshold, the adjacency verification passes; if the neighboring voxel region consistency coefficient is less than the preset coefficient threshold, the adjacency verification fails.

[0050] Specifically, firstly, the region labeling matrix obtained from the preliminary region segmentation steps described above is acquired. This region labeling matrix is ​​a three-dimensional matrix of the same size as the 3D HRCT volumetric data. Each element at each voxel position in the matrix corresponds to its region category label, where semantically reliable regions correspond to a label value of 1, and semantically uncertain regions correspond to a label value of 0. The region labeling matrix is ​​traversed element-wise using a three-dimensional matrix traversal method. For each target voxel traversed, its three-dimensional spatial neighborhood range is determined. The neighborhood range can be selected based on the segmentation accuracy requirements: 3×3×1 (18 neighbors), 3×3×3 (26 neighbors), or 5×5×5 (124 neighbors), with 3×3×3 (26 neighbors) being preferred. When the target voxel is located at the spatial boundary of the 3D HRCT volumetric data, and the selected neighborhood range exceeds the volumetric data spatial boundary, only valid neighboring voxels within the neighborhood range that are within the volumetric data space are counted. Invalid positions exceeding the boundary are removed, completing the acquisition of valid neighboring voxels of the target voxel, and the region category label corresponding to each valid neighboring voxel is read.

[0051] Next, the number of voxels whose region category labels match the target voxel's among the collected valid neighboring voxels is counted, denoted as the consistent voxel count. Simultaneously, the total number of valid neighboring voxels within the selected neighborhood is counted, denoted as the total neighborhood voxel count. The neighboring voxel region consistency coefficient is calculated using a division operation, calculated as the ratio of the consistent voxel count to the total neighborhood voxel count. The resulting ratio is the neighboring voxel region consistency coefficient for the target voxel, ranging from 0 to 1. A higher coefficient value indicates a higher consistency between the target voxel and its neighboring voxels in terms of region category, and thus a more reliable region division result. After completing the full range traversal, a consistency coefficient matrix of the same size as the region label matrix is ​​generated, where each element represents the neighboring voxel region consistency coefficient for the corresponding voxel.

[0052] A pre-set threshold for regional consistency determination was established, determined through statistical analysis in a preliminary experiment. During the preliminary experiment, 100 sets of temporal bone HRCT body data samples were collected. The inclusion criteria were sub-millimeter temporal bone HRCT data with a slice thickness of 0.5 mm or less, covering both normal temporal bone anatomical structures and diseased temporal bone samples with cholesteatoma lesions. The pathological types of the samples covered all pre-defined anatomical structure categories. Multi-class probability maps and preliminary region division were completed for the samples. Multiple candidate coefficient thresholds ranging from 0.4 to 0.8, with intervals of 0.05, were used to perform adjacency validation and region adjustment for each set of samples. The accuracy of category labeling for semantically reliable regions and the integrity of minute structural boundaries in semantically uncertain regions were statistically analyzed under different candidate thresholds. Using the criteria of annotation accuracy of not less than 95% and boundary integrity of not less than 90%, a candidate threshold range that meets the criteria is determined, and the median value of the range is finally selected as the preferred value of the preset coefficient threshold. For example, if the candidate threshold range is 0.5 to 0.7, the preset coefficient threshold is 0.6. The coefficient threshold can be linearly adjusted within the range of 0.5 to 0.7 according to the accuracy requirements of the actual application scenario.

[0053] Then, an element-wise comparison method is used to traverse each element in the consistency coefficient matrix. The consistency coefficient of the neighboring voxel region corresponding to each target voxel is compared one-to-one with a preset coefficient threshold. When the consistency coefficient of the neighboring voxel region of the target voxel is greater than or equal to the preset coefficient threshold, the region division result of the target voxel is determined to meet the neighborhood space consistency requirement, the adjacency verification is passed, and the original region category label of the target voxel is retained without adjustment. When the consistency coefficient of the neighboring voxel region of the target voxel is less than the preset coefficient threshold, the region division result of the target voxel is determined to not meet the neighborhood space consistency requirement, and there is a possibility of misclassification of the region. The adjacency verification is failed, the target voxel is marked as a voxel to be adjusted, and a binary label matrix to be adjusted with the same size as the region label matrix is ​​generated. The corresponding position of the voxel to be adjusted in the matrix is ​​assigned a value of 1, and the other positions are assigned a value of 0, completing the standardization labeling and storage of the voxel to be adjusted.

[0054] By performing full-range neighborhood traversal, boundary voxel adaptation processing, quantitative calculation of the consistency coefficient of adjacent voxel regions, standardized pre-experiment to determine the judgment threshold, and graded judgment of verification results and marking of voxels to be adjusted, the reliability verification of temporal bone image region division results and accurate identification of misclassification results were achieved. This provides quantitative judgment basis and standardized connection data for subsequent region adjustment and generation of high-precision semantic region masks.

[0055] Furthermore, the method provided in this application embodiment includes: When adjacency verification fails, determine whether the voxel belongs to the semantically predictable reliable region. If it does, add it to the semantically uncertain region and remove it from the semantically predictable reliable region; otherwise, do not perform any operation.

[0056] Specifically, the process first reads the binarized adjustment marker matrix, region marker matrix, semantically predictable reliable region mask matrix, semantically uncertain region mask matrix, and corresponding voxel anatomical structure category label matrix generated through the aforementioned adjacency verification step. All matrices are three-dimensional matrices of the same size as the three-dimensional HRCT volume data and have a one-to-one spatial mapping relationship. A method of element-wise traversal of the three-dimensional matrix is ​​used to perform a full-range traversal of the binarized adjustment marker matrix, locating all voxels with a marker value of 1. These voxels are the adjustment voxels that failed the adjacency verification, thus completing the full-range localization of the adjustment voxels and the matching and reading of the corresponding data.

[0057] For each voxel to be adjusted that has been located, the original region category label corresponding to that voxel in the region label matrix is ​​read, and a region attribution determination operation is performed. In the region category label, semantically reliable regions correspond to a fixed label value of 1, and semantically uncertain regions correspond to a fixed label value of 0. By numerical matching, it is determined whether the original attribution region of the voxel to be adjusted is a semantically reliable region, and a clear binary judgment result is obtained.

[0058] When the classification result of the voxel to be adjusted is that it belongs to the semantically predicted reliable region, the region adjustment operation is performed. First, in the semantically predicted reliable region mask matrix, the value of the spatial position corresponding to the voxel to be adjusted is changed from 1 to 0, completing the removal of the voxel from the semantically predicted reliable region; simultaneously, in the semantically uncertain region mask matrix, the value of the spatial position corresponding to the voxel to be adjusted is changed from 0 to 1, completing the addition of the voxel to the semantically uncertain region; at the same time, in the anatomical structure category label matrix, the anatomical structure category label corresponding to the voxel is cleared, so that the voxel has no fixed category label constraint, completing the full-process region adjustment of a single voxel.

[0059] When the classification result of the voxel to be adjusted is that it does not belong to the semantically predictable reliable region, that is, the original classification of the voxel is the semantically uncertain region, no region adjustment operation is performed. The original assignment of the voxel in the region label matrix, the semantically predictable reliable region mask matrix, and the semantically uncertain region mask matrix remains unchanged. At the same time, the original state of the voxel without a fixed anatomical structure category label is retained to avoid damaging the boundary integrity of the semantically uncertain region.

[0060] After completing the judgment and corresponding operation for all voxels to be adjusted, a consistency check of the region adjustment results is performed. An element-by-element pairwise comparison method is used to traverse the updated semantic prediction reliable region mask matrix and the semantic uncertain region mask matrix, checking that there are no conflicting cases where voxel assignments at the same spatial location in the two matrices are simultaneously set to 1. After the check is completed, the updated semantic prediction reliable region mask matrix and the semantic uncertain region mask matrix are integrated to generate the final semantic region mask.

[0061] By accurately locating voxels that fail adjacency verification, performing binary region attribution determination, and conducting differentiated region adjustment operations and result consistency verification, the system achieves accurate correction of misclassified voxels within the reliable semantic prediction region. At the same time, it fully preserves the original boundary information of semantically uncertain regions, further improving the segmentation accuracy and reliability of semantic region masks, and providing accurate spatial constraints for subsequent differentiated geometric consistency optimization.

[0062] Furthermore, the method provided in this application embodiment includes: Voxel spatial gradient information is calculated based on the three-dimensional HRCT volume data; local surface normal vectors are calculated based on the voxel spatial gradient information; normal consistency constraint optimization is performed on the positions marked as semantically reliable regions in the semantic region mask to reduce the normal differences between neighboring voxels; geometric consistency constraints are not applied to the positions marked as semantically uncertain regions in the semantic region mask; geometric consistency enhanced volume data is generated based on the geometric consistency constraint optimization results.

[0063] In one embodiment, preprocessed standardized 3D HRCT volumetric data is first read. This volumetric data is a single-channel 3D grayscale matrix, where each voxel position corresponds to a unique grayscale value and has coordinate information in three spatial dimensions: X-axis, Y-axis, and Z-axis. The physical dimensions of the voxels are consistent in all three dimensions. The 3D central difference method is used to perform a full-range voxel traversal of the 3D HRCT volumetric data, calculating the grayscale gradient components of each voxel in the three spatial dimensions. The gradient component in the X-axis direction is calculated by subtracting the grayscale value of the adjacent voxels in the negative X-axis direction from the grayscale value of the adjacent voxels in the positive X-axis direction, and then dividing by twice the spatial size of a single voxel. The gradient components in the Y-axis and Z-axis directions are obtained using the same calculation logic. The gradient components of a single voxel in the three dimensions are combined to form the 3D spatial gradient vector of that voxel. After the full-range traversal calculation is completed, a 3D gradient vector field of the same size as the 3D HRCT volumetric data is generated, completing the calculation of the voxel spatial gradient information.

[0064] For each voxel in the 3D HRCT volumetric data, its corresponding 3D spatial gradient vector is read, and the local surface normal vector is calculated and normalized. First, the magnitude of the 3D spatial gradient vector is calculated. The magnitude is the square root of the sum of the squares of the three components of the gradient vector. Each component of the gradient vector is divided by this magnitude to obtain a normalized 3D vector, which is the local surface normal vector corresponding to that voxel. When the magnitude of the voxel's 3D spatial gradient vector is 0, the voxel is determined to be in a uniform grayscale region with no clear surface orientation, and its local surface normal vector is assigned the value of a 3D zero vector. After the full-range voxel-by-voxel calculation is completed, a 3D normal vector field of the same size as the 3D HRCT volumetric data is generated, with each voxel position corresponding to a unique normalized local surface normal vector.

[0065] Next, the semantic region mask is read. This mask contains spatial location markers that correspond one-to-one with the 3D HRCT volumetric data, accurately distinguishing between semantically reliable and semantically uncertain regions. A local neighborhood weighted smoothing method is used to perform normal consistency constraint optimization on voxels marked as semantically reliable regions in the semantic region mask. The neighborhood range for optimization calculation is determined to be a 3×3×3 26-neighborhood. All target voxels within semantically reliable regions are traversed, and effective neighboring voxels within the 26-neighborhood of each target voxel that also belong to semantically reliable regions are extracted. The local surface normal vectors corresponding to all effective neighboring voxels are read. A Gaussian weighted average is calculated for the effective voxel normal vectors within the neighborhood, with the Gaussian kernel standard deviation set to 1. The weight coefficient of a single neighboring voxel is calculated using the following formula: ,in Let σ be the Euclidean distance between neighboring voxels and the target voxel, and let σ be the standard deviation of the Gaussian kernel. Neighboring voxels closer to the target voxel in spatial distance have higher weight coefficients. The normal vectors of all effective neighboring voxels are multiplied by their corresponding weight coefficients, summed, and then divided by the sum of the weight coefficients to obtain the optimized local surface normal vector of the target voxel. The optimization iteration count is set to 2. In each iteration, the normal vector is updated only for voxels within the semantically reliable region, gradually reducing the normal vector differences between neighboring voxels and strengthening the consistency of the local surface normal vectors.

[0066] Traverse all voxels marked as semantically uncertain regions in the semantic region mask. For all voxels in this region, do not perform any normal consistency constraint optimization operation, and completely preserve the original calculated local surface normal vector. At the same time, voxels in this region do not participate in the normal optimization calculation process of adjacent semantically reliable region voxels, so as to avoid the optimization operation from damaging the detailed features of the fine anatomical structure boundary of the temporal bone and the edge region of the lesion.

[0067] Finally, the optimized 3D normal vector field is obtained, and the optimized 3D gradient vector field is reconstructed based on the optimized 3D normal vector field. The 3D Poisson reconstruction method is used to perform grayscale reconstruction on the optimized 3D gradient vector field. The reconstruction process uses Neumann boundary conditions, the number of iterations is set to 20, and the convergence threshold is set to 1e-6. The optimized grayscale value of each voxel is obtained by solving the problem, and a 3D grayscale volume data with the same size, spatial coordinate system, and grayscale value range as the original 3D HRCT volume data is generated. This volume data is the geometric consistency enhancement volume data.

[0068] By performing differentiated normal consistency constraint optimization on semantically reliable regions, the original information of semantically uncertain regions is fully preserved, achieving precise enhancement of the geometric consistency of temporal bone three-dimensional volume data. While enhancing the surface smoothness of reliable regions, the detailed features of fine structures such as ossicles, facial nerve canal, and lesion boundaries are fully preserved, providing a high-quality volume data foundation for subsequent high-precision three-dimensional surface reconstruction.

[0069] Furthermore, the method provided in this application embodiment includes: The geometric consistency enhancement data is subjected to isosurface extraction processing to obtain isosurface extraction results; a three-dimensional triangular mesh is constructed based on the isosurface extraction results to generate a three-dimensional mesh model of the temporal bone.

[0070] Optionally, first, geometrically consistent volumetric data is read. This volumetric data is a single-channel 3D grayscale volumetric data with the same size and spatial coordinate system as the original 3D HRCT volumetric data. Each voxel position corresponds to a unique grayscale value, and the grayscale value range linearly corresponds to the CT value range of the original HRCT image. The target grayscale threshold for isosurface extraction is preset. The CT value distribution range of temporal bone structures is 100HU to 3000HU. 150HU is selected as the basic threshold for isosurface extraction of bone structures. It can be linearly adjusted within the range of 100HU to 500HU according to the density characteristics of the target anatomical structure. The threshold for fine hard tissue structures such as the ossicles and facial nerve canal is adapted to 200HU to 300HU, the threshold for semicircular canals is adapted to 150HU to 250HU, and the threshold for bone structures around cholesteatoma lesions is adapted to 100HU to 200HU. The grayscale boundary between bone structures and surrounding soft tissues is clearly distinguished, and the preset isosurface extraction parameters are completed.

[0071] Then, the moving cube algorithm is used to perform full-range isosurface extraction on the geometrically consistent enhanced volume data. During execution, the algorithm sequentially traverses each cube cell along the positive X, Y, and Z axes, dividing the spatial range of the 3D grayscale volume data into multiple continuous and non-overlapping cube cells. The eight vertices of each cube cell correspond to eight adjacent voxels in the 3D volume data. Shared vertices of adjacent cube cells are merged, and duplicate vertices with spatial coordinate deviations less than one voxel size are averaged and merged, eliminating redundant vertex data. The grayscale values ​​corresponding to the eight vertices of each cube cell are read and compared with a preset target grayscale threshold to determine whether each vertex is inside or outside the isosurface. Based on the internal and external distribution of the eight vertices, the intersecting edges and triangular facet topology of the isosurfaces within the cube cell are determined. The coordinates of the intersection points of the isosurfaces and the cube cell's intersecting edges are calculated, completing the isosurface fragment extraction for a single voxel cube cell. By traversing all cube cells within the 3D volume data and integrating the isosurface fragments of all cube cells, a continuous closed isosurface extraction result is obtained.

[0072] Next, based on the vertex coordinates and topological connectivity of all triangular facets in the isosurface extraction results, a 3D triangular mesh is constructed. Spatial coordinate mapping is performed on the vertex coordinates of all triangular facets, converting the voxel coordinate system of the 3D grayscale data into the real physical spatial coordinate system based on DICOM sequence data, ensuring that the size ratio of the 3D mesh is completely consistent with the actual anatomical size of the target user's temporal bone. A topological consistency check is then performed on the constructed 3D triangular mesh, removing duplicate vertices and triangular facets, correcting facet connectivity relationships with topological errors, and ensuring that all facets of the 3D triangular mesh are manifold structures with no self-intersections and no redundant facets. After the check is completed, a 3D mesh model of the temporal bone with a complete anatomical topology is generated.

[0073] Furthermore, the method provided in this application embodiment includes: The surface smoothing, mesh simplification, and hole repair processes are performed on the three-dimensional mesh model of the temporal bone to output the three-dimensional model of the temporal bone.

[0074] In one embodiment, the temporal bone 3D mesh model constructed in the aforementioned steps is a 3D triangular manifold mesh built based on a real physical space coordinate system, consisting of multiple triangular facets and vertices topologically connected, preserving the complete anatomical structural features and spatial size proportions of the temporal bone. The model is then subjected to surface smoothing, mesh simplification, and hole repair processing in sequence, as detailed below: Step d: Perform surface smoothing on the 3D mesh model of the temporal bone. The Laplacian smoothing algorithm is used to eliminate jagged edges on the mesh surface. All vertices in the model are traversed. For each target vertex, its one-ring neighbor vertices in the triangular mesh are extracted. The weighted average of the spatial coordinates of the neighboring vertices is calculated, with the weighting coefficients calculated according to the formula: calculate, Let be the Euclidean distance between the target vertex and its neighboring vertices. When the distance is 0, the weight coefficient is assigned a value of 1, and the closer the distance, the larger the weight coefficient. The smoothing iteration count is set to 5 times, with each iteration only fine-tuning the coordinates of the mesh vertices. This eliminates burrs and irregular protrusions on the mesh surface while preserving the contour features of fine anatomical structures such as the ossicles and facial nerve canal. After smoothing, an effectiveness verification is performed. The passing criteria are a burr elimination rate of no less than 90% and a contour deviation of no more than 0.1 mm for fine anatomical structures. Upon successful verification, a smooth 3D mesh model of the temporal bone is obtained. This process reduces jagged edges and uneven structures on the mesh surface while preserving the fine anatomical features of the temporal bone, achieving a smooth mesh surface.

[0075] Step e: Perform mesh simplification processing on the smoothed temporal bone 3D mesh model, and use an edge folding algorithm to reasonably reduce the number of facets in the model. First, calculate the folding cost of each edge in the model. The folding cost is calculated according to the formula: Quantization calculation, where L is the actual length of the edge. Let θ be the maximum length of an edge in the model, θ be the normal angle between the triangular faces on both sides of the edge, and S be the importance coefficient of vertex anatomical features. Vertices with critical anatomical structures are assigned a value of 0.1 for S, while non-critical vertices are assigned a value of 1. A lower cost value indicates a smaller impact on the model's anatomical morphology after folding. The mesh simplification retention ratio is set to 80%, which can be linearly adjusted within the range of 70% to 90% depending on the actual application scenario. Folding operations are performed on the edges in the model sequentially in ascending order of folding cost, merging the two vertices of a folded edge into a new vertex. Redundant triangular faces and invalid vertices generated after folding are removed. Vertices and faces of key anatomical structures of the temporal bone are retained during the folding process. After simplification, validity verification is performed. The pass criteria are that the anatomical morphology similarity between the simplified model and the original model is not less than 95%, and the number of vertices meets the preset retention ratio. After successful verification, a 3D mesh model of the temporal bone that meets the lightweight requirements is obtained. This achieves lightweight simplification of the mesh model by reducing the number of triangular faces while ensuring that the overall anatomical structure and key fine structures of the temporal bone are not lost.

[0076] Step f: Perform hole repair processing on the simplified temporal bone 3D mesh model. First, accurately detect mesh holes. Traverse all edges in the model and count the number of edges shared by triangular patches. Mark edges with a sharing count of 1 as open edges. The closed area enclosed by connecting adjacent open edges in sequence is the hole region in the mesh. Extract all boundary vertices of the hole region and sort them in order according to spatial adjacency. Use the Delaunay triangulation filling method. Based on the ordered boundary vertices, follow the empty circle criterion to connect adjacent vertices in sequence to generate new triangular patches. Gradually fill the entire hole region. After filling, extract the neighboring patches of the newly generated patches, calculate the mean normal of the neighboring patches, and adjust the normal of the new patches to make the normal of the new patches consistent with the normal of the neighboring patches. Complete the topological and geometric feature adaptation of the hole region and ensure the integrity of the model's topological structure.

[0077] After completing surface smoothing, mesh simplification, and hole repair, the model undergoes an overall topological consistency check. Redundant vertices and triangular faces generated during the process are removed, and potential issues such as face self-intersection and topological connection errors are corrected. The model retains the true anatomical features, spatial dimensions, and fine structural details of the temporal bone, ultimately generating and outputting a 3D model of the temporal bone that meets the requirements for clinical application and visualization.

[0078] In summary, the temporal bone three-dimensional modeling method based on CT images provided in this application has the following technical effects: This application constructs three-dimensional volume data by preprocessing temporal bone HRCT image data, generates multi-class probability maps through three-dimensional semantic segmentation, generates semantic region masks through thresholding and adjacency verification, performs geometric consistency optimization in combination with mask constraints, and accurately generates a three-dimensional model of the temporal bone through three-dimensional surface reconstruction and model optimization. This achieves the technical effect of accurately restoring the anatomical structure of the temporal bone, improving the stability and realism of the model, and making the model highly consistent with the actual anatomical state of the human temporal bone.

[0079] Example 2, as Figure 2 As shown, based on the same inventive concept as in Embodiment 1 above, this application provides a three-dimensional modeling system for the temporal bone based on CT images, the system comprising: The three-dimensional data acquisition module 1 uses a high-resolution computer to perform a tomographic scan of the temporal bone of the target user to obtain HRCT image data, and performs preprocessing on the HRCT image data to construct three-dimensional HRCT volume data.

[0080] Multi-class probability map generation module 2 is used to input the three-dimensional HRCT volume data into a three-dimensional semantic segmentation network to generate a multi-class probability map, wherein the multi-class probability map is used to describe the semantic probability distribution of each voxel under multiple structural categories.

[0081] The semantic region mask generation module 3 is used to generate a semantic region mask based on the multi-class probability map, wherein the semantic region mask is used to characterize semantically reliable regions and semantically uncertain regions.

[0082] Augmentation volume data generation module 4 is used to perform geometric consistency optimization on the three-dimensional HRCT volume data under the constraints of the semantic region mask to generate geometrically consistent augmentation volume data.

[0083] The three-dimensional mesh model construction module 5 is used to perform three-dimensional surface reconstruction based on the geometric consistency enhancement data to generate a three-dimensional mesh model of the temporal bone.

[0084] Temporal bone 3D model acquisition module 6 is used to perform model optimization processing on the temporal bone 3D mesh model and output the temporal bone 3D model.

[0085] Furthermore, the three-dimensional data acquisition module 1 is used to perform the following steps: Acquire DICOM sequence data from HRCT image data; read spatial location information from the DICOM sequence data and reconstruct three-dimensional volume data; perform grayscale normalization, noise reduction filtering, and isotropic resampling on the three-dimensional volume data to construct three-dimensional HRCT volume data.

[0086] Furthermore, the multi-class probability map generation module 2 is used to perform the following steps: A three-dimensional semantic segmentation network is constructed, and the three-dimensional HRCT volume data is input into the three-dimensional semantic segmentation network; multi-scale spatial features of the three-dimensional HRCT volume data are extracted through an encoder and decoder structure; the network output results are subjected to probability normalization processing through the Softmax function to generate a multi-class probability map.

[0087] Furthermore, the semantic region mask generation module 3 is used to perform the following steps: Based on the multi-class probability map, the maximum class probability value in the semantic probability distribution of each voxel is extracted; the maximum class probability value in the semantic probability distribution of each voxel is compared with a preset probability threshold. When the maximum class probability value is greater than the preset probability threshold, the voxel is added to the semantic prediction reliable region; when the maximum class probability value is less than or equal to the preset probability threshold, the voxel is added to the semantic uncertainty region; adjacency verification is performed on the semantic prediction reliable region and the semantic uncertainty region. If the adjacency verification passes, the semantic region mask is obtained.

[0088] Furthermore, the semantic region mask generation module 3 is used to perform the following steps: Traverse each voxel in the semantically reliable region and the semantically uncertain region, and calculate the neighboring voxel region consistency coefficient; if the neighboring voxel region consistency coefficient is greater than or equal to a preset coefficient threshold, the adjacency verification passes; if the neighboring voxel region consistency coefficient is less than the preset coefficient threshold, the adjacency verification fails.

[0089] Furthermore, the semantic region mask generation module 3 is used to perform the following steps: When adjacency verification fails, determine whether the voxel belongs to the semantically predictable reliable region. If it does, add it to the semantically uncertain region and remove it from the semantically predictable reliable region; otherwise, do not perform any operation.

[0090] Furthermore, the augmentation data generation module 4 is used to perform the following steps: Voxel spatial gradient information is calculated based on the three-dimensional HRCT volume data; local surface normal vectors are calculated based on the voxel spatial gradient information; normal consistency constraint optimization is performed on the positions marked as semantically reliable regions in the semantic region mask to reduce the normal differences between neighboring voxels; geometric consistency constraints are not applied to the positions marked as semantically uncertain regions in the semantic region mask; geometric consistency enhanced volume data is generated based on the geometric consistency constraint optimization results.

[0091] Furthermore, the three-dimensional mesh model construction module 5 is used to perform the following steps: The geometric consistency enhancement data is subjected to isosurface extraction processing to obtain isosurface extraction results; a three-dimensional triangular mesh is constructed based on the isosurface extraction results to generate a three-dimensional mesh model of the temporal bone.

[0092] Furthermore, the temporal bone three-dimensional model acquisition module 6 is used to perform the following steps: The surface smoothing, mesh simplification, and hole repair processes are performed on the three-dimensional mesh model of the temporal bone to output the three-dimensional model of the temporal bone.

[0093] The CT image-based temporal bone three-dimensional modeling system provided in this embodiment of the invention can execute the CT image-based temporal bone three-dimensional modeling method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0094] Although this application makes various references to certain modules in the system according to the embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of this invention.

[0095] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application. In some cases, the actions or steps described in this application can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for three-dimensional modeling of the temporal bone based on CT images, characterized in that, The method includes: High-resolution computer is used to perform tomographic scanning of the temporal bone of the target user to obtain HRCT image data, and the HRCT image data is preprocessed to construct three-dimensional HRCT volume data. The three-dimensional HRCT volume data is input into a three-dimensional semantic segmentation network to generate a multi-class probability map, wherein the multi-class probability map is used to describe the semantic probability distribution of each voxel under multiple structural categories; A semantic region mask is generated based on the multi-class probability map, wherein the semantic region mask is used to characterize semantically reliable regions and semantically uncertain regions; Under the constraints of the semantic region mask, geometric consistency optimization is performed on the three-dimensional HRCT volume data to generate geometrically consistent enhanced volume data; Three-dimensional surface reconstruction is performed based on the geometrically consistent augmentation data to generate a three-dimensional mesh model of the temporal bone. The model optimization process is performed on the three-dimensional mesh model of the temporal bone, and the three-dimensional model of the temporal bone is output. A semantic region mask is generated based on the multi-class probability map, wherein the semantic region mask is used to characterize semantically reliable regions and semantically uncertain regions, including: Based on the multi-class probability map, extract the maximum class probability value from the semantic probability distribution of each voxel; Compare the maximum class probability value in the semantic probability distribution of each voxel with a preset probability threshold. When the maximum class probability value is greater than the preset probability threshold, add the voxel to the semantic prediction reliable region. When the maximum category probability value is less than or equal to the preset probability threshold, the voxel is added to the semantic uncertainty region. Adjacency verification is performed on the semantically predictable reliable region and the semantically uncertain region. If the adjacency verification passes, the semantic region mask is obtained. Adjacency verification is performed on the semantically predictable reliable region and the semantically uncertain region, including: Traverse each voxel in the semantically reliable region and the semantically uncertain region, and calculate the neighboring voxel region consistency coefficient; If the consistency coefficient of adjacent voxel regions is greater than or equal to the preset coefficient threshold, the adjacency verification passes. If the consistency coefficient of adjacent voxel regions is less than the preset coefficient threshold, the adjacency verification fails. When the adjacency verification fails, determine whether the voxel belongs to the semantically predictable reliable region. If so, add it to the semantically uncertain region and remove it from the semantically predictable reliable region. If not, the operation will not be performed.

2. The method for three-dimensional modeling of the temporal bone based on CT images as described in claim 1, characterized in that, High-resolution computed tomography (HRCT) scans of the target user's temporal bone are performed to obtain HRCT image data. Preprocessing of the HRCT image data is then performed to construct three-dimensional HRCT volumetric data, including: Acquire DICOM sequence data from HRCT images; Spatial location information is read from the DICOM sequence data and three-dimensional volume data is reconstructed. The three-dimensional volume data is subjected to grayscale normalization, noise reduction filtering, and isotropic resampling to construct three-dimensional HRCT volume data.

3. The method for three-dimensional modeling of the temporal bone based on CT images as described in claim 1, characterized in that, The three-dimensional HRCT volume data is input into a three-dimensional semantic segmentation network to generate a multi-class probability map, including: A three-dimensional semantic segmentation network is constructed, and the three-dimensional HRCT volume data is input into the three-dimensional semantic segmentation network; Multi-scale spatial features of the three-dimensional HRCT volume data are extracted using an encoder and decoder structure. The network output is normalized using the Softmax function to generate a multi-class probability map.

4. The method for three-dimensional modeling of the temporal bone based on CT images as described in claim 1, characterized in that, Under the constraints of the semantic region mask, geometric consistency optimization is performed on the 3D HRCT volume data to generate geometrically consistent enhanced volume data, including: Calculate the voxel spatial gradient information based on the three-dimensional HRCT volume data; Calculate the local surface normal vector based on the voxel space gradient information; For the locations marked as semantically reliable regions in the semantic region mask, normal consistency constraint optimization is performed to reduce the normal differences between neighboring voxels; No geometric consistency constraint is applied to the positions marked as semantically uncertain regions in the semantic region mask; Generate geometrically consistent augmented body data based on the optimization results of geometric consistency constraints.

5. The method for three-dimensional modeling of the temporal bone based on CT images as described in claim 1, characterized in that, Based on the geometrically consistent augmentation data, a three-dimensional surface reconstruction is performed to generate a three-dimensional mesh model of the temporal bone, including: Perform isosurface extraction processing on the geometric consistency enhancement volume data to obtain isosurface extraction results; A three-dimensional triangular mesh is constructed based on the isosurface extraction results to generate a three-dimensional mesh model of the temporal bone.

6. The method for three-dimensional modeling of the temporal bone based on CT images as described in claim 1, characterized in that, The surface smoothing, mesh simplification, and hole repair processes are performed on the three-dimensional mesh model of the temporal bone to output the three-dimensional model of the temporal bone.

7. A three-dimensional modeling system for the temporal bone based on CT images, characterized in that, The system is used to implement the CT image-based three-dimensional modeling method for the temporal bone according to any one of claims 1-6, the system comprising: The three-dimensional data acquisition module uses a high-resolution computer to perform a tomographic scan of the temporal bone of the target user to obtain HRCT image data, and performs preprocessing on the HRCT image data to construct three-dimensional HRCT volume data. A multi-class probability map generation module is used to input the three-dimensional HRCT volume data into a three-dimensional semantic segmentation network to generate a multi-class probability map, wherein the multi-class probability map is used to describe the semantic probability distribution of each voxel under multiple structural categories; A semantic region mask generation module is used to generate a semantic region mask based on the multi-class probability map, wherein the semantic region mask is used to characterize semantically reliable regions and semantically uncertain regions; The augmented volume data generation module is used to perform geometric consistency optimization on the three-dimensional HRCT volume data under the constraints of the semantic region mask to generate geometrically consistent augmented volume data; A 3D mesh model construction module is used to perform 3D surface reconstruction based on the geometric consistency enhancement body data to generate a 3D mesh model of the temporal bone. The temporal bone 3D model acquisition module is used to perform model optimization processing on the temporal bone 3D mesh model and output the temporal bone 3D model.

Images