A progressively expanded method and system for automatic localization of anatomical landmarks in oral and maxillofacial CBCT images.

Abstract

This invention discloses a progressively expanded automatic localization method and system for anatomical landmarks in oral and maxillofacial CBCT images. The method employs a two-stage localization framework, from coarse to fine: a Region of Interest (ROI) alignment network is used to determine the region of interest containing the target landmark, and a global feature extraction network is used for coarse localization; then, local image blocks are cropped centered on the coarse localization coordinates, and a local fine-tuning network fuses global contextual features to achieve precise localization. When it is necessary to expand the detection of new anatomical landmarks, a progressive expansion training strategy is adopted: the pre-trained network branches corresponding to historical landmarks are frozen to form a base branch set; training branches with the same structure are initialized for the new landmarks, and the contextual information of the base branches is transferred to the new branches through a cross-branch attention fusion module; the outputs of all branches are merged during inference. This method achieves high localization accuracy while realizing low-cost landmark detection capability expansion without retraining the entire model.

Description

Technical Field

[0001] This invention relates to medical image processing technology, and more specifically, to a method and system for automatic localization of anatomical landmarks in progressively expanding oral and maxillofacial CBCT images. Background Technology

[0002] Automatic detection of anatomical landmarks in the oral and maxillofacial region is a crucial foundation for orthodontic treatment planning and surgical navigation in maxillofacial surgery. In recent years, deep learning-based heatmap regression methods combined with cascaded coarse-fine detection frameworks have become the mainstream technical approach. For example, the three-stage framework proposed by Zhu et al. uses a multi-scale global information extraction module to fuse features from different levels of the encoder and combines it with a landmark attention mechanism to achieve global-local feature fusion, achieving some progress in CBCT landmark detection tasks.

[0003] Current technologies still have significant shortcomings. Multi-scale feature extraction relies on spatial resolution adjustment, requiring upsampling and pooling operations to unify the resolution of feature maps at different levels, leading to loss of spatial detail and affecting the accurate localization of anatomical landmark boundaries. The loss function design lacks targeted optimization for difficult examples; the use of fixed-weight mask heatmap loss fails to effectively mine difficult negative samples in regions with blurred boundaries, resulting in insufficient generalization ability of the model at complex anatomical structures. Global-local feature fusion mechanisms suffer from information redundancy; existing attention mechanisms fail to adequately filter semantic channel information related to specific landmarks, leading to invalid features interfering with the local refinement process. Furthermore, existing network architectures are designed based on a fixed number of classes, lacking scalability; when new anatomical landmarks need to be added, the network structure must be modified and all parameters retrained, which can cause shifts or even forgetting of the localization of learned landmarks.

[0004] In clinical applications, the quantitative evaluation of the efficacy of maxillary expansion treatment relies on the precise measurement of the spatial positional changes of specific anatomical landmarks before and after treatment. However, a standardized three-dimensional evaluation system is currently lacking, and evaluation mainly depends on manual measurements by doctors based on two-dimensional cephalometric radiographs or manual marking by three-dimensional software. This approach suffers from drawbacks such as strong subjectivity, poor repeatability, and high time and labor costs. Existing deep learning methods, due to the aforementioned technical limitations, cannot provide accurate detection of hard tissue boundary landmarks and quantitative analysis of displacement before and after treatment, making it difficult to meet the clinical need for objective evaluation of the efficacy of expansion treatment.

[0005] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0006] The purpose of this invention is to provide an automatic localization method and system for anatomical landmarks in oral and maxillofacial CBCT images with progressive expansion, which has the advantages of improving localization accuracy, optimizing the processing of difficult samples, reducing spatial detail loss, and supporting the progressive expansion of new landmarks.

[0007] The above-mentioned technical objective of the present invention is achieved through the following technical solution:

[0008] An automated method for locating anatomical landmarks in oral and maxillofacial CBCT images with progressive expansion includes the following steps:

[0009] S1. Acquire raw oral CBCT image data and perform voxel standardization processing to convert the coordinates of anatomical landmarks in the physical coordinate system into coordinate labels in the voxel coordinate system, generating three-dimensional voxel data with anatomical landmark coordinate labels.

[0010] S2. Input the three-dimensional voxel data into the ROI alignment network to predict the three-dimensional region of interest bounding box containing the target anatomical landmarks, and crop and resample the original image data based on the three-dimensional region of interest bounding box to obtain the first-scale image patch;

[0011] S3. Input the first-scale image block into the global feature extraction network, and perform feature extraction and fusion through its built-in multi-scale dilated convolution module and dual attention mechanism to generate a global feature map with both semantic and spatial information. Based on the global feature map, decode and generate a coarse localization heatmap, and calculate the initial voxel coordinates of each anatomical landmark based on the coarse localization heatmap.

[0012] S4. Using the initial voxel coordinates as the center, crop out the second-scale local image patch from the original image data; at the same time, perform channel dimensionality reduction on the global feature map to generate global context features;

[0013] S5. Input the second-scale local image patch into the local fine-tuning network to extract local image features. In the local fine-tuning network, the local image features are fused with the global context features to generate a refined heatmap. Finally, the precise voxel coordinates of each anatomical landmark are decoded.

[0014] Furthermore, this invention proposes an ROI alignment network consisting of two levels of 3D convolutional layers and a fully connected layer; wherein the fully connected layer is used to output 6-dimensional bounding box parameters.

[0015] During the training phase of the ROI-aligned network, training samples are constructed as follows:

[0016] For the minimum bounding 3D rectangle calculated based on the coordinates of the gold standard marker points, random expansion is applied in the horizontal and vertical directions respectively to obtain the expanded bounding box for image cropping.

[0017] The ROI alignment network is trained by minimizing the alignment loss value;

[0018] The alignment loss value is calculated as follows:

[0019] Calculate the sum of squared differences between the 6D bounding box parameters predicted by the ROI alignment network and the true bounding box parameters in each dimension. Then, take the square root of the sum of squared differences to obtain the bounding box bias at the sample level. Finally, calculate the average of the bounding box biases of all samples in a training batch.

[0020] Furthermore, this invention proposes a global feature extraction network with a four-level encoder-three-level decoder architecture, and introduces a multi-scale dilated convolution module in its bottleneck layer. The output of the multi-scale dilated convolution module is connected to a dual attention mechanism.

[0021] The multi-scale dilated convolution module includes three parallel dilated convolution branches and one convolution bypass branch. The multi-scale dilated convolution module concatenates the output features of the dilated convolution branches and the convolution bypass branch in the channel dimension, and then performs channel dimensionality reduction to generate multi-scale fused features.

[0022] Furthermore, this invention also proposes a dual attention mechanism including:

[0023] The channel attention branch receives the multi-scale fusion features output by the multi-scale dilated convolution module, performs adaptive average pooling on them, and then processes them through a fully connected layer and the Tanh activation function to generate channel weights.

[0024] The spatial attention branch receives the channel-weighted features obtained by multiplying the channel weights with the features output by the multi-scale dilated convolution module channel by channel. Max pooling and average pooling are then performed on the channel-weighted features. The pooling results are concatenated along the channel dimension and then passed through a convolutional layer to generate a spatial weight map.

[0025] The spatial weight map is multiplied element-wise with the channel weighted features to obtain spatial modulation features; the spatial modulation features are then fused with the multi-scale fusion features output by the multi-scale dilated convolution module to generate the global feature map.

[0026] Furthermore, this invention proposes a local fine-tuning network that employs a three-level encoder-decoder architecture, and the fine-tuning processes for each anatomical landmark are independent of each other;

[0027] In the local fine-tuning network, the local image features of the second-scale local image block are concatenated with the global context features in the channel dimension.

[0028] The three-level encoder-decoder increases the number of feature channels step by step through downsampling operations, and performs feature fusion through upsampling operations and skip connections with the corresponding encoder layers. Finally, it outputs a refined heatmap representing the probability distribution of each anatomical landmark in the local space.

[0029] Furthermore, this invention proposes a joint optimization of the global feature extraction network and the local fine-tuning network using a dynamic, phased training strategy;

[0030] Joint optimization is based on the hard example mining loss function, specifically including:

[0031] During the training phase, dynamic phased training is performed, including:

[0032] In the first stage, the parameters of the local fine-tuning network are fixed, and only the global feature extraction network is trained;

[0033] In the second stage, after the global feature extraction network converges, the local fine-tuning network is unfrozen, and the loss of the two stages is weighted and jointly trained with dynamically decaying weight coefficients.

[0034] The loss function for difficult example mining is based on Gaussian heatmap labels. The number of positive and negative samples K is determined according to the Gaussian radius R, where K=R³ / 8. The Smooth L1 loss for the positive sample region and the difficult example negative sample region is calculated respectively.

[0035] When it is necessary to expand the detection of new anatomical landmarks, perform progressive expansion training, including:

[0036] Freeze the global feature extraction network branches and local fine-tuning network branches that have been trained in the previous training phase and correspond to historical landmarks, and use them as the basic branch set;

[0037] Initialize the global feature extraction network training branch and the local fine-tuning network training branch, which are structurally isomorphic and correspond to the newly added marker points, and randomly initialize their parameters;

[0038] At the bottleneck layer of the global feature extraction network training branch, the cross-branch attention fusion module integrates the contextual information provided by the basic branch set into the features of the training branch. The cross-branch attention fusion module uses the bottleneck feature of the training branch as the query and the bottleneck features of one or more branches in the basic branch set as the key and value to calculate the attention weight to generate historical context enhancement features, and adds the enhancement features to the bottleneck features of the training branch.

[0039] The local fine-tuning network training branch independently handles the fine-tuning task of adding new markers;

[0040] During the inference phase, the outputs of the base branch set and the training branch are merged to obtain a complete localization result that includes historical and newly added landmarks.

[0041] Furthermore, this invention also proposes the above method for quantitative evaluation of the effectiveness of orthodontic arch expansion treatment, including the following steps:

[0042] Obtain oral CBCT images of the same patient at T0 before treatment and T1 after treatment;

[0043] Using a landmark detection network trained through progressive expansion, anatomical landmarks were detected in CBCT images of T0 and T1 stages respectively, and the corresponding landmark coordinate sets of T0 stage and T1 stage were obtained.

[0044] Based on the coordinate sets of T0 and T1 anatomical landmarks, the displacement vector of each anatomical landmark in three-dimensional space is calculated. The displacement vector includes the displacement distance and the displacement direction relative to the craniomaxillary reference plane.

[0045] An arch expansion effect assessment report is generated based on the displacement vector to quantitatively evaluate the lateral expansion of the maxilla, changes in arch width, and the range of action of the arch expander.

[0046] Furthermore, this invention also proposes an automatic localization system for CBCT landmarks in the oral and maxillofacial region that supports progressive expansion, comprising:

[0047] The data preprocessing module is used to acquire raw oral CBCT image data, perform voxel standardization processing, convert the coordinates of anatomical landmarks in the physical coordinate system into coordinate labels in the voxel coordinate system, and generate three-dimensional voxel data with anatomical landmark coordinate labels.

[0048] The ROI alignment network module is used to receive three-dimensional voxel data, predict the three-dimensional region of interest bounding box containing the target anatomical landmarks, and crop and resample the original image data based on the bounding box to obtain the first-scale image patch.

[0049] The global feature extraction and coarse localization module includes a global feature extraction network, which receives the first-scale image block, extracts and fuses features through its built-in multi-scale dilated convolution module and dual attention mechanism, generates a global feature map with both semantic and spatial information, and decodes and generates a coarse localization heatmap based on the global feature map, and calculates the initial voxel coordinates of each anatomical landmark based on the coarse localization heatmap.

[0050] The local region and feature extraction module is used to crop out a second-scale local image patch from the original image data with the initial voxel coordinates as the center, and to perform channel dimensionality reduction on the global feature map to generate global context features.

[0051] The local fine-tuning and coordinate output module includes a local fine-tuning network, which receives local image blocks at the second scale to extract local image features and fuses the local image features with global context features to generate a refined heatmap. Finally, it decodes to obtain the precise voxel coordinates of each anatomical landmark.

[0052] In summary, the present invention has the following beneficial effects:

[0053] By acquiring CBCT data, ROI alignment, global feature extraction, and local fine-tuning, accurate anatomical landmark localization is achieved, and progressive expansion of new landmarks is supported. This approach offers advantages such as improved localization accuracy, optimized handling of difficult cases, reduced spatial detail loss, and support for progressive expansion of new landmarks. Attached Figure Description

[0054] Figure 1 This is a flowchart of the automatic localization method for anatomical landmarks in oral and maxillofacial CBCT images according to the present invention.

[0055] Figure 2 This is a schematic diagram of the automatic localization method for anatomical landmarks in oral and maxillofacial CBCT images described in this invention.

[0056] Figure 3 This is a block diagram of the automatic localization system for anatomical landmarks in oral and maxillofacial CBCT images described in this invention.

[0057] Figure 4 This is a schematic diagram of the precise voxel coordinates of each anatomical landmark described in this invention. Detailed Implementation

[0058] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to the figures and specific embodiments.

[0059] like Figure 1 As shown, the present invention proposes a progressively expanding method for automatic localization of anatomical landmarks in oral and maxillofacial CBCT images, comprising the following steps:

[0060] S1. By acquiring raw oral CBCT image data and performing voxel standardization, three-dimensional voxel data with anatomical landmark coordinate labels are generated.

[0061] S2. Input the three-dimensional voxel data into the ROI alignment network to predict the three-dimensional region of interest bounding box containing the target anatomical landmarks, and crop and resample the original image data based on the bounding box to obtain the first-scale image patch.

[0062] S3. Input the first-scale image block into the global feature extraction network, and perform feature extraction and fusion through its built-in multi-scale dilated convolution module and dual attention mechanism to generate a global feature map with both semantic and spatial information. Based on the global feature map, decode and generate a coarse localization heatmap, and calculate the initial voxel coordinates of each anatomical landmark based on the coarse localization heatmap.

[0063] S4. Using the initial voxel coordinates as the center, crop out the second-scale local image patch from the original image data; at the same time, perform channel dimensionality reduction on the global feature map to generate global context features.

[0064] S5. Input the second-scale local image patch into the local fine-tuning network to extract local image features. In the local fine-tuning network, the local image features are fused with the global context features to generate a refined heatmap. The precise voxel coordinates of each anatomical landmark are then decoded.

[0065] For ease of understanding, the following explains some key terms in this embodiment:

[0066] Raw oral CBCT image data: refers to the raw three-dimensional volumetric data of the oral and maxillofacial region obtained by cone-beam computed tomography (CBCT) equipment, which contains anatomical information of the region.

[0067] Voxel standardization: This refers to the preprocessing of raw oral CBCT image data to unify data format, intensity range, or spatial resolution, ensuring data consistency and facilitating subsequent processing by network models.

[0068] Anatomical landmark coordinates: refers to the positional information of specific anatomical structures (such as tooth cusps, bony prominences, etc.) in the physical coordinate system or voxel coordinate system in CBCT images of the oral and maxillofacial region.

[0069] Three-dimensional voxel data: refers to oral and maxillofacial image data represented in the form of a three-dimensional mesh after voxel standardization. Each mesh unit (voxel) contains intensity value and spatial location information.

[0070] ROI Alignment Network: This refers to a deep learning network model whose function is to predict and locate the bounding box of a 3D region of interest containing anatomical landmarks of a target based on the input 3D voxel data.

[0071] 3D Region of Interest (ROI) bounding box: refers to a 3D rectangular region predicted by the ROI alignment network, which is designed to precisely enclose one or more target anatomical landmarks to narrow down the scope of subsequent processing.

[0072] First-scale image patch: refers to the image data patch obtained by cropping and resampling the original image data based on the three-dimensional region of interest bounding box. It usually has a low resolution or small size and is used for global feature extraction.

[0073] Global Feature Extraction Network: This refers to a deep learning network model used to extract features with global semantic and spatial information from a first-scale image patch, providing a basis for coarse localization.

[0074] Multi-scale dilated convolution module: refers to a component in the global feature extraction network that expands the receptive field without loss of resolution by using dilated convolution kernels with different dilation rates to capture multi-scale contextual information.

[0075] Dual attention mechanism: refers to a feature enhancement mechanism that combines channel attention and spatial attention to adaptively adjust the importance of different channels and spatial locations in the feature map to enhance key features.

[0076] Global feature map: refers to the feature representation extracted by a global feature extraction network and fused with semantic and spatial information, which contains the overall information of the image patch.

[0077] Coarse localization heatmap: refers to a probability distribution map generated based on global feature map decoding, which represents the initial position probability of each anatomical landmark within a large spatial range and is used to calculate the initial voxel coordinates.

[0078] Initial voxel coordinates: These refer to the preliminary positioning results of each anatomical landmark calculated based on the coarse positioning heatmap. Their accuracy is relatively low, but they provide the starting point for local refinement.

[0079] Second-scale local image patch: refers to a high-resolution local image data patch cropped from the original image data with the initial voxel coordinates as the center, which contains detailed local information around the marker point.

[0080] Global context features: These refer to the feature representations obtained after channel dimensionality reduction of the global feature map. They retain global semantic information and are provided to the local fine-tuning network in a more compact form.

[0081] Local fine-tuning network: refers to a deep learning network model that is used to extract local image features from second-scale local image patches and combine them with global context features to accurately correct the landmark points.

[0082] Local image features: refers to the feature representations reflecting local anatomical details extracted from local image blocks at the second scale by a local fine-tuning network.

[0083] Refined heatmap: refers to a high-resolution probability distribution map generated by a locally fine-tuned network, which represents the precise location probability of each anatomical landmark in local space.

[0084] Precise voxel coordinates: refer to the final positioning results of each anatomical landmark obtained from the decoding of the refined heat map, which has high positioning accuracy.

[0085] See Figure 2 This invention provides a method for automatically locating anatomical landmarks in oral and maxillofacial CBCT images with progressive expansion, the specific implementation of which is as follows:

[0086] The process involves acquiring raw oral CBCT image data and performing voxel standardization to convert the coordinates of anatomical landmarks in the physical coordinate system to coordinate labels in the voxel coordinate system, generating 3D voxel data with anatomical landmark coordinate labels. Raw oral CBCT image data can be acquired by directly connecting to the CBCT scanning device or by reading DICOM format image files from a storage medium. After acquiring the raw image data, voxel standardization is performed. This process may include normalizing the image grayscale values ​​to a specific range or performing linear scaling to unify image characteristics under different devices or scanning parameters. Simultaneously, the conversion of anatomical landmark coordinates can be achieved by using a preset transformation matrix to map physical coordinates to voxel coordinates, or by manually inputting the voxel positions of the landmarks to generate coordinate labels. The resulting 3D voxel data contains the image and its corresponding anatomical landmark coordinate labels.

[0087] Three-dimensional voxel data is input into a ROI alignment network to predict 3D region-of-interest (ROI) bounding boxes containing target anatomical landmarks. Based on these ROI bounding boxes, the original image data is cropped and resampled to obtain a first-scale image patch. Specifically, the ROI alignment network can be a regression model based on a convolutional neural network. It receives 3D voxel data as input and outputs bounding box parameters representing the 3D ROI, such as center coordinates and length, width, and height. This network predicts these parameters by learning from training samples with labeled bounding boxes. After obtaining the predicted ROI bounding boxes, the original image data is cropped according to these bounding boxes to remove irrelevant background areas. Subsequently, the cropped image is resampled, for example, through bilinear interpolation, to unify it to a preset size and resolution, resulting in a first-scale image patch.

[0088] The first-scale image patch is input into the global feature extraction network. Its built-in multi-scale dilated convolutional module and dual attention mechanism perform feature extraction and fusion to generate a global feature map containing both semantic and spatial information. Based on this global feature map, a coarse localization heatmap is generated, and the initial voxel coordinates of each anatomical landmark are calculated using the heatmap. Specifically, the global feature extraction network can be a neural network with an encoder-decoder architecture. The encoder part receives the first-scale image patch. The multi-scale dilated convolutional module can consist of multiple parallel dilated convolutional layers, each with a different dilation rate to capture contextual information at different scales, and their outputs are concatenated. The dual attention mechanism can be a cascaded channel attention module and a spatial attention module. The channel attention module generates channel weights through global average pooling and fully connected layers, while the spatial attention module generates a spatial weight map through convolutional layers and applies the weights to the feature map. Thus, after feature extraction and fusion, a global feature map is generated. Subsequently, the decoder part upsamples the global feature map and outputs a coarse localization heatmap, where the pixel values ​​represent the probability that the corresponding location is a landmark. Finally, the initial voxel coordinates of each anatomical landmark are determined by finding the peak points in the heatmap or by calculating the weighted average of the heatmap.

[0089] Using the initial voxel coordinates as the center, a second-scale local image patch is cropped from the original image data; simultaneously, the global feature map undergoes channel dimensionality reduction to generate global contextual features. Specifically, after obtaining the initial voxel coordinates of each anatomical landmark, a smaller cubic region is cropped from the original image data using these coordinates as the center, serving as the second-scale local image patch. This local image patch typically has the same resolution as the original image data to preserve fine anatomical details. Meanwhile, channel dimensionality reduction of the global feature map is achieved through a 1x1 convolutional layer, mapping the high-dimensional feature map to a lower-dimensional feature representation to generate global contextual features containing global semantic information.

[0090] The second-scale local image patch is input into the local fine-tuning network to extract local image features. In the local fine-tuning network, the local image features are fused with global context features to generate a refined heatmap. Finally, the precise voxel coordinates of each anatomical landmark are obtained by decoding.

[0091] The proposed automatic localization method for CBCT landmarks in the oral and maxillofacial region effectively avoids the spatial detail loss in traditional multi-scale feature extraction by employing a multi-scale dilated convolution module, thus improving the accuracy of anatomical landmark boundary localization. Simultaneously, the fusion mechanism of global and local features combines global semantics with local details, reducing information redundancy and enhancing feature effectiveness. Furthermore, this method supports progressive expansion, solving the problem that existing network architectures require retraining all parameters when adding new landmarks, ensuring the localization stability of learned landmarks, thereby achieving accurate and scalable automatic localization of CBCT landmarks in the oral and maxillofacial region.

[0092] In some of the embodiments of the present invention described above, an ROI alignment network is proposed to predict the bounding box of a three-dimensional region of interest containing anatomical landmarks of the target, and the original image data is cropped and resampled based on the bounding box. However, in its implementation, the uniformity of the training samples and the unreasonable design of the loss function may lead to inaccurate bounding box prediction, especially in the area of ​​blurred boundaries where there is a lack of generalization ability, thereby affecting the accuracy of subsequent cropping and the overall localization effect.

[0093] To address this, the present invention further proposes that the ROI alignment network consists of two levels of 3D convolutional layers and fully connected layers; wherein, the fully connected layers are used to output 6D bounding box parameters; during the training phase of the ROI alignment network, training samples are constructed in the following way: the minimum bounding 3D rectangle calculated based on the gold standard marker coordinates is randomly expanded in the horizontal and vertical directions respectively, and the expanded bounding box is used for image cropping; the ROI alignment network is trained by minimizing the alignment loss value; the alignment loss value is calculated by: calculating the sum of squared differences between the 6D bounding box parameters predicted by the ROI alignment network and the true bounding box parameters in each dimension, then taking the square root of the sum of squared differences to obtain the bounding box deviation at the sample level, and averaging the bounding box deviation of all samples in a training batch.

[0094] Specifically, the ROI alignment network aims to predict 3D region-of-interest (ROI) bounding boxes containing target anatomical landmarks from 3D voxel data, providing accurate local regions for subsequent feature extraction and localization. This network employs two levels of 3D convolutional layers, effectively extracting spatial features from 3D image data and capturing depth information and contextual relationships between voxels. For example, the first-level convolutional layer can use a smaller kernel (e.g., 3x3x3) to capture local details, while the second-level convolutional layer can use a larger kernel (e.g., 5x5x5) or more feature channels to obtain broader contextual information. Furthermore, batch normalization and activation functions (e.g., ReLU, Leaky ReLU) can be combined after the convolutional layers to enhance the network's learning ability and stability. The fully connected layers are responsible for mapping the high-level features extracted by the convolutional layers to specific output parameters. Their implementation can include, but is not limited to: using a single-layer or multi-layer fully connected structure, with each layer containing a different number of neurons, and adding non-linearity through activation functions (e.g., ReLU). The final output layer typically does not have an activation function or uses a linear activation function to directly output numerical values. The 6D bounding box parameters are typically used to define the spatial location and size of a 3D bounding box. For example, they can be represented as the 3D coordinates (x, y, z) of the bounding box center point and the dimensions (width, height, depth) of the bounding box in the three axes, or as the minimum and maximum coordinates of the bounding box.

[0095] During the training phase of the ROI alignment network, the quality and diversity of training samples directly affect the model's generalization ability. By appropriately transforming and enhancing the original data, various real-world scenarios can be simulated, thereby improving the model's robustness to unseen data. The gold standard marker coordinates refer to the locations of anatomical landmarks, considered accurate and real, obtained by professional doctors or experts through manual annotation or high-precision measurement; these coordinates serve as the true labels for model training. The minimum bounding box (MBC) refers to the smallest cubic region that can completely enclose all gold standard markers, providing an initial, compact definition of the region of interest. By applying random expansion to the MBC in the horizontal (e.g., x-axis and y-axis) and vertical (e.g., z-axis) directions, the diversity of training samples is increased, simulating the slight offsets of markers or uncertainties in image cropping that may occur in real-world applications. This expansion can be implemented in various ways, such as randomly increasing or decreasing the size by a certain proportion in each direction, or randomly translating a certain voxel distance outward or inward on each face of the bounding box. In addition to random expansion, other data augmentation techniques, such as random rotation, random scaling, brightness and contrast adjustment, or elastic deformation, can be combined to further enrich the training samples.

[0096] The ROI alignment network is trained by minimizing the alignment loss. The core objective of training the neural network is to minimize the difference (i.e., the loss value) between the network's predictions and the true labels through iterative optimization. This is typically achieved using backpropagation and gradient descent optimizers (such as Adam and SGD) to continuously adjust the network's weights and biases. The alignment loss is calculated to accurately measure the deviation between the predicted and true bounding boxes. Specifically, the difference between the predicted and true parameters in each dimension is calculated, and then the squares of these differences are summed and the square root is taken to obtain a sample-level Euclidean distance, which effectively penalizes larger prediction errors. Averaging the bounding box deviations across all samples in a training batch provides a stable gradient signal to guide the update of the network parameters.

[0097] Through the above technical solution, this invention effectively solves the problems of inaccurate bounding box prediction caused by the single training samples and unreasonable loss function design in existing ROI alignment networks. Specifically, the ROI alignment network is composed of two levels of 3D convolutional layers and fully connected layers, which can fully extract the deep spatial features of 3D images and directly output accurate 6D bounding box parameters through the fully connected layers, avoiding the loss caused by intermediate information transformation. During the training phase, by applying random horizontal and vertical expansion to the minimum bounding box obtained by calculating the coordinates of the gold standard marker points, the diversity of training samples is greatly enriched, simulating the anatomical variations and noise that may exist in actual clinical images, thereby significantly enhancing the model's generalization ability and robustness to complex anatomical structures and regions with blurred boundaries. In addition, by calculating the square root of the sum of squares of the differences between the predicted and true bounding box parameters and then averaging the results over the batch of samples, the prediction error can be quantified more accurately, and stronger penalties are imposed on larger errors, thereby stabilizing the training process and guiding the network parameters to optimize towards more accurate bounding box prediction. These improvements enable the ROI alignment network to output more accurate 3D region of interest bounding boxes, providing high-quality input image blocks for subsequent global feature extraction networks and local fine-tuning networks, thereby improving the overall accuracy and reliability of the automatic localization method for CBCT landmarks in the oral and maxillofacial region.

[0098] In some of the embodiments of the present invention described above, a global feature extraction network is proposed to extract features and generate coarse localization heatmaps. However, in its implementation, multi-scale feature extraction depends on spatial resolution adjustment, which leads to loss of spatial details and affects the accurate localization of anatomical landmark boundaries. Furthermore, the existing attention mechanism fails to adequately filter semantic channel information related to specific landmarks, resulting in invalid features interfering with local refinement.

[0099] To address this, the present invention further proposes that the global feature extraction network adopts a four-level encoder-three-level decoder architecture, and introduces a multi-scale dilated convolution module at its bottleneck layer. The output of the multi-scale dilated convolution module is connected to a dual attention mechanism. Specifically, the multi-scale dilated convolution module includes three parallel dilated convolution branches and one convolutional bypass branch; the multi-scale dilated convolution module concatenates the output features of the dilated convolution branches and the convolutional bypass branch in the channel dimension, and then performs channel dimensionality reduction to generate multi-scale fused features.

[0100] The four-level encoder-three-level decoder architecture employed in this global feature extraction network is a deep learning network structure. The encoder progressively extracts high-level semantic features and reduces spatial resolution, while the decoder restores spatial resolution and generates the target output through upsampling and feature fusion. The four-level encoder and three-level decoder refer to the number of downsampling and upsampling layers in the network, respectively. This design effectively captures both global contextual information and local detail information of the image. The encoder progressively compresses the input image blocks into low-dimensional feature representations through multiple convolutional and pooling operations, extracting high-level semantic information. The decoder, through deconvolution or upsampling operations, combines features from different encoder levels to progressively restore the spatial resolution of the feature maps, ultimately generating a refined output. For example, the encoder part can consist of a series of 3D convolutional layers, batch normalization layers, and activation functions (such as ReLU), with max pooling or stride convolution used for downsampling at the end of each level. Alternatively, the encoder and decoder can use residual blocks or densely connected blocks as basic building blocks to alleviate the vanishing gradient problem in deep networks and enhance feature propagation.

[0101] The multi-scale dilated convolutional module introduced in the bottleneck layer, where the bottleneck layer typically refers to the lowest-resolution, highest-semantic-information feature layer of the encoder output, is the most significant part of the network in terms of information compression. The multi-scale dilated convolutional module is a module that captures information from different receptive fields by using dilated convolutions with varying dilation rates. Its core advantage lies in expanding the receptive field without increasing the number of parameters or losing resolution, thereby acquiring multi-scale contextual information. Introducing this module into the bottleneck layer aims to further enhance the network's ability to capture global contextual information while avoiding the spatial information loss caused by traditional pooling operations, thus better understanding the overall structure and complex relationships of the image. For example, the bottleneck layer can be one or more convolutional layers with the smallest output feature map size. The multi-scale dilated convolutional module can consist of multiple parallel dilated convolutional layers, each using a different dilation rate (e.g., 1, 2, 4, 8, etc.) to capture contextual information at different scales. The output feature maps of these parallel branches are concatenated along the channel dimension.

[0102] The output of the multi-scale dilated convolutional module is connected to a dual attention mechanism. Dual attention typically refers to an attention mechanism that considers both channel and spatial dimensions simultaneously, aiming to enable the network to adaptively focus on more important channels and spatial regions in the feature map, thereby enhancing effective features and suppressing irrelevant features. This mechanism further optimizes the features extracted by the multi-scale dilated convolutional module, making them more discriminative. Through channel attention, the network can learn which feature channels are more important for the current task; through spatial attention, the network can learn which spatial locations in the feature map are more worthy of attention. This dual selection helps reduce information redundancy and improve feature quality. For example, the dual attention mechanism can include a channel attention module and a spatial attention module. The channel attention module can perform global average pooling and global max pooling on the input feature map, then feed the pooling results into a shared multilayer perceptron or convolutional layer, and then generate channel weights through a sigmoid activation function. The spatial attention module can perform average pooling and max pooling on the channel-weighted feature map, concatenate the results, and then generate a spatial weight map through a convolutional layer and a sigmoid activation function. Another approach is to combine channel attention and spatial attention in a series or parallel manner. For example, channel attention can be calculated first, its weights can be used to adjust the feature map, and then spatial attention can be calculated on the adjusted feature map.

[0103] The multi-scale dilated convolution module specifically includes three parallel dilated convolution branches and one convolutional bypass branch. The three parallel dilated convolution branches use different dilation rates to capture contextual information at different scales, while the convolutional bypass branch is typically a standard convolutional layer used to capture local details or as a residual connection to preserve the integrity of the original features. This design ensures that the module can acquire a broad receptive field and multi-scale information through dilated convolution, while preserving fine local features through the bypass branch, avoiding the "mesh effect" or information loss that may result from dilated convolution, thus extracting features more comprehensively. For example, the three dilated convolution branches can use 3x3 convolutional kernels with dilation rates of 2, 4, and 8, respectively, while the convolutional bypass branch can use a standard 1x1 or 3x3 convolutional kernel. The input to all branches is the feature map of the bottleneck layer.

[0104] The multi-scale dilated convolution module concatenates the output features of the dilated convolution branch and the convolutional bypass branch along the channel dimension, and then performs channel dimensionality reduction to generate multi-scale fused features. Channel dimension concatenation stacks multiple feature maps along the channel axis, increasing the number of channels in the feature map, thereby fusing information from different branches. Channel dimensionality reduction reduces the number of channels in the feature map through operations such as 1x1 convolution, thereby reducing computational complexity and the number of model parameters, while compressing redundant information and extracting more refined features. Multi-scale fused features refer to feature representations that include overall semantic information and multi-scale spatial information of the image. The concatenation operation effectively integrates feature information of different scales and types captured by the multi-scale dilated convolution branch and the bypass branch, forming a richer feature representation. Channel dimensionality reduction refines the features after fusion, removing redundant information and generating a compact and efficient multi-scale fused feature, providing high-quality input for subsequent coarse localization and local refinement. Channel dimensionality reduction can be achieved through a 1x1 convolutional layer, the number of output channels of which is set to the number of channels of the desired multi-scale fused feature.

[0105] Through the above technical solution, the global feature extraction network adopts a four-level encoder-three-level decoder architecture, introducing a multi-scale dilated convolution module at the bottleneck layer. This module captures contextual information at different scales through three parallel dilated convolution branches, while preserving original feature details through a convolutional bypass branch. This avoids the spatial detail loss caused by spatial resolution adjustments (such as upsampling and pooling) in traditional multi-scale feature extraction, thereby improving the accuracy of anatomical landmark boundary localization. Furthermore, the output of the multi-scale dilated convolution module is connected to a dual attention mechanism, which adaptively filters channel and spatial features related to specific anatomical landmarks, effectively reducing interference from invalid features. This results in a global feature map with richer semantic information and more accurate spatial information, providing a more reliable foundation for subsequent coarse localization heatmap generation, and significantly improving the accuracy and robustness of automatic localization of CBCT landmarks in the oral and maxillofacial region.

[0106] In some of the embodiments of the present invention described above, a dual attention mechanism is proposed to handle feature fusion. However, in its implementation, the existing attention mechanism fails to adequately filter semantic channel information related to specific landmarks, resulting in invalid features interfering with local refinement, thereby affecting the accuracy of anatomical landmark localization.

[0107] In response, this invention further proposes a dual attention mechanism including:

[0108] The channel attention branch is used to receive the multi-scale fusion features output by the multi-scale dilated convolution module, perform adaptive average pooling on them, and then process them through a fully connected layer and the Tanh activation function to generate channel weights.

[0109] The spatial attention branch is used to receive the channel-weighted features obtained by multiplying the channel weights with the features output by the multi-scale dilated convolution module channel by channel. Max pooling and average pooling are performed on the channel-weighted features respectively. The pooling results are concatenated in the channel dimension and then passed through a convolutional layer to generate a spatial weight map.

[0110] The spatial weight map is multiplied element-wise with the channel weighted features to obtain spatial modulation features; the spatial modulation features are then fused with the multi-scale fusion features output by the multi-scale dilated convolution module to generate the global feature map.

[0111] Specifically, dual attention is a feature enhancement technique used in deep learning models. It aims to improve the model's ability to perceive key information by explicitly modeling the interdependencies of feature maps in both channel and spatial dimensions. Its core idea is to assign different weights to different channels and spatial locations of the feature map, enabling the model to focus more on features relevant to the current task and suppress irrelevant noise.

[0112] The channel attention branch receives multi-scale fused features from the multi-scale dilated convolutional modules in the global feature extraction network. These features, obtained after processing by the multi-scale dilated convolutional modules, fuse information from different receptive fields and possess rich semantic and spatial context. This branch first performs adaptive average pooling on the received features. This operation aims to compress the spatial dimension of each channel of the input feature map into a single value, thereby capturing the global spatial information of each channel. For example, for a 3D feature map, its spatial dimensions (e.g., D x H x W) can be pooled to 1 x 1 x 1, resulting in a scalar representing the overall activation intensity of that channel. Besides adaptive average pooling, global max pooling can also be used to capture the most salient feature activations in each channel. Subsequently, the pooled features are transformed through one or more fully connected layers to learn the complex nonlinear relationships between channels, thereby predicting the importance of each channel. Following this, the Tanh activation function maps the output of the fully connected layers to a specific range (e.g., -1 to 1) as the final channel weights. Besides the Tanh activation function, the Sigmoid activation function can be used to map the weights to the range of 0 to 1, or the ReLU activation function can be used followed by normalization. These generated channel weights represent the importance of each feature channel to the current anatomical landmark localization task.

[0113] The spatial attention branch receives the channel-weighted features obtained by multiplying the channel weights by the features output from the multi-scale dilated convolution module channel by channel. This step, by performing channel-by-channel multiplication of the previously generated channel weights with the original feature map, achieves preliminary screening and weighting of feature channels, enhancing important semantic information. This branch performs max pooling and average pooling on the channel-weighted features. Max pooling captures the most significant feature responses at each spatial location in the feature map, highlighting texture or edge information in local regions; while average pooling captures the overall feature distribution at each spatial location, providing smoother contextual information. For 3D image data, these pooling operations are typically 3D, for example, using a 3x3x3 pooling kernel. In addition to max pooling and average pooling, other forms of pooling operations such as Lp norm pooling can also be considered. The different types of spatial information captured by max pooling and average pooling are concatenated along the channel dimension, providing a more comprehensive input for subsequent convolutional layers. Finally, a spatial weight map is generated through a convolutional layer. This convolutional layer learns how to extract spatial attention information from the concatenated features and generates a weight map with the same spatial size as the input feature map. For example, a 3x3x3 convolutional kernel with one output channel can be used to generate a single-channel spatial weight map. This convolutional layer is typically followed by a sigmoid activation function to normalize the weights to the range of 0 to 1, representing the importance of each spatial location.

[0114] Through the above technical solution, this invention effectively solves the redundancy problem that may exist in global feature extraction networks when fusing multi-scale information by introducing a dual attention mechanism, and improves the ability to filter semantic channels and spatial location information related to specific landmarks. Specifically, the channel attention branch dynamically learns and generates the weights of each channel by adaptively average pooling the features output by the multi-scale dilated convolution module and processing them through a fully connected layer and Tanh activation function, thereby strengthening the semantic information related to the dissected landmarks and effectively suppressing the interference of irrelevant channels. On this basis, the spatial attention branch receives the channel-weighted features, performs max pooling and average pooling respectively, and concatenates them before generating a spatial weight map through a convolutional layer, further focusing on the key spatial regions in the feature map, effectively filtering background noise and irrelevant local details. This mechanism of synergistic interaction between channel and spatial attention enables the global feature map to more accurately highlight features strongly correlated with the localization of anatomical landmarks while possessing both semantic and spatial information. This provides a cleaner and more discriminative input for the subsequent generation of coarse localization heatmaps, thereby significantly improving the accuracy and robustness of CBCT landmark localization in the oral and maxillofacial region, especially in areas with complex anatomical structures.

[0115] In some of the above-mentioned solutions of the present invention, a local fine-tuning network is proposed to accurately locate anatomical landmarks. However, in this process, due to the lack of a specific feature fusion mechanism and an independent fine-tuning process, information redundancy may occur when global and local features are fused, invalid features may interfere with the fine-tuning, and different landmarks may influence each other, thereby reducing the positioning accuracy.

[0116] In response, this invention further proposes that the local fine-tuning network adopts a three-level encoder-decoder architecture, and the fine-tuning process for each anatomical landmark is independent of each other;

[0117] In the local fine-tuning network, the local image features of the second-scale local image block are concatenated with the global context features in the channel dimension.

[0118] The three-level encoder-decoder increases the number of feature channels step by step through downsampling operations, and performs feature fusion through upsampling operations and skip connections with the corresponding encoder layers. Finally, it outputs a refined heatmap representing the probability distribution of each anatomical landmark in the local space.

[0119] Specifically, the local fine-tuning network employs an encoder-decoder architecture. The encoder is responsible for feature extraction and dimensionality reduction of the input image data, while the decoder restores the extracted features to the target output space. In this invention, the architecture is three-level, meaning it includes three downsampling and three upsampling layers, effectively capturing feature information at different scales, progressively refining feature representations from coarse to fine granular. For example, the encoder can consist of a series of 3D convolutional layers, batch normalization layers, and activation functions (such as ReLU), with downsampling achieved through max pooling or stride convolution. Another implementation approach is to use residual connections or dense connections between the encoder and decoder to enhance feature propagation and gradient flow, further improving network performance.

[0120] Meanwhile, in the local fine-tuning network of this invention, the processing paths or output branches for different anatomical landmarks are independent and do not interfere with each other during the fine-tuning process. This independence ensures that the fine-tuning process of each landmark can focus on its own local features, avoiding feature confusion and mutual influence between different landmarks. Specifically, the decoder part of the local fine-tuning network can set an independent output head for each target anatomical landmark, for example, an independent sequence of convolutional layers, with each output head responsible for generating the fine-tuned heatmap of the corresponding landmark. In addition, the independence of the fine-tuning process can also be indirectly achieved by designing independent feature processing branches within the network or by using a masking mechanism during training, so that the loss calculation of each landmark only focuses on its corresponding region.

[0121] Before inputting data into the local fine-tuning network, this invention concatenates second-scale local image patches with global contextual features along the channel dimension. Channel-dimensional concatenation refers to connecting two or more feature maps along the channel axis to form a new feature map with more channels. This operation aims to directly fuse the local detail information (from the second-scale local image patch) and global semantic information (from the global contextual features) required by the local fine-tuning network, providing a richer and more comprehensive input to the local fine-tuning network. For example, before the input layer of the local fine-tuning network, the appropriately sized second-scale local image patch and the global contextual features can be connected along the channel dimension via a programming interface. Another implementation is to adjust the channels of the global contextual features using a 1x1x1 convolutional layer to match the number of channels of the second-scale local image patch before concatenation, or to perform weighted fusion using an attention mechanism before concatenation.

[0122] This three-stage encoder-decoder architecture progressively increases the number of feature channels through downsampling operations in the encoder stage to capture more abstract, higher-level semantic information. In the decoder stage, spatial resolution is restored through upsampling operations, and feature fusion is performed using skip connections with the corresponding encoder layers. Skip connections directly pass feature maps of the same resolution from the encoder to the decoder, fusing them with the decoder's own upsampled features to compensate for spatial details that may be lost during upsampling. For example, downsampling can be achieved through max pooling layers or convolutional layers with a stride of 2. Upsampling can be achieved through deconvolutional layers (transposed convolutions) or bilinear / trilinear interpolation followed by convolutional layers. Skip connections typically concatenate the output of a layer in the encoder with the output (or upsampled result) of the corresponding layer in the decoder along the channel dimension or add them element-wise. Finally, the network outputs a refined heatmap representing the probability distribution of each anatomical landmark in local space. This heatmap visually represents the precise location probability of the landmark within the local region, facilitating subsequent coordinate decoding.

[0123] Through the above technical solutions, the local fine-tuning network of this invention can effectively solve the problems of feature fusion redundancy and insufficient fine-tuning accuracy in existing technologies. Specifically, a three-level encoder-decoder architecture is adopted. By increasing the number of feature channels step by step through downsampling operations, the detailed features of local images can be gradually extracted and deepened, avoiding the loss of shallow information, thereby capturing richer semantic information in local areas. At the same time, feature fusion is performed through upsampling operations and skip connections with corresponding encoder layers, combining feature maps from different levels, effectively preserving spatial details, reducing resolution loss that may occur during upsampling, and enabling the decoder to generate more accurate fine-tuning heatmaps. Furthermore, the fine-tuning process for each anatomical landmark is independent of each other, preventing cross-interference between features of different landmarks, ensuring that the fine-tuning of each landmark focuses on its local space, thereby significantly improving the localization accuracy of individual landmarks and the generalization ability of the model. In addition, the local fine-tuning network is input after concatenating the second-scale local image patch with global context features in the channel dimension, realizing the direct and efficient fusion of local details and global semantics. This fusion method can filter out effective features related to the current landmark, reduce interference from invalid information, and enable the local fine-tuning network to fully utilize global context information to guide the identification of local details during the refinement process. This ultimately generates a high-precision probability distribution heatmap, supporting accurate coordinate decoding. Combined with the global context features provided in step S4, the local fine-tuning network of this invention, through this refined architecture and fusion mechanism, can achieve more precise localization of anatomical landmarks on the basis of coarse localization, significantly improving the overall accuracy and robustness of automatic localization of CBCT landmarks in the oral and maxillofacial region.

[0124] In some of the solutions described above in this invention, a dynamic phased training strategy is proposed to optimize network training. However, during the training process, when it is necessary to expand the detection of new anatomical landmarks, the existing methods will cause the localization of historical landmarks to be shifted or forgotten. There is a lack of effective mechanisms to maintain the stability of learned knowledge, and at the same time, it is impossible to efficiently integrate historical context information to support the learning of new landmarks.

[0125] To address this, this invention further proposes a progressively expanded automatic localization method for anatomical landmarks in oral and maxillofacial CBCT images. The global feature extraction network and the local fine-tuning network are jointly optimized using a dynamic, phased training strategy. This dynamic, phased training strategy aims to improve the model's convergence speed, stability, and final performance by adjusting the training focus and parameter update strategy at different stages. Implementation methods include a freeze-thaw strategy, where some network layers are frozen in the early stages of training, while only other network layers are trained, and then all network layers are unfrozen for joint training after convergence or reaching a certain performance level; or a learning rate scheduling strategy, using different learning rates at different training stages; or a loss weight adjustment strategy, assigning dynamically changing weights to the loss function for different tasks or stages. This strategy aims to solve the problem of inconsistent convergence speeds or mutual interference between different network modules during joint training, thereby improving the overall training efficiency and stability.

[0126] The joint optimization is based on the hard example mining loss function. The hard example mining loss function is a loss function that identifies and focuses more on samples that the model struggles to predict correctly (i.e., "hard examples") during training. By imposing larger loss weights on these hard examples or training more frequently, the model can better learn these ambiguous and easily confused samples. This can be implemented through online hard example mining (OHEM), dynamically selecting the sample with the highest loss value for backpropagation in each training batch based on the sample's loss value; or by using focal loss, which introduces a modulation factor to reduce the weight of easily distinguishable samples, thus making the model focus more on hard examples. In the CBCT landmark localization task of the oral and maxillofacial region, some anatomical landmarks may be located in areas with ambiguous image boundaries, low contrast, or complex structures; these are "hard examples" where the model is prone to errors. The hard example mining loss function enables the model to learn the features of these areas more effectively, improving its generalization ability and localization accuracy in complex situations.

[0127] Specifically, during the training phase, dynamic phased training is performed, including: In the first phase, the parameters of the local fine-tuning network are fixed, and only the global feature extraction network is trained; in the second phase, after the global feature extraction network converges, the local fine-tuning network is unfrozen, and the losses from both phases are weighted and jointly trained using dynamically decaying weight coefficients. In the first phase, only the global feature extraction network is trained, the weight parameters of the local fine-tuning network remain unchanged, and it does not participate in gradient updates. The global feature extraction network is responsible for extracting global feature maps from the first-scale image patch and generating a coarse localization heatmap. In the second phase, when the global feature extraction network reaches a certain convergence state, the parameters of the local fine-tuning network are unfrozen, allowing it to participate in training as well. At this time, the losses of the two network modules are weighted and jointly trained using dynamically decaying weight coefficients. For example, initially, the global network can be given a higher weight, and as training progresses, the weight of the global network can be gradually reduced while the weight of the local fine-tuning network can be increased, or the weights can be dynamically adjusted according to their respective loss values. This phased training helps to avoid interference with the learning of the global feature extraction network in the early stages of training due to the uninitialized or unstable parameters of the local fine-tuning network. By first allowing the global network to learn stable coarse localization capabilities, and then introducing a local fine-tuning network for refinement, the overall stability and efficiency of training can be improved, and the two network modules can be ensured to work together.

[0128] The hard example mining loss function is based on Gaussian heatmap labels. The number of positive and negative samples K is determined according to the Gaussian radius R, where K = R³ / 8. Smooth L1 losses are calculated separately for the positive sample region and the hard example negative sample region. In marker localization tasks, the true location of each marker is typically represented as a Gaussian heatmap centered on that point. The peak value of the heatmap indicates the marker location, and its intensity decreases with distance from the center. The Gaussian radius R determines the diffusion range of the Gaussian heatmap and indirectly affects the size of the "positive sample region." The number of positive and negative samples K is calculated based on the Gaussian radius R. For example, K = R³ / 8 means that when calculating the loss, K points will be sampled from the positive sample region (near the center of the Gaussian heatmap) and the negative sample region (areas far from the marker but potentially misclassified). Smooth L1 loss is an improvement on L1 loss, approaching L2 loss when the error is small and approaching L1 loss when the error is large. It is less sensitive to outliers and is differentiable near zero, contributing to training stability. Hard example negative sample regions refer to those areas far from the ground truth markers but with high model predictions, or regions easily confused with ground truth markers. This loss function design allows for more precise guidance of model learning. Using Gaussian heatmap labels, the model can learn the probability distribution information around the markers; by determining the sampling number K based on the Gaussian radius R, the focus range for positive and negative samples can be dynamically adjusted; by calculating the Smooth L1 loss for both positive and hard example negative sample regions, the model can more effectively learn the precise location of the markers and suppress incorrect predictions of background regions, especially in hard example regions with blurred boundaries.

[0129] When it is necessary to expand the detection of new anatomical landmarks, progressive expansion training is performed. Progressive expansion training is an incremental learning strategy designed to add the ability to recognize new categories or tasks to an existing model without retraining the entire model. This can be achieved through parameter freezing and new branch addition: freezing the parameters of the old task-related data that have already been trained, adding a new network branch for the new task, and training only the new branch; or using knowledge distillation, using the old model as the "teacher" model and the new model as the "student" model, distilling the loss so that the new model retains knowledge of the old task while learning the new task. In CBCT landmark localization in the oral and maxillofacial region, clinical needs may constantly change, requiring the detection of new landmarks. Progressive expansion training avoids training from scratch for each new landmark, thus saving computational resources and time, and preventing "catastrophic forgetting" of already learned landmarks.

[0130] The progressive expansion training includes freezing the global feature extraction network branches and local fine-tuning network branches corresponding to historical landmarks that have been trained in the previous training phase, as a base branch set. This means that during progressive expansion training, the parameters of the corresponding parts (which can be understood as specific paths or output heads in the network) of the previously trained global feature extraction network and local fine-tuning network used to detect old landmarks are fixed and no longer updated. These frozen branches together constitute the "base branch set". By freezing the base branch set, the learned historical landmark detection capability can be effectively protected, preventing the localization accuracy of old landmarks from decreasing or shifting when learning new landmarks, thus avoiding the "catastrophic forgetting" problem.

[0131] Simultaneously, the global feature extraction network training branch and the local fine-tuning network training branch, which are structurally isomorphic to the newly added landmark, are initialized with randomly initialized parameters. For each newly added anatomical landmark, a set of global feature extraction network training branches and local fine-tuning network training branches with the same (isomorphic) network branch structure as the existing ones are created. The parameters of these new branches are randomly initialized, for example, using a Gaussian or uniform distribution. This approach ensures that the learning of new landmarks starts from a completely new and unbiased state, and that their network structure remains consistent with the historical landmark branches, facilitating subsequent feature fusion and unified processing.

[0132] In the bottleneck layer of the global feature extraction network training branch, a cross-branch attention fusion module fuses the contextual information provided by the base branch set into the features of the training branch. In deep learning networks, the bottleneck layer typically refers to a layer with low feature dimensionality and high information density, containing a highly abstract and compressed representation of the input data. The cross-branch attention fusion module is a specially designed module for information interaction and fusion between different network branches, utilizing an attention mechanism to selectively integrate information. Contextual information refers to the semantic and spatial features extracted from the base branch set (i.e., the historical landmark branch) about the entire image or related region. By introducing the cross-branch attention fusion module into the bottleneck layer of the training branch for new landmarks, the learning of new landmarks can utilize the rich contextual information already learned by historical landmarks. This helps new landmarks learn faster and more accurately, especially when there is an anatomical connection or spatial proximity between the new and old landmarks.

[0133] The cross-branch attention fusion module uses the bottleneck features of the training branch as the Query and the bottleneck features of one or more branches in the base branch set as the Key and Value. It calculates attention weights to generate historical context enhancement features and adds these enhancement features to the bottleneck features of the training branch. The Query comes from the bottleneck features of the newly added marker training branch, representing the information that the new marker needs to focus on. The Key and Value come from the bottleneck features of one or more historical marker branches in the base branch set. The Key is used to calculate the attention weights, and the Value is used for weighted summation to generate the fused features. The attention weights are calculated based on the similarity between the Query and the Key, reflecting the degree of attention the new marker pays to historical context information. The historical context enhancement features are obtained by weighted summation of the Value with the attention weights; they contain the historical context information most relevant to the new marker. The generated historical context enhancement features are added element-wise to the original training branch bottleneck features, thus incorporating historical context information into the features of the training branch. This attention-based fusion method allows the newly added marker training branch to adaptively extract useful context information from historical marker branches, rather than simply concatenating features. This helps avoid interference from invalid features and ensures that the fused information is highly relevant to the current task of learning new landmarks, thereby improving the localization accuracy and learning efficiency of new landmarks.

[0134] The local fine-tuning network training branches independently handle the refinement task of newly added landmarks. This means that for each newly added landmark, its corresponding local fine-tuning network training branch independently receives the second-scale local image patch and global context features, and independently performs feature extraction, fusion, and decoding of the refined heatmap, ultimately outputting the precise voxel coordinates of the newly added landmark. Independent processing ensures that the refinement process of newly added landmarks is not interfered with by the refinement tasks of other landmarks, allowing the local fine-tuning network of each landmark to focus on learning its own fine features and localization details, thereby ensuring the accuracy of the localization of newly added landmarks.

[0135] During the inference phase, the outputs of the base branch set and the training branches are merged to obtain a complete localization result including historical and newly added landmarks. After model training is complete, when performing actual landmark localization (inference), the outputs of all frozen base branch sets (for historical landmarks) and all trained new landmark training branches are integrated. These outputs are typically the precise voxel coordinates of each landmark. By merging the outputs of all branches, the system can provide complete localization results for all historical and new landmarks at once, achieving comprehensive and progressively expanding automatic localization of CBCT landmarks in the oral and maxillofacial region.

[0136] Through the above technical solution, this invention proposes a progressively expanding automatic localization method for anatomical landmarks in oral and maxillofacial CBCT images. This effectively solves the problem that existing methods may lead to historical landmark location shifts or forgetting when expanding to detect new anatomical landmarks, and improves the efficiency and accuracy of learning new landmarks. Specifically, by adopting a dynamic phased training strategy, the parameters of the local fine-tuning network are first fixed, and only the global feature extraction network is trained. After convergence, the local fine-tuning network is unfrozen and weighted jointly trained with dynamically decaying weight coefficients. This strategy avoids mutual interference between different network modules in the early stages of training, ensuring the stable learning of the global feature extraction network. Based on this, the local fine-tuning network is gradually optimized, thereby improving the overall training stability and convergence efficiency. Simultaneously, based on the hard example mining loss function, the number of positive and negative samples K (where K = R³ / 8) is determined by Gaussian heatmap labels and the Gaussian radius R, and the Smooth L1 loss is calculated for the positive sample region and the hard example negative sample region respectively. This allows the model to focus more on learning those anatomical landmark regions with blurred boundaries and difficult distinctions. This significantly enhances the model's generalization ability and localization accuracy at complex anatomical structures, especially when dealing with difficult clinical cases. Furthermore, when it's necessary to expand the detection of new anatomical landmarks, this invention introduces a progressive expansion training mechanism. By freezing the global feature extraction network branches and local fine-tuning network branches corresponding to historical landmarks, which were trained in the previous training phase, as the base branch set, the learned historical knowledge is effectively protected, preventing "catastrophic forgetting." Simultaneously, isomorphic network training branches corresponding to the newly added landmarks are initialized. At the bottleneck layer of the global feature extraction network training branch, a cross-branch attention fusion module is used. Using the bottleneck features of the training branch as the query and the bottleneck features of one or more branches in the base branch set as the key and value, attention weights are calculated to generate historical context-enhanced features, which are then added to the bottleneck features of the training branch. This attention-based fusion method allows the learning of new landmarks to adaptively utilize the rich contextual information provided by historical landmark branches, avoiding interference from invalid features, thereby significantly improving the learning efficiency and localization accuracy of new landmarks. Furthermore, the local fine-tuning network training branch independently handles the refinement task of newly added landmarks, ensuring that each new landmark receives focused and precise localization. Finally, by merging the outputs of all branches during the inference phase, a complete and accurate localization result, including both historical and new landmarks, is obtained, greatly improving the system's scalability and usability.

[0137] In some of the embodiments of the present invention described above, a progressively expanding method for automatically locating anatomical landmarks in CBCT images of the maxillofacial region is proposed for automatically detecting anatomical landmarks. However, in the process of its implementation, when these methods need to be applied to the quantitative evaluation of the effect of orthodontic arch expansion treatment, the existing technology lacks specific steps to efficiently calculate displacement vectors and generate standardized evaluation reports. This results in the evaluation process relying on manual operation, being highly subjective, and unable to accurately quantify the transverse expansion of the maxilla, changes in arch width, and the range of action of the expander.

[0138] To address this, the present invention further proposes a progressively expanded automatic localization method for anatomical landmarks in oral and maxillofacial CBCT images, used for quantitative evaluation of the effects of orthodontic arch expansion treatment. The method includes the following steps: acquiring oral CBCT images of the same patient before treatment (T0 phase) and after treatment (T1 phase); using a landmark detection network trained through progressive expansion, detecting anatomical landmarks in the T0 and T1 phase CBCT images respectively, obtaining corresponding T0 phase landmark coordinate sets and T1 phase landmark coordinate sets; calculating the displacement vector of each anatomical landmark in three-dimensional space based on the T0 and T1 phase landmark coordinate sets, wherein the displacement vector includes displacement distance and displacement direction relative to the craniomaxillary reference plane; and generating an arch expansion effect evaluation report based on the displacement vectors to quantitatively evaluate the lateral expansion of the maxilla, changes in arch width, and the effective range of the arch expander.

[0139] Specifically, the step of acquiring oral CBCT images of the same patient at the pre-treatment T0 and post-treatment T1 stages aims to provide time-series comparative data for subsequent quantitative analysis, ensuring the accuracy and individualization of the assessment. This step can be achieved in several ways, for example, by performing two independent scans of the patient before treatment (T0 stage) and after treatment (T1 stage) using medical imaging equipment (such as a CBCT scanner), and storing the acquired raw DICOM data in a PACS system or local storage device; or by retrieving existing, time-compliant patient CBCT image data from a hospital information system (HIS) or a picture archiving and communication system (PACS).

[0140] The core of the step in detecting anatomical landmarks in T0 and T1 CBCT images using a progressively expanded marker detection network lies in replacing traditional manual marker placement with an automated, high-precision deep learning model. This improves efficiency and reduces subjective errors. This step can be implemented by inputting the T0 and T1 CBCT image data into a pre-trained, progressively expanded CBCT marker auto-localization network for the oral and maxillofacial region. This network can output the precise coordinates of each anatomical landmark in a three-dimensional voxel coordinate system. Alternatively, the CBCT image data can be uploaded to a server by calling a deployed marker detection service interface. The server-side marker detection network processes the data and returns the detection results (i.e., the marker coordinate set) to the client.

[0141] The step of calculating the displacement vector of each anatomical landmark in three-dimensional space based on the coordinate sets of T0 and T1 phase landmarks, where the displacement vector includes the displacement distance and the displacement direction relative to the craniomaxillary reference plane, aims to quantify the spatial changes of anatomical landmarks before and after treatment, providing precise numerical basis for the arch expansion effect. This step can begin by registering the T0 and T1 phase CBCT images to eliminate the influence of patient position changes and ensure coordinate system consistency. Then, for each corresponding anatomical landmark, the displacement distance is obtained by calculating the Euclidean distance between its T1 and T0 phase coordinates, and the displacement direction is obtained through vector subtraction. This direction is then projected onto a predefined craniomaxillary reference plane to determine the relative direction. Alternatively, without image registration, the difference vector between the T1 and T0 phase landmark coordinates can be directly calculated as the displacement vector. The craniomaxillary reference plane can be constructed using preset anatomical landmarks (such as orbital points, ear points, etc.), and then the angle or projection between the displacement vector and this reference plane is calculated to determine the relative displacement direction.

[0142] In the step of generating an arch expansion effect assessment report based on the displacement vector to quantify the assessment of maxillary lateral expansion, arch width changes, and the expander's effective range, its role is to transform the calculated raw displacement data into assessment indicators that are easy for clinicians to understand and use. This step can be achieved by designing an assessment report template containing predefined assessment indicators, such as maxillary lateral expansion (calculated by the distance change between specific landmark pairs), arch width changes (calculated by the distance change between cusp or alveolar bone landmark pairs), and the expander's effective range (determined by the displacement distribution of landmarks in the affected area). The system automatically fills the calculated displacement vector data into the report template and generates visual charts. Alternatively, an interactive software interface can be developed, allowing doctors to select landmark pairs of interest, display their displacement distance and direction in real time, and automatically calculate and display preset arch expansion effect indicators, such as the percentage increase in maxillary width and changes in molar spacing, ultimately generating a printable or electronic assessment report.

[0143] Through the above technical solution, this invention applies a progressively expanding method for automatically locating anatomical landmarks in oral and maxillofacial CBCT images to the quantitative evaluation of orthodontic arch expansion treatment effects. This effectively solves the problems of traditional evaluation methods, such as high subjectivity, poor repeatability, time-consuming and labor-intensive processes, and the inability to accurately quantify arch expansion effects. By acquiring CBCT images of the same patient before and after treatment and automatically obtaining landmark coordinates using a high-precision landmark detection network, the efficiency and accuracy of data acquisition are greatly improved. Based on this, three-dimensional displacement vectors are accurately calculated, including displacement distance and displacement direction relative to the craniomaxillary reference plane, providing clinicians with objective and quantitative indicators of treatment effectiveness. Finally, a standardized arch expansion effect evaluation report is generated based on these displacement vectors, clearly presenting key information such as the transverse expansion of the maxilla, changes in arch width, and the range of action of the arch expander. This achieves a comprehensive, accurate, and automated evaluation of arch expansion efficacy, significantly improving the scientific rigor and convenience of clinical diagnosis and treatment planning.

[0144] See Figure 3This invention proposes an automatic localization system for CBCT landmarks in the oral and maxillofacial region that supports progressive expansion. The system achieves a progressive processing flow from coarse to fine through modular design, effectively solving the aforementioned technical problems. The data preprocessing module first acquires the raw oral CBCT image data, performs voxel standardization, and converts the coordinates of anatomical landmarks in the physical coordinate system into coordinate labels in the voxel coordinate system, generating three-dimensional voxel data with anatomical landmark coordinate labels. In specific implementation, the raw oral CBCT image data is directly acquired from the scanning device via a DICOM interface. Voxel standardization includes grayscale normalization and linear scaling to unify the image characteristics of different devices. Simultaneously, a preset transformation matrix maps the physical coordinates to voxel coordinates, ensuring the consistency of the input data and reducing localization errors caused by coordinate system differences.

[0145] Furthermore, the ROI alignment network module receives 3D voxel data, predicts 3D region-of-interest (ROI) bounding boxes containing target anatomical landmarks, and crops and resamples the original image data based on these bounding boxes to obtain a first-scale image patch. This module employs a 3D ResNet-18-based regression network, learning the center coordinates and length, width, and height parameters from training samples labeled with bounding boxes. After cropping, it resamples to a 64×64×64 voxel size using bilinear interpolation. By narrowing the search area, this module preserves the spatial details of key regions, avoiding spatial information loss caused by global processing, and providing high-fidelity input for subsequent coarse localization.

[0146] Building upon this foundation, the global feature extraction and coarse localization module includes a global feature extraction network that receives image patches at the first scale. It extracts and fuses features using its built-in multi-scale dilated convolutional module and a dual attention mechanism. The multi-scale dilated convolutional module consists of three parallel dilated convolutional layers with dilation rates set to 1, 3, and 6, capturing contextual information at different scales before feature concatenation. The dual attention mechanism sequentially performs channel attention and spatial attention operations. Channel attention generates channel weights through global average pooling and fully connected layers, while spatial attention generates a spatial weight map through convolutional layers, thereby filtering semantic channel information related to specific landmarks. Due to the synergistic effect of these mechanisms, a global feature map containing both semantic and spatial information is generated. Based on this global feature map, a coarse localization heatmap is decoded, and the initial voxel coordinates of each anatomical landmark are calculated based on the weighted average of the heatmap. This process effectively reduces information redundancy and improves the accuracy of coarse localization at complex anatomical structures.

[0147] The local region and feature extraction module uses the initial voxel coordinates as the center to crop a second-scale local image patch of 128×128×128 voxels from the original image data, preserving the local anatomical details at the original resolution. The global feature map is then subjected to channel dimensionality reduction through a 1×1 convolutional layer, compressing 512 channels to 64 channels to generate global contextual features. This dimensionality reduction operation preserves key semantic information while reducing computational complexity, providing a compact global semantic supplement for local refinement.

[0148] The local fine-tuning and coordinate output module includes a local fine-tuning network that receives second-scale local image blocks to extract local image features and fuses these features with global contextual features through channel concatenation to generate a refined heatmap. The local fine-tuning network employs a lightweight U-Net architecture, performing high-resolution decoding on the fused features and finally obtaining the precise voxel coordinates of each anatomical landmark through sub-pixel interpolation decoding. Considering the scalability requirements of the network architecture, the output layer of the local fine-tuning network uses a dynamic category adaptation mechanism. When a new anatomical landmark is added, only the number of output channels needs to be expanded and the parameters of the local fine-tuning network finely adjusted; there is no need to retrain the global feature extraction network, thus ensuring the stability of the learned landmark location. Through the above technical solutions, this invention avoids the spatial detail loss in traditional multi-scale feature extraction and improves the localization accuracy of blurred boundary areas. The dual attention mechanism and channel dimensionality reduction effectively suppress information redundancy and enhance feature discrimination ability. The modular design supports progressive expansion, providing a precise quantitative basis for landmark displacement for the evaluation of maxillary expansion efficacy, meeting the objective clinical need for three-dimensional evaluation of expansion effects.

[0149] See Figure 4 The system provides precise anatomical landmark localization for the quantitative evaluation of orthodontic arch expansion treatment effects. For example, it acquires CBCT images of the same patient before treatment (T0 phase) and after treatment (T1 phase). Using a landmark detection network trained through progressive expansion, it detects anatomical landmarks in the T0 and T1 phase CBCT images, obtaining corresponding T0 and T1 phase landmark coordinate sets. Based on these coordinate sets, the system calculates the displacement vector of each anatomical landmark in three-dimensional space, including displacement distance and displacement direction relative to the craniomaxillary reference plane. Finally, it generates an arch expansion effect evaluation report based on the displacement vectors to quantitatively assess the lateral expansion of the maxilla, changes in arch width, and the effective range of the expander. This provides a standardized and objective three-dimensional evaluation system for clinical practice, solving the current problem of lacking precise hard tissue boundary landmark detection capabilities and quantitative analysis methods.

[0150] In this document, the terms "upper," "lower," "front," "back," "left," "right," "top," "bottom," "inner," "outer," "vertical," and "horizontal," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only used for the clarity of expressing the technical solution and for the convenience of description, and therefore should not be construed as limiting the present invention.

[0151] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, which includes not only the elements listed but also other elements not expressly listed.

[0152] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for automatic localization of anatomical landmarks in oral and maxillofacial CBCT images with progressive expansion, characterized in that, Includes the following steps: S1. Acquire raw oral CBCT image data and perform voxel standardization processing to convert the coordinates of anatomical landmarks in the physical coordinate system into coordinate labels in the voxel coordinate system, generating three-dimensional voxel data with anatomical landmark coordinate labels. S2. Input the three-dimensional voxel data into the ROI alignment network to predict the three-dimensional region of interest bounding box containing the target anatomical landmarks, and crop and resample the original image data based on the three-dimensional region of interest bounding box to obtain the first-scale image patch; S3. Input the first-scale image block into the global feature extraction network, and perform feature extraction and fusion through its built-in multi-scale dilated convolution module and dual attention mechanism to generate a global feature map with both semantic and spatial information. Based on the global feature map, decode and generate a coarse localization heatmap, and calculate the initial voxel coordinates of each anatomical landmark based on the coarse localization heatmap. S4. Using the initial voxel coordinates as the center, crop out a second-scale local image block from the original image data; at the same time, perform channel dimensionality reduction on the global feature map to generate global context features; S5. Input the second-scale local image block into a local fine-tuning network to extract local image features. In the local fine-tuning network, the local image features are fused with the global context features to generate a refined heatmap. Finally, the precise voxel coordinates of each anatomical landmark are decoded to obtain the precise voxel coordinates of each anatomical landmark.

2. The method for automatic localization of anatomical landmarks in progressively expanding CBCT images of the oral and maxillofacial region according to claim 1, characterized in that, The ROI alignment network consists of two levels of 3D convolutional layers and fully connected layers; the fully connected layers are used to output 6D bounding box parameters. During the training phase of the ROI alignment network, training samples are constructed in the following manner: For the minimum bounding 3D rectangle calculated based on the coordinates of the gold standard marker points, random expansion is applied in the horizontal and vertical directions respectively to obtain the expanded bounding box for image cropping. The ROI alignment network is trained by minimizing the alignment loss value; The alignment loss value is calculated as follows: Calculate the sum of squared differences between the 6-dimensional bounding box parameters predicted by the ROI alignment network and the true bounding box parameters in each dimension. Then, take the square root of the sum of squared differences to obtain the bounding box bias at the sample level. Finally, calculate the average of the bounding box biases for all samples in a training batch.

3. The method for automatic localization of anatomical landmarks in oral and maxillofacial CBCT images with progressive expansion as described in claim 1, characterized in that, The global feature extraction network adopts a four-level encoder-three-level decoder architecture, and introduces a multi-scale dilated convolution module in its bottleneck layer. The output of the multi-scale dilated convolution module is connected to a dual attention mechanism. The multi-scale dilated convolution module includes three parallel dilated convolution branches and one convolution bypass branch. The multi-scale dilated convolution module concatenates the output features of the dilated convolution branches and the convolution bypass branch in the channel dimension, and then performs channel dimensionality reduction to generate multi-scale fused features.

4. The method for automatic localization of anatomical landmarks in oral and maxillofacial CBCT images with progressive expansion according to claim 3, characterized in that, The dual attention mechanism includes: The channel attention branch receives the multi-scale fusion features output by the multi-scale dilated convolution module, performs adaptive average pooling on them, and then processes them through a fully connected layer and the Tanh activation function to generate channel weights. The spatial attention branch receives the channel-weighted features obtained by multiplying the channel weights with the features output by the multi-scale dilated convolution module channel by channel, performs max pooling and average pooling on the channel-weighted features respectively, concatenates the pooling results along the channel dimension, and generates a spatial weight map through a convolutional layer. The spatial weight map is multiplied element-wise with the channel weighted features to obtain spatial modulation features; the spatial modulation features are then fused with the multi-scale fusion features output by the multi-scale dilated convolution module to generate the global feature map.

5. The method for automatic localization of anatomical landmarks in oral and maxillofacial CBCT images with progressive expansion according to claim 1, characterized in that, The local fine-tuning network adopts a three-level encoder-decoder architecture, and the fine-tuning process for each anatomical landmark is independent of each other; In the local fine-tuning network, the local image features of the second-scale local image block are concatenated with the global context features in the channel dimension. The three-level encoder-decoder increases the number of feature channels step by step through downsampling operations, and performs feature fusion through upsampling operations and skip connections with the corresponding encoder layers. Finally, it outputs a refined heatmap representing the probability distribution of each anatomical landmark in the local space.

6. The method for automatic localization of anatomical landmarks in progressively expanding CBCT images of the oral and maxillofacial region according to claim 1, characterized in that, The global feature extraction network and the local fine-tuning network are jointly optimized using a dynamic phased training strategy. The joint optimization is based on a hard example mining loss function, specifically including: During the training phase, dynamic phased training is performed, including: In the first stage, the parameters of the local fine-tuning network are fixed, and only the global feature extraction network is trained; In the second stage, after the global feature extraction network converges, the local fine-tuning network is unfrozen, and the loss of the two stages is weighted and jointly trained with dynamically decaying weight coefficients. The loss function for difficult example mining is based on Gaussian heatmap labels. The number of positive and negative samples K is determined according to the Gaussian radius R, where K = R³ / 8. The Smooth L1 loss for the positive sample region and the difficult example negative sample region is calculated respectively. When it is necessary to expand the detection of new anatomical landmarks, perform progressive expansion training, including: Freeze the global feature extraction network branches and local fine-tuning network branches that have been trained in the previous training phase and correspond to historical landmarks, and use them as the basic branch set; Initialize the global feature extraction network training branch and the local fine-tuning network training branch, which are structurally isomorphic and correspond to the newly added marker points, and randomly initialize their parameters; In the bottleneck layer of the training branch of the global feature extraction network, the context information provided by the basic branch set is fused into the features of the training branch through the cross-branch attention fusion module; the cross-branch attention fusion module uses the bottleneck feature of the training branch as the query, and the bottleneck features of one or more branches in the basic branch set as the key and value, calculates the attention weight to generate historical context enhancement features, and adds the enhancement features to the bottleneck features of the training branch. The local fine-tuning network training branch independently handles the fine-tuning task of adding new marker points; During the inference phase, the outputs of the base branch set and the training branch are merged to obtain a complete localization result that includes historical and newly added landmarks.

7. The method for automatic localization of anatomical landmarks in oral and maxillofacial CBCT images with progressive expansion according to any one of claims 1-6, characterized in that, The method described is used for the quantitative evaluation of the effectiveness of orthodontic arch expansion treatment, and includes the following steps: Obtain oral CBCT images of the same patient at T0 before treatment and T1 after treatment; Using the landmark detection network trained through progressive expansion, anatomical landmarks are detected in the CBCT images of T0 and T1 stages respectively, and the corresponding T0 stage landmark coordinate sets and T1 stage landmark coordinate sets are obtained. Based on the coordinate set of the T0 stage markers and the coordinate set of the T1 stage markers, the displacement vector of each anatomical marker in three-dimensional space is calculated. The displacement vector includes the displacement distance and the displacement direction relative to the craniomaxillary reference plane. An arch expansion effect evaluation report is generated based on the displacement vector to quantitatively evaluate the lateral expansion of the maxilla, changes in arch width, and the range of action of the arch expander.

8. An automatic localization system for anatomical landmarks in oral and maxillofacial CBCT images, characterized in that, include: The data preprocessing module is used to acquire raw oral CBCT image data, perform voxel standardization processing, convert the coordinates of anatomical landmarks in the physical coordinate system into coordinate labels in the voxel coordinate system, and generate three-dimensional voxel data with anatomical landmark coordinate labels. The ROI alignment network module is used to receive the three-dimensional voxel data, predict the three-dimensional region of interest bounding box containing the target anatomical landmarks, and crop and resample the original image data based on the bounding box to obtain a first-scale image patch. The global feature extraction and coarse localization module includes a global feature extraction network, which receives the first-scale image block, extracts and fuses features through its built-in multi-scale dilated convolution module and dual attention mechanism, generates a global feature map with both semantic and spatial information, and decodes and generates a coarse localization heatmap based on the global feature map, and calculates the initial voxel coordinates of each anatomical landmark based on the coarse localization heatmap. The local region and feature extraction module is used to crop out a second-scale local image block from the original image data with the initial voxel coordinates as the center, and to perform channel dimensionality reduction on the global feature map to generate global context features. The local fine-tuning and coordinate output module includes a local fine-tuning network, which receives the second-scale local image block to extract local image features, and fuses the local image features with the global context features to generate a refined heatmap, and finally decodes to obtain the precise voxel coordinates of each anatomical landmark.

Images