A semi-supervised cephalometric key point detection method based on structural feature perception

Abstract

This paper discloses a semi-supervised cephalometric keypoint detection method based on structural feature perception. The method first preprocesses data and constructs a dataset. Next, a two-stage detection model is constructed and a loss function is designed. The backbone network of the two-stage semi-supervised model framework consists of two phases: the first phase uses global context information to detect the rough positions of keypoints through a heatmap regression method; the second phase performs refined predictions by capturing the global-local correlations between keypoints; and finally, the prediction results are obtained through student network pre-training and semi-supervised teacher-student model training. This paper adopts a two-stage detection method from coarse to fine to predict cephalometric keypoints and introduces an iterative network design of the semi-supervised teacher-student model architecture. In each round, the teacher network guides the student network, and the network parameters are iterated cyclically to obtain the final prediction results. Experimental results on an internal dataset demonstrate the effectiveness of this method.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology and relates to a semi-supervised cephalometric key point detection method based on structural feature perception. It mainly involves a method of using a semi-supervised method to produce pseudo labels to improve the quality of training data to assist in improving the detection accuracy of a backbone network, and a method of using a graph convolutional network to capture the structural features of anatomical points and enhance the reliability of pseudo labels. Background Art

[0002] Cephalometry is a technique used to study the relationship between dental and skeletal features of the human skull by analyzing bone and soft tissue landmarks in head X-rays. This analysis is commonly used by dentists, orthodontists, and oral and maxillofacial surgeons in treatment planning. Traditionally, cephalometric analysis involves manually delineating landmarks on X-rays and calculating the associated linear and angular measurements using a goniometer. However, this manual analysis method is cumbersome and prone to random and systematic errors, including errors in X-ray acquisition, technical measurement, and landmark identification.

[0003] To improve work efficiency, digital cephalometric analysis is gradually replacing traditional manual methods. Digital cephalometric analysis utilizes computer software and digital image processing to automatically identify and measure landmarks. This technology not only increases analysis speed and accuracy but also allows for multiple analyses to be performed simultaneously, facilitating storage and comparison of results. Furthermore, digital radiographs reduce radiation dose to patients and minimize environmental hazards associated with chemicals.

[0004] Although digital technology has improved the accuracy and efficiency of landmark recognition, medical image processing requires extremely high precision. Therefore, preprocessing such as image enhancement and denoising is often required before model training. In recent years, with the development of artificial intelligence technology, automated landmark recognition technology has gradually become a research hotspot. Especially in the context of the increasing demand for craniofacial surgery worldwide, automated systems can help doctors more efficiently locate landmarks and improve treatment outcomes.

[0005] Medical imaging and deep learning technologies have facilitated the transition from 2D to 3D cephalometrics. 3D cephalometrics offers significant advantages over 2D methods in terms of comprehensive spatial information, measurement accuracy, personalized treatment, and visual communication, and has gradually become the mainstream choice for modern medical image analysis.

[0006] Currently, prominent deep learning-based 3D cephalometric detection technologies include convolutional neural networks, such as R-CNN and Unet. The Unet architecture has shown outstanding performance in medical applications. However, significant challenges remain in the field of cephalometric keypoint detection, primarily in the following three areas:

[0007] 1) The labeled data available for training is scarce;

[0008] Detecting 3D skull keypoints requires a large amount of high-quality annotated data. Labeling keypoints in 3D skull images is a highly specialized task, typically requiring experts with extensive medical backgrounds. Due to the complexity and time-consuming nature of the labeling process, annotated data is extremely scarce, making it difficult to train efficient deep learning models.

[0009] 2) Traditional models lose the global information of skull CT;

[0010] Traditional neural networks (such as CNN) mainly rely on local convolution operations and focus on extracting local features. They are unable to effectively capture global anatomical structural relationships, especially when dealing with complex craniofacial bones and soft tissue landmarks, and are prone to ignoring long-distance dependencies.

[0011] 3) The traditional model does not take into account the structural nature of medical anatomical points;

[0012] Traditional neural networks ignore the inherent spatial relationships and geometric structures between anatomical points. Medical anatomical points have stable relative positions and specific geometric arrangements in anatomy. Traditional models fail to effectively utilize this structural information, resulting in insufficient accuracy and robustness in landmark detection in complex medical images. Summary of the Invention

[0013] The present invention proposes a semi-supervised cephalometric key point detection method based on structural feature perception, which provides an effective solution to the above problems.

[0014] 1. To address the problem that traditional CNNs cannot capture global information, this paper adopts the SwinUnetr model with an attention mechanism. SwinUnetr combines the Transformer's multi-head self-attention mechanism to effectively capture the global features of the image. Specifically, SwinUnetr divides the image into non-overlapping patches and applies the self-attention mechanism to these patches, thereby capturing the long-range dependencies between key points, solving the problem of traditional models losing global information when processing complex craniofacial structures.

[0015] 2. Traditional models do not consider the structural information between anatomical points. Therefore, this paper introduces a graph attention network (GAT) to perceive and utilize these structural features. GAT constructs a graph structure between anatomical points and uses an attention mechanism to calculate the dependencies between each anatomical point and its neighbors, thereby capturing the geometric and spatial characteristics of the anatomical points.

[0016] 3. To address the scarcity of labeled data, this paper proposes a teacher-student model based on semi-supervised learning to generate pseudo-labels. First, a teacher model is trained using labeled data. The teacher model then generates pseudo-labels based on unlabeled data. These pseudo-labels are used to guide the training of the student model, effectively utilizing unlabeled data, expanding the training set, and improving model generalization.

[0017] The technical solution of the present invention includes the following steps:

[0018] Step (1), data preprocessing and construction of labeled datasets and unlabeled datasets;

[0019] Step (2), two-stage semi-supervised model construction and loss function design;

[0020] The backbone network of the two-stage semi-supervised model framework consists of two stages: the first stage uses global context information to detect the rough location of key points through the heat map regression method; the second stage makes refined predictions by capturing the global-local correlation between key points.

[0021] Step (3), obtain the prediction results through student network pre-training and semi-supervised teacher-student model training;

[0022] The two-stage semi-supervised model is divided into two main components: a teacher network and a student network. The teacher network is trained using parameters learned from existing labeled data and generates pseudo-labels on unlabeled data. The quality of these pseudo-labels is improved by combining a graph attention model. The student network is first pre-trained to obtain initial parameters. The student network is then iteratively trained based on the pseudo-labels generated by the teacher network to predict the final results.

[0023] Furthermore, the step (1) is specifically implemented as follows:

[0024] First, all 3D CCT voxel data in the internal dataset is standardized to a uniform resolution and pixel spacing. Data augmentation, such as rotation and elastic deformation, is then applied. The data is then divided into labeled and unlabeled components based on the presence or absence of labels. Furthermore, key point information for the labeled data is stored in corresponding JSON files. This method addresses the clinical needs of cephalometric analysis by covering n key points, such as OR, OL, S, N, and ANS. The internal dataset consists of annotation files corresponding to 3D CCT and some CT scans. Clinical CBCT data obtained from hospitals is divided into labeled and unlabeled groups. The labeled group has key points annotated by physicians to generate annotation files.

[0025] Furthermore, the step (2) is specifically implemented as follows:

[0026] The backbone network of the two-stage semi-supervised model framework consists of two stages: the first stage uses global context information to detect the rough location of key points through the heat map regression method; the second stage makes refined predictions by capturing the global-local correlation between key points.

[0027] Phase 1: Heatmap regression prediction network;

[0028] The heat map regression prediction network structure is divided into two parts: encoder and decoder. The encoder uses Swin Transformer to extract features step by step, and the decoder predicts the heat map by combining multi-scale features step by step through skip connections and residual blocks.

[0029] The encoder of the heat map regression prediction network:

[0030] The input accepted by the encoder is a 3D voxel image. Considering the computing power limitation, the original volume image is uniformly downsampled to the size of H×W×D with S channels as the original input of the network. First, the input image is divided into multiple non-overlapping 3D patches to form a 3D token sequence with a dimension of It is then projected into a feature space with a feature dimension of C. The size of each patch is H'×W'×D'. The encoder structure has a total of 4 levels, and each level includes 2 Transformer blocks. When calculating self-attention, the model divides the input image into non-overlapping windows to improve the interaction efficiency between patches. Specifically, in the first Transfomer block of each level, a window size of M×M×M is used to evenly divide the 3D small blocks into multiple regions. In the second Transfomer block, these windows are shifted by M / 2 voxel units along each dimension to enhance the ability to capture information. This "moving window" mechanism helps to obtain richer feature information at different levels. In order to effectively calculate the shift window mechanism, 3D cyclic shift is used and self-attention is calculated according to the following formula:

[0031]

[0032] Where Q, K, and V represent query, key, and value respectively; d represents the size of the query and key.

[0033] In the first stage of the encoder, a linear embedding layer is used to generate 3D tokens. To maintain the hierarchical structure of the U-shaped encoder, a patch merging layer is used to reduce the resolution by a factor of 2 at the end of each level. In addition, the patch merging layer groups and concatenates patches of resolution 2×2×2, resulting in a 4C-dimensional feature embedding. The final feature size is reduced to 2C by a linear layer. The resolutions of the second, third, and fourth levels are Use the same network design as the first level.

[0034] The heat map regression prediction network decoder:

[0035] The heatmap regression prediction network adopts a U-shaped network architecture design, in which the image features of different resolutions extracted by the encoder are used as the input of the corresponding level of the decoder through skip connections. The decoder structure has 5 levels (t∈0,1,2,3,4), the 0th level receives the original input as input, and the 1st to 4th levels receive the output features of the corresponding 4-level structure of the encoder as input through skip connections. The input of the tth level of the decoder is resized And is fed into a residual block consisting of two 3×3×3 convolutional layers, which are normalized by instance normalization layers. Subsequently, the deconvolution operation is used to increase the resolution of the feature map processed by the residual block by a factor of 2, and the output is spliced ​​with the output of the t-1th level of the decoder in the channel number dimension, where the features obtained by processing the output of the 4th level of the decoder and the 4th level of the encoder through the bottleneck layer are spliced. Then, using the same residual operation, the spliced ​​features are fed into another residual block to obtain the output of the tth level of the decoder. The output of the 0th level of the decoder is processed using a 1×1×1 convolutional layer and a sigmoid activation function to obtain the final heat map output. Each predicted heat map represents the probability distribution of the key point, indicating the rough position of the key point. By i The integration operation is used to calculate the spatial coordinates x of the i-th rough key point i ∈R 3 , the formula is as follows:

[0036]

[0037] Among them H i (ρ) is the voxel ρ∈R 3 The i-th heat map H at i The value of , and γ i is the spatial normalization term.

[0038] The second stage: structure-aware network of graph attention;

[0039] In the first stage, the input image is downsampled for model size and inference speed reasons, resulting in a loss of local details and low prediction accuracy. To overcome this limitation, 3D patches are cropped from the original, unsampled volumetric image near keypoints as input to the structure-aware graph attention network, preserving the geometric details used for fine-scale detection. In the second stage of fine-grained prediction, considering the consistency of spatial relationships between keypoints in the cephalometric task, this method constructs a structure-aware graph attention network using a graph attention module and an image decoding module to extract structure-aware feature vectors and embed them into global structural features to ensure prediction accuracy.

[0040] The graph attention module:

[0041] The global structure of this module is represented by the graph G = {D, V}: the predicted key points are the node sets Where n is the number of key points, x i ∈R 3 is the rough spatial coordinate value predicted at the i-th rough key point in the first stage, i.e., the spatial feature, l i ∈R d The feature vector representing the feature map extracted at the i-th coarse key point at level 0 in the first stage is the visual feature. i That is, the spatial visual feature is composed of the spatial feature x i and visual features i Edge set D = {w(v i ,v j )|i=1,2,...,n;j=1,2,...,n} represents node v i and v j The spatial relationship between them, where w(v i ,v j ) is a learnable parameter representation v i and v j The edge weights of γ From the spatial visual features v i Get the structure-aware embedding of each node in

[0042]

[0043] Among them U i is the normalization term, g is the linear transformation function. The attention function w is v i and v j The dot product similarity between:

[0044] w(v i ,v j )=θ(vi ) T φ(v j ),

[0045] where θ and φ represent the key and query functions respectively.

[0046] The image decoding module:

[0047] The original size is the spatial coordinate x of the i-th coarse keypoint i The 3D patch obtained by cutting around is represented as Through the image decoding module F β Learn visual feature embedding

[0048]

[0049] Eventually and Feature fusion passes through the linear layer to obtain the offset x offset :

[0050]

[0051] The further loss function design is as follows:

[0052] Loss function in the first stage heatmap regression prediction network for:

[0053]

[0054] where g i is the real coordinate of the key point. i (ρ) is the i-th heat map H i At voxel ρ∈E 3 The value of σ h is the standard deviation.

[0055] Loss function in structure-aware networks in the second stage of graph attention for:

[0056]

[0057] Therefore, the total loss function for:

[0058]

[0059] Among them, λ represents the weight coefficient between the two loss functions.

[0060] Furthermore, the step (3) is specifically implemented as follows:

[0061] The student network pre-training stage uses labeled data. During the training process, the Adam optimizer, setting learning rate, learning rate parameters, cosine annealing scheduler and other optimization methods are used to obtain the initial student network and then conduct semi-supervised architecture training.

[0062] The teacher-student network architecture is designed as follows:

[0063] The teacher network uses pre-trained student model parameters, and uses unlabeled data in the training phase to participate in the training to generate pseudo labels for the student model. No pre-training parameters are introduced in the student network training, and labeled data and unlabeled data are trained together. Unlabeled image data is used to guide the student model to predict more accurate results. Specifically, the teacher network does not set gradient updates, but is updated through exponential averaging (EMA), so that the prediction results of the teacher network will be more stable. The student network updates its parameters after each training epoch to better imitate the teacher network. The quality of the pseudo labels of the teacher network is often uneven, so this method uses the structure-aware network results of the second-stage graph attention to screen valid data to guide the student network. During the initial training, the graph attention module's perception of the spatial structure of key points is often a small range of changes. If a large offset is generated, it will lead to incorrect adjustments of other key points. Therefore, this method proposes the following pseudo label generation strategy:

[0064]

[0065] where X α The spatial coordinate value x predicted in the first stage i , X β The offset x predicted for the second stage offset , the threshold Δ is set to 2 mm error, which is clinically acceptable accuracy, X γ This is the final reliable pseudo label.

[0066] Finally, the prediction result obtained by the student network through iterative learning of the teacher network is the final prediction.

[0067] The beneficial effects of the present invention are as follows:

[0068] This paper proposes a new cephalometric detection method and designs a network model of semi-supervised cephalometric key points based on structural feature perception.

[0069] This paper uses a two-stage detection method, from coarse to fine, to predict cephalometric key points. It also incorporates an iterative network design based on a semi-supervised teacher-student model architecture. In each round, the teacher network guides the student network, and the network parameters are iterated to obtain the final prediction result. Experimental results on internal datasets demonstrate the effectiveness of this method. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 1 is a schematic diagram of a specific process of an embodiment of the method of the present invention;

[0071] Figure 2 2. It is a schematic diagram of the structure of a two-stage network model according to an embodiment of the method of the present invention;

[0072] Figure 3 2 is a schematic diagram of the structure of the graph attention module according to the method embodiment of the present invention;

[0073] Figure 4 2 is a schematic diagram of the structure of a semi-supervised teacher-student network model according to an embodiment of the method of the present invention. DETAILED DESCRIPTION

[0074] The method of the present invention and its detailed parameters are further described in detail below with reference to the accompanying drawings and examples.

[0075] like Figure 1 As shown in Figure 1, a semi-supervised cephalometric key point detection method based on structural feature perception is proposed. The specific steps are as follows:

[0076] Step (1), data preprocessing, constructing labeled datasets and unlabeled datasets;

[0077] An internal dataset consisting of high-quality 3D whole-skull CBCT scans of 138 patients was used, including 38 labeled CT data and 100 unlabeled raw CT scans. All 3D CBCT voxel data were first normalized to a uniform resolution (512*512*512) and pixel spacing (1.0*1.0*1.0). Random rotations were then applied in the x, y, and z directions, a random deformable grid was generated, and small-scale coordinate displacements were applied. The volumetric geometry was adjusted for data augmentation, and the data was divided into labeled and unlabeled parts based on the presence or absence of labels. The key point information for the labeled data was stored in a corresponding JSON file. This method covers n commonly used key points, such as OR, OL, S, N, and ANS, to meet the clinical needs of cephalometric analysis. The internal dataset consists of the annotation files corresponding to the 3D CBCT and some CT scans. The clinical CBCT data obtained from the hospital were divided into labeled and unlabeled groups. The labeled group had key points annotated by doctors to generate the annotation file; the number of key points, n, was 24.

[0078] Step (2), two-stage semi-supervised model construction and loss function design;

[0079] like Figure 2As shown in the figure, the backbone network of the two-stage semi-supervised model framework consists of two stages: the first stage uses global context information to detect the rough position of key points through the heat map regression method; the second stage makes refined predictions by capturing the global-local correlation between key points.

[0080] Phase 1: Heatmap regression prediction network

[0081] The heat map regression prediction network structure is divided into two parts: encoder and decoder. The encoder uses Swin Transformer to extract features step by step, and the decoder predicts the heat map by combining multi-scale features step by step through skip connections and residual blocks.

[0082] The encoder of the heat map regression prediction network:

[0083] The input accepted by the encoder is a 3D voxel image. Considering the computing power limitation, the original volume image is uniformly downsampled to the size of H×W×D (128*128*128), with the number of channels S=1, as the original input of the network. First, the input image is divided into multiple non-overlapping 3D patches to form a 3D token sequence with a dimension of It is then projected into a feature space with a feature dimension of C. The size of each patch is H'×W'×D'. The encoder structure has a total of 4 levels, and each level includes 2 Transformer blocks. When calculating self-attention, the model divides the input image into non-overlapping windows to improve the interaction efficiency between patches. Specifically, in the first Transfomer block of each level, a window size of M×M×M is used to evenly divide the 3D small blocks into multiple regions. In the second Transfomer block, these windows are shifted by M / 2 voxel units along each dimension to enhance the ability to capture information. In order to effectively calculate the shift window mechanism, 3D cyclic shift is used and self-attention is calculated according to the following formula:

[0084]

[0085] Where Q, K, and V represent query, key, and value respectively; d represents the size of the query and key.

[0086] In the first stage of the encoder, a linear embedding layer is used to generate 3D tokens. To maintain the hierarchical structure of the U-shaped encoder, the patch merging layer is used to reduce the resolution by a factor of 2 at the end of each level. In addition, the patch merging layer groups patches of resolution 2×2×2 and concatenates them, resulting in a 4C-dimensional feature embedding. The final feature size is reduced to 2C by a linear layer. The resolutions of the second, third, and fourth levels are Use the same network design as the first level.

[0087] This module is used to address challenge (2). Swin Transformer uses the SwinUnetr model combined with the Transformer's multi-head self-attention mechanism to effectively capture the global features of the image. SwinUnetr divides the image into non-overlapping patches and applies the self-attention mechanism to these patches, thereby capturing the long-range dependencies between landmarks. This solves the problem of traditional models losing global information when processing complex craniofacial structures.

[0088] The heat map regression prediction network decoder:

[0089] The heatmap regression prediction network adopts a U-shaped network architecture design, in which the image features of different resolutions extracted by the encoder are used as the input of the corresponding level of the decoder through skip connections. The decoder structure has 5 levels (t∈0,1,2,3,4), the 0th level receives the original input as input, and the 1st to 4th levels receive the output features of the corresponding 4-level structure of the encoder through skip connections as input. The input of the encoder level t is resized to And is fed into a residual block consisting of two 3×3×3 convolutional layers, which are normalized by instance normalization layers. Subsequently, the deconvolution operation is used to increase the resolution of the feature map processed by the residual block by 2 times, and the output is spliced ​​with the output of the decoder level t-1 in the channel number dimension, where the features obtained by processing the output of the decoder level 4 and the encoder level 4 through the bottleneck layer are spliced. Then, using the same residual operation, the spliced ​​features are fed into another residual block to obtain the output of the decoder level t. The output of the decoder level 0 is processed using a 1×1×1 convolutional layer and a sigmoid activation function to obtain the final heat map output. Each predicted heat map represents the probability distribution of key points, indicating the rough location of the key points. By i The integration operation is used to calculate the spatial coordinates x of the i-th rough key point i ∈R 3 , the formula is as follows:

[0090]

[0091] Key points Key points Key points Among them H i (ρ) is the voxel ρ∈R 3 The i-th heat map H at i The value of , and γi is the spatial normalization term.

[0092] Phase 2: Structure-aware network of graph attention

[0093] This method uses a graph attention module and an image decoding module to build a structure-aware network to extract structure-aware feature vectors and embed global structural features to ensure prediction accuracy.

[0094] like Figure 3 As described above, the graph attention module:

[0095] The global structure of this module is represented by the graph G = {D, V}: the predicted key points are the node sets Where n is the number of key points, x i ∈R 3 is the rough spatial coordinate value predicted at the i-th rough key point in the first stage, i.e., the spatial feature, l i ∈R d The feature vector representing the feature map extracted at the i-th coarse key point at level 0 in the first stage is the visual feature. i That is, the spatial visual feature is composed of the spatial feature x i and visual features i Edge set D = {w(v i ,v j )|i=1,2,...,n;j=1,2,...,n} represents node v i and v j The spatial relationship between them, where w(v i ,v j ) is a learnable parameter representation v i and v j The edge weights of γ From the spatial visual features v i Get the structure-aware embedding of each node in

[0096]

[0097] Among them U i is the normalization term, g is the linear transformation function. The attention function w is v i and v j The dot product similarity between:

[0098] w(v i ,v j )=θ(v i ) T φ(v j )

[0099] where θ and φ represent the key and query functions respectively.

[0100] The image decoding module:

[0101] The original size is the spatial coordinate x of the i-th coarse keypoint i The 3D patch obtained by cutting around is represented as Through the image decoding module F β Learn visual feature embedding

[0102]

[0103] Eventually and Feature fusion passes through the linear layer to obtain the offset x offset :

[0104]

[0105] This module is designed to address the above challenge (3) by introducing a graph attention module to perceive and utilize these structural features. The graph attention module constructs a graph structure between anatomical points and uses the attention mechanism to calculate the dependency of each anatomical point with its neighboring points, thereby capturing the geometric and spatial features of the anatomical points.

[0106] The further loss function design is as follows:

[0107] Loss function in the first stage heatmap regression prediction network for:

[0108]

[0109] where g i is the real coordinate of the key point. i (ρ) is the i-th heat map H i At voxel ρ∈R 3 The value of σ h is the standard deviation.

[0110] Loss function in structure-aware networks in the second stage of graph attention for:

[0111]

[0112] Therefore, the total loss function for:

[0113]

[0114] Among them, λ = 0.5 represents the weight coefficient between the two loss functions.

[0115] Step (3) obtains the prediction results through student network pre-training and semi-supervised teacher-student model training:

[0116] The student network pre-training phase used labeled data, and the training used an L20 graphics card. A total of 1000 epochs were trained using the Adam optimizer. The learning rate was set to 1e-4, and the cosine annealing scheduler was set to every 50 epochs. The minimum learning rate was set to 1e-6. After the initial student network was obtained, semi-supervised architecture training was performed.

[0117] like Figure 4 As shown, the teacher-student network architecture is designed as follows:

[0118] The teacher network uses pre-trained student model parameters, and uses unlabeled data in the training phase to participate in the training to generate pseudo labels for the student model. No pre-training parameters are introduced in the student network training, and labeled data and unlabeled data are trained together. A large amount of unlabeled high-quality image data is used to guide the student model to predict more accurate results. Specifically, the teacher network does not set gradient updates, but is updated through exponential averaging (EMA), so that the prediction results of the teacher network will be more stable. The student network updates the parameters after each training epoch to better imitate the teacher network. In order to keep the student network training smoother, the teacher network parameters are updated when >20 epochs are set. The quality of the pseudo labels of the teacher network is often uneven. This method uses the structure-aware network results of the second-stage graph attention to screen effective data to guide the student network. Key points: This method proposes the following pseudo label generation strategy:

[0119]

[0120] where X α The spatial coordinate value x predicted in the first stage i , X β The offset x predicted for the second stage offset , the threshold Δ is set to 2mm error, X γ This is the final reliable pseudo label.

[0121] Using the optimization methods and parameter settings of the pre-training stage, the student network finally obtains the final prediction after iteratively learning the teacher network for a total of 300 epochs.

[0122] This module is designed to address the aforementioned challenge (3). This paper proposes a teacher-student model based on semi-supervised learning to generate pseudo-labels. First, a teacher model is trained using labeled data. The teacher model generates pseudo-labels on unlabeled data. These pseudo-labels are used to guide the training of the student model, thereby effectively utilizing unlabeled data, expanding the training set size, and improving the model's generalization ability.

[0123] The above description is a further detailed description of the present invention in conjunction with specific / preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. Those skilled in the art of the present invention may make various substitutions or modifications to the described embodiments without departing from the scope of the present invention, and such substitutions or modifications should be considered to fall within the scope of protection of the present invention.

[0124] Parts of the present invention that are not described in detail belong to the common knowledge of those skilled in the art.

Claims

1. A semi-supervised cephalometric key point detection method based on structural feature perception, characterized in that: The steps are as follows: Step (1), data preprocessing and construction of labeled datasets and unlabeled datasets; Step (2), two-stage semi-supervised model construction and loss function design; The backbone network of the two-stage semi-supervised model framework consists of two stages: the first stage uses global context information to detect the rough location of key points through a heatmap regression method; The second stage makes refined predictions by capturing the global-local correlations between key points; Step (3), obtain the prediction results through student network pre-training and semi-supervised teacher-student model training; The two-stage semi-supervised model is divided into two main parts: a teacher network and a student network. The teacher network is trained using network parameters learned from existing labeled data and generates pseudo labels on unlabeled data. The quality of the pseudo labels is improved by combining a graph attention model. The student network is first pre-trained to obtain initial student network parameters. Then, the student network is iteratively trained based on the pseudo labels generated by the teacher network to predict the final results. The first stage of the two-stage semi-supervised model is the heat map regression prediction network: The heatmap regression prediction network structure is divided into two parts: encoder and decoder. The encoder uses Swin Transformer to extract features step by step, while the decoder uses skip connections and residual blocks to combine multi-scale features step by step to predict the heatmap. The encoder of the heat map regression prediction network: The input accepted by the encoder is a 3D voxel image, which is uniformly downsampled to a size of , the number of channels is S, which is the original input of the network; first, the input image is divided into multiple non-overlapping 3D patches to form a 3D token sequence with a dimension of , and then projected into a feature space with a feature dimension of C; the size of each patch is ; The encoder structure has 4 levels, each level includes 2 Transformer blocks; When calculating self-attention, the model divides the input image into non-overlapping windows to improve the interaction efficiency between patches; Specifically, in the first Transformer block of each level The window size is used to evenly divide the 3D block into multiple regions; the second Transfomer block Shift M / 2 voxel units along each dimension to enhance the ability to capture information; in order to effectively calculate the shift window mechanism, 3D circular shift is used and self-attention is calculated according to the self-attention calculation formula; In the first stage of the encoder, a linear embedding layer is used to generate 3D tokens; In order to maintain the hierarchical structure of the U-shaped encoder, the patch merging layer is used to reduce the resolution by a factor of 2 at the end of each level; In addition, the patch merging layer converts the resolution The patches are grouped and concatenated to produce a 4C-dimensional feature embedding; finally The feature size is reduced to 2C through the linear layer; the resolution of the second, third and fourth levels are 、 、 , using the same network design as the first level; The heat map regression prediction network decoder: The heat map regression prediction network adopts a U-shaped network architecture design, in which the image features of different resolutions extracted by the encoder are used as the input of the corresponding layer of the decoder through skip connections; the decoder structure has a total of 5 levels. , Level 0 receives the original input as input, and Levels 1-4 receive the output features of the 4-level structure of the corresponding encoder as input through skip connections; The input to the decoder stage t is resized to , and is fed to the In the residual block composed of convolutional layers, these convolutional layers are normalized by instance normalization layers; Subsequently, the deconvolution operation is used to increase the resolution of the feature map after the residual block processing by 2 times, and the output is spliced ​​with the output of the decoder level t-1 in the channel number dimension, where the output of the decoder level 4 is spliced ​​with the features obtained by the bottleneck layer processing of the encoder level 4; Then, using the same residual operation, the concatenated features are fed into another residual block to obtain the output of the decoder at level t; the output of the decoder at level 0 is fed into another residual block using The final heat map output is obtained after the convolution layer and sigmoid activation function processing; each predicted heat map represents the probability distribution of key points and indicates the rough location of key points; The integral operation is used to calculate the Key point space coordinates , the formula is as follows: in In voxels The first , and is the spatial normalization term; The second stage of the two-stage semi-supervised model is the structure-aware network of graph attention, which is as follows: The 3D image near the key point is cropped from the original volume image without downsampling to obtain a 3D patch as the input of the structure-aware network of the graph attention so as not to lose the geometric details used for fine-scale detection; A graph attention module and an image decoding module are used to build a graph attention structure-aware network to extract structure-aware feature vectors and embed global structural features to ensure prediction accuracy. The graph attention module: The global structure of this module is represented by Representation: The predicted key point is the node set , where n is the number of key points In the first stage The rough spatial coordinate value predicted by the rough key point is the spatial feature. Represents the first stage of level 0 in the Feature maps extracted from coarse keypoints The feature vector is the visual feature; node That is, the spatial visual features are composed of spatial features splicing to obtain; edge set Representation node The spatial relationship between is a learnable parameter representation and The edge weights of From the spatial visual features Get the structure-aware embedding of each node in : in is the normalization term, is a linear transformation function; attention function yes The dot product similarity between: in and Represents key and query functions respectively; The image decoding module: Original size in Coarse keypoint space coordinates The 3D patch obtained by cutting around is represented as ; Through the image decoding module Learn visual feature embedding : Eventually and Feature fusion is passed through the linear layer to obtain the offset : 。 2. The method for semi-supervised cephalometric key point detection based on structural feature perception according to claim 1, characterized in that: The step (1) is specifically implemented as follows: First, all 3DCBCT voxel data in the internal dataset are standardized to a uniform resolution and pixel interval, and then data augmentation is applied. The data are divided into labled and unlabled parts based on whether or not they are labeled. At the same time, the key point information of the labeled data is stored in the corresponding json file. Among them, the internal dataset consists of the annotation files corresponding to 3DCBCT and some CT. The clinical CBCT data obtained from the hospital are divided into labeled and unlabeled groups. The labeled group is annotated by doctors to generate annotation files by annotating key points.

3. The method for semi-supervised cephalometric key point detection based on structural feature perception according to claim 1, characterized in that: The loss function design of the two-stage semi-supervised model is as follows: Loss function in the first stage heatmap regression prediction network for: in are the real coordinates of the key points; It is Heatmap In voxel The value of is the standard deviation; Loss function in structure-aware networks in the second stage of graph attention for: Therefore, the total loss function for: in, Represents the weight coefficient between the two loss functions.

4. The method for semi-supervised cephalometric key point detection based on structural feature perception according to claim 1, characterized in that: The specific implementation of step (3) is as follows: The student network pre-training phase uses labeled data. During the training process, the Adam optimizer is used to set the learning rate, learning rate parameters, and cosine annealing scheduler to optimize the initial student network, and then semi-supervised architecture training is performed. The teacher-student network architecture is designed as follows: The teacher network uses pre-trained student model parameters. During the training phase, unlabeled data is used to participate in the training and generate pseudo labels for the student model. No pre-trained parameters are introduced during the student network training, and both labeled and unlabeled data are trained together. Specifically, the teacher network is updated using exponential averaging, and the student network updates its parameters after each training epoch to better imitate the teacher network. The quality of the teacher network's pseudo labels is often uneven. Therefore, the results of the structure-aware network of the second stage of graph attention are used to screen effective data to guide the student network. The following pseudo label generation strategy is proposed: in The spatial coordinate value predicted in the first stage , Offset predicted for the second stage , threshold Set to 2mm error, This is the final reliable pseudo label; Finally, the prediction result obtained by the student network through iterative learning of the teacher network is the final prediction.

Images