A method for detecting landmarks in lateral cephalometric radiographs based on hybrid encoders and convolutional embedding.

Abstract

This invention discloses a method for landmark detection in lateral cephalometric images based on a hybrid encoder and convolutional embedding. The method preprocesses the original lateral cephalometric image to obtain a multi-scale Gaussian pyramid centered at the predicted location as the input image. This pyramid is then input into a hybrid encoder to obtain low-level feature vectors encoded by a convolutional neural network and high-level feature vectors encoded by a Swing Transformer. These two vectors are concatenated to obtain a hybrid feature vector. This hybrid feature vector is then input into a convolutional embedding layer for embedding, reducing the dimensionality to obtain an embedded feature vector. This embedded feature vector is then input into a graph attention module to calculate the attention values ​​between different landmarks, resulting in an attention feature vector. Finally, a three-layer MLP regression is used to obtain the relative offset, update the landmark positions, and iterate a certain number of times. This invention significantly enhances the encoder's ability to encode images at different scales and extract contextual information, and the model exhibits good performance in terms of accuracy and parameter size.

Description

Technical Field

[0001] This application belongs to the field of medical image processing technology, and in particular relates to a method for detecting landmarks in lateral cephalometric radiographs based on hybrid encoders and convolutional embedding. Background Technology

[0002] With increasing demands for appearance and growing concern for oral health, cosmetic surgery and orthodontic diagnosis and treatment have become popular medical services. In cosmetic and orthodontic treatment, the detection of anatomical landmarks of the skull is a primary and crucial task. Doctors need to measure and analyze the lines and angles drawn from relevant anatomical landmarks to understand the angular relationships between them. This allows for quantitative analysis of craniofacial morphology, diagnosis of malocclusions, evaluation of treatment effectiveness, and observation of growth and development. Lateral cephalometric radiographs are taken to observe any abnormalities within the skull; taking lateral cephalometric radiographs is an indispensable and preferred examination step in orthodontic treatment.

[0003] In previous clinical practice, orthodontists had to manually mark each anatomical landmark from images one by one, which was time-consuming, labor-intensive, and highly dependent on the operator's experience, resulting in individual biases. AI-based automated anatomical landmark detection methods can help reduce the workload of orthodontists, decrease errors, and facilitate treatment simulation and plan development.

[0004] The patent title is "Automatic Keypoint Detection Method and System for Lateral Cranial Radiographs Based on Deep Learning," publication number CN110706203A, authorization date 2020-01-17. The patent describes a method and system for automatic keypoint detection in lateral cephalographic radiographs based on deep learning. The method first prepares a lateral cephalographic radiograph and labels it with predicted bounding boxes and keypoints. Then, two deep neural network models are established, one for predicting bounding boxes and the other for keypoints. Both models are trained under supervised supervision using labeled samples. Finally, the lateral radiograph is input into the predicting bounding box model to obtain predicted bounding boxes. The image within the predicted bounding box region is then extracted, and the extracted image is input into the keypoint model to obtain the keypoint coordinates.

[0005] The patent title is "An Intelligent Cephalometric Measurement Method Based on Deep Neural Networks for Lateral Views," publication number CN113065552A, authorized on 2021-12-21. The patent describes an intelligent cephalometric measurement method based on deep neural networks for lateral view images. It imports and calibrates lateral view image data, and performs data segmentation and preprocessing. First, feature extraction is performed on the lateral view images. Then, an encoder module is constructed to generate a backbone network for the overall heatmap and heatmaps of all landmark points. A relationship reasoning module is constructed to infer the existence and meaning of relationships between landmark points. Next, a decoder module is constructed to utilize the positional and spatial information of all landmark points, selecting core key landmark points to increase priority and improve prediction accuracy. An end-to-end model is trained, optimizing the loss functions for positional, spatial, and measurement metrics on the lateral view images to obtain a trained model. Finally, landmark detection and cephalometric measurement analysis are performed based on the trained end-to-end model.

[0006] The patent, titled "Automatic Keypoint Detection System and Method for Lateral Cranial Radiographs Based on Deep Learning," with publication number CN114820517A and authorization date of 2022-07-29, describes an automatic keypoint detection system and method for lateral cephalial radiographs based on deep learning. The method first involves collecting and preprocessing lateral cephalial radiograph images, and then labeling the keypoint locations of each sample in the training set. Next, an encoder-decoder keypoint recognition model is constructed. This is followed by training and evaluation of the keypoint recognition model. Finally, keypoint prediction is performed on new lateral cephalial radiographs.

[0007] To date, most deep learning-based methods for automatic anatomical landmark detection employ convolutional neural networks (CNNs) as encoders to process images. The input images are typically of one or two scales, limiting the extraction of only the highest and lowest level features; features between these scales are difficult to extract. Furthermore, the detail-capturing capabilities of CNNs weaken their global modeling ability, making them less adept at extracting high-level features. Traditional CNNs also often use fully connected layers to embed extracted features, reducing feature dimensionality and resulting in large model parameters. Therefore, a method for automatic anatomical landmark detection that combines high accuracy with small model parameters is still underdeveloped. Summary of the Invention

[0008] The purpose of this application is to propose a method for detecting cephalographic landmarks based on hybrid encoders and convolutional embedding, which improves the problems of insufficient detection accuracy caused by using only convolutional neural networks to process images in existing technologies and excessively large model parameters caused by using fully connected layers to embed features for dimensionality reduction in the encoding structure.

[0009] To achieve the above objectives, the technical solution of this application is as follows:

[0010] A method for detecting landmarks in lateral cephalometric radiographs based on hybrid encoders and convolutional embeddings includes:

[0011] Calculate the prior distribution of the marker points based on the annotations in the known dataset to obtain the initial predicted positions of the marker points. Preprocessing was performed on the original lateral cephalometric radiograph to obtain... The multi-scale Gaussian pyramid I centered on the center;

[0012] After downsampling each layer of the Gaussian pyramid I to a uniform size, the feature vector F is obtained by inputting it into the hybrid encoder. CNN and F ST , will F CNN and F ST The feature vector F is obtained by concatenation. concat ;

[0013] The feature vector F concat The input is fed into a convolutional embedding layer for embedding operations, and the dimensionality is reduced to obtain the feature vector F. embedding ;

[0014] The obtained feature vector F embedding The input is fed into the graph attention module to obtain the feature vector F. attention ;

[0015] The feature vector F attention The input is fed into a three-layer MLP for regression to obtain the relative offset. And according to and Update the position of the marker. Get new A Gaussian pyramid centered on the central point is constructed, and multiple iterations are performed to obtain the final positions of each marker.

[0016] Furthermore, the prior distribution of the marker points is calculated based on the annotations in the known dataset to obtain the initial predicted positions of the marker points. Preprocessing was performed on the original lateral cephalometric radiograph to obtain... The multi-scale Gaussian pyramid I centered on [center] includes:

[0017] Step 1.1: Calculate the prior distribution of the marker points based on the annotations in the known dataset, and obtain the mean position μ and standard deviation σ;

[0018] Step 1.2: Obtain the random initial positions that follow a normal distribution based on μ and σ.

[0019] Step 1.3: Apply Gaussian convolution to the original lateral cephalometric radiograph to obtain a Gaussian blurred image. By cropping regions of different sizes centered on the data, a multi-scale Gaussian pyramid I with N marker points and M layers is obtained as input. The dimensions of each layer of the image are (H1×2) ^(i-1) )×(W1×2 ^(i-1) ), where H1 is the height of the first layer of the pyramid image, W1 is the width of the first layer of the pyramid image, and i corresponds to the i-th layer image, with values ​​ranging from 1 to M.

[0020] Furthermore, the feature vector F is obtained by downsampling each layer of the Gaussian pyramid I to a uniform size and then inputting it into the hybrid encoder. CNN and F ST , will F CNN and F ST The feature vector F is obtained by concatenation. concat ,include:

[0021] Step 2.1: Divide the Gaussian Pyramid I into two layers based on its scale. CNN and I ST Each layer of the image is downsampled to a uniform size, with dimensions of N×K×1×H2×W2 and N×(MK)×1×H2×W2, respectively. N is the number of pyramids and equal to the number of marker points, M is the number of layers in the pyramid, and K is the number of layers in layer M that are input to the convolutional encoder.

[0022] Step 2.2: Convert the original smaller-scale layer I... CNN The input is fed into the convolutional layer of the convolutional encoder to obtain the feature map F1, which has dimensions of N×K×C1×H3×W3;

[0023] Step 2.3: Max pool the feature map F1 and then input it into the first convolutional block in the convolutional encoder to obtain the feature map F2, which has dimensions of N×K×C1×H4×W4.

[0024] Step 2.4: Input feature map F2 into the second convolutional block in the convolutional encoder to obtain feature map F3, whose dimensions are N×K×C2×H5×W5;

[0025] Step 2.5: Input feature map F3 into the third convolutional block in the convolutional encoder to obtain feature map F4, whose dimensions are N×K×C3×H6×W6;

[0026] Step 2.6: Input feature map F4 into the fourth convolutional block in the convolutional encoder to obtain feature map F5, whose dimensions are N×K×C4×H7×W7;

[0027] Step 2.7: Perform average pooling on feature map F5 to obtain feature vector F. CNN Its dimension size is This indicates that the vector length is equal to the number of channels, C4.

[0028] Step 2.8: Convert the original larger-scale layer I... ST The image patch embedding module of the Swin Transformer encoder is input to obtain the embedding vector F. PE Its dimension size is in This means the vector length is equal to the product of the height H4 and the width W4 in feature map F2. This indicates that the vector length is equal to the number of channels, C1.

[0029] Step 2.9: Embed the vector F PE The input is fed into the first layer of the Swing Transformer encoder to obtain the feature vector F6, which has a dimension of [missing information].

[0030] Step 2.10: Input feature vector F6 into the second coding block of the Swing Transformer encoder to obtain feature vector F7, whose dimension is... in This indicates that the vector length is equal to the product of the height H5 and the width W5 in feature map F3. This indicates that the vector length is equal to the number of channels, C2.

[0031] Step 2.11: Input feature vector F7 into the third coding block of the Swing Transformer encoder to obtain feature vector F8, whose dimension is... in This indicates that the vector length is equal to the product of the height H6 and the width W6 in feature map F4. This indicates that the vector length is equal to the number of channels, C3.

[0032] Step 2.12: Input feature vector F8 into the fourth coding block of the Swing Transformer encoder to obtain feature vector F9, whose dimension is... in This indicates that the vector length is equal to the product of the height H7 and the width W7 in feature map F5. This indicates that the vector length is equal to the number of channels, C4.

[0033] Step 2.13: Swap the third and fourth dimensions of the feature vector F9, and perform average pooling on the fourth dimension to obtain the feature vector F. ST Its dimension size is

[0034] Step 2.14, for the feature vector F CNN and F ST Performing a concatenation operation on the second dimension yields F. concat Its dimension size is

[0035] Where C1 to C4 are the number of channels, H2 to H7 are the height of the image, and W2 to W7 are the width of the image.

[0036] Furthermore, the feature vector F concat The input is fed into a convolutional embedding layer for embedding operations, and the dimensionality is reduced to obtain the feature vector F. embedding ,include:

[0037] Step 3.1, in the feature vector F concat Expanding the second dimension yields the feature vector F. flatten Its dimensions are in The representative performs an expansion operation on F. concat M of length The feature vectors are concatenated, where N is the number of pyramids and equals the number of marker points, and M is the number of layers in the pyramid.

[0038] Step 3.2: Convert the feature vector F flatten The input is fed into a convolutional embedding layer, and then convolved using grouped one-dimensional convolutional layers to obtain the dimensionality-reduced feature vector F. embedding Its dimension is N×L1, where L1 is the vector length.

[0039] Furthermore, the obtained feature vector F embedding The input is fed into the graph attention module to obtain the feature vector F. attention ,include:

[0040] Step 4.1: Convert the feature vector F embedding By performing fully connected operations on three linear layers Q, K, and V, we obtain vector F. Q F K F V Each of them has a dimension of N×L2, where the vector length L2 is the number of output nodes of the linear layers Q, K, and V;

[0041] Step 4.2, for vector F Q and the transposed F K Perform a dot product operation to obtain the attention matrix G. matrix Its dimensions are N×N;

[0042] Step 4.3: Transfer the attention matrix G matrixThe input is standardized by a batch processing normalization layer, then activated and normalized using the following formula to obtain the attention heatmap G. attention Its dimensions are N×N;

[0043]

[0044] Step 4.4: Transfer the attention heatmap G attention sum vector F V Perform a dot product operation to obtain the attention feature F between each marker point. G Its dimension is N×L2;

[0045] Step 4.5: Transfer attention features F G A fully connected operation is performed through a linear layer R to restore the attention feature vector to a dimension of N×L1. Then, the attention feature vector is combined with F. embedding Performing a residual connection operation yields the feature vector F. attention Its dimension is N×L1.

[0046] Furthermore, the feature vector F attention The input is fed into a three-layer MLP for regression to obtain the relative offset. And according to and Update the position of the marker. Get new A Gaussian pyramid centered on the central point is constructed, and multiple iterations are performed to obtain the final positions of each marker. include:

[0047] Step 5.1: Convert the feature vector F attention The input is fed into a fully connected linear layer P1, activated by the ReLU function, and then dropped out to obtain the feature vector F. MLP1 Its dimension is N×L3, where the vector length L3 is the number of output nodes of the linear layer P1;

[0048] Step 5.2: Convert the feature vector F MLP1 The input is fed into a fully connected linear layer P2, activated by the ReLU function, and then dropped out to obtain the feature vector F. MLP2 Its dimension is N×L4, where the vector length L4 is the number of output nodes of the linear layer P2;

[0049] Step 5.3: Transfer the feature vector F MLP2 The input is fed into a fully connected linear layer P3, and regression is performed to obtain the relative offset. Its dimension is N×2;

[0050] Step 5.4: Update according to the formula below. and with A new multi-scale Gaussian pyramid I is constructed around the center and used as the input for the next iteration. After 10 iterations... The final positions of each marker:

[0051]

[0052] Where t represents the current iteration number, and its value ranges from 1 to 10.

[0053] Furthermore, the convolutional embedding layer includes N grouped one-dimensional convolutions with a kernel size of 384 and a stride of 3. Each type of marker point is embedded using a one-dimensional convolutional layer to achieve dimensionality reduction. The length L1 of the dimensionality-reduced vector is calculated using the following formula:

[0054]

[0055] This application proposes a method for lateral cephalometric radiograph landmark detection based on a hybrid encoder and convolutional embedding. On one hand, by constructing a Gaussian pyramid and using a hybrid encoder for multi-scale images, the model can better extract information from different scale levels in the lateral radiograph, significantly improving the network's landmark detection accuracy. On the other hand, by introducing a convolutional embedding layer, grouped one-dimensional convolutional layers are used instead of fully connected layers for feature embedding and dimensionality reduction, greatly reducing model parameters. Furthermore, through a graph attention module, the model can combine feature information between different landmarks to extract global contextual information, enhancing the model's generalization ability when processing lateral radiographs with inconspicuous texture information, ultimately resulting in more accurate lateral radiograph landmark detection. Attached Figure Description

[0056] Figure 1 This is a flowchart of the cephalometric landmark detection method based on hybrid encoder and convolutional embedding in this application;

[0057] Figure 2 A schematic diagram illustrating the generation and downsampling of a Gaussian pyramid for the image in this application;

[0058] Figure 3 This is a schematic diagram of the network structure in an embodiment of this application;

[0059] Figure 4 This is a schematic diagram of the structure of the convolutional embedding layer in this application;

[0060] Figure 5 This is a schematic diagram of the structure of the attention module in this application;

[0061] Figure 6 This is a schematic diagram of the three-layer MLP structure of this application. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0063] In one embodiment, such as Figure 1 As shown, a method for detecting landmarks in lateral cephalometric radiographs based on a hybrid encoder and convolutional embedding is provided, including:

[0064] Step S1: Calculate the prior distribution of the marker points based on the annotations in the known dataset to obtain the initial predicted positions of the marker points. Preprocessing was performed on the original lateral cephalometric radiograph to obtain... The multi-scale Gaussian pyramid I centered on the image is used as the input image.

[0065] The method described in this application calculates the prior distribution of marker points based on the annotations in a known dataset to obtain the initial predicted positions of the marker points. Preprocessing was performed on the original lateral cephalometric radiograph to obtain... The multi-scale Gaussian pyramid I centered on the image is used as the input image, such as... Figure 2 As shown, it includes:

[0066] Step 1.1: Calculate the prior distribution of the marker points based on the annotations in the known dataset, and obtain the mean position μ and standard deviation σ;

[0067] Step 1.2: Obtain the random initial positions that follow a normal distribution based on μ and σ.

[0068] Step 1.3: Apply Gaussian convolution to the original lateral cephalometric radiograph to obtain a Gaussian blurred image. By cropping regions of different sizes centered on the data, a multi-scale Gaussian pyramid I with N marker points and M layers is obtained as input. The dimensions of each layer are (H1×2^) (i-1) )×(W1×2^ (i-1) ), where H1 is the height of the first layer of the pyramid image, W1 is the width of the first layer of the pyramid image, and i corresponds to the i-th layer image, with values ​​ranging from 1 to M;

[0069] Step S2: After downsampling the Gaussian pyramid I to a uniform size, input it into the hybrid encoder to obtain the feature vector F. CNN and F ST , will F CNN and F ST The feature vector F is obtained by concatenation. concat .

[0070] The feature vector F is obtained by downsampling I to a uniform size and then inputting it into a hybrid encoder.CNN and F ST , will F CNN and F ST The feature vector F is obtained by concatenation. concat ,include:

[0071] Step 2.1: Divide the Gaussian Pyramid I into two layers based on its scale. CNN and I ST Each layer of the image is downsampled to a uniform size, with dimensions of N×K×1×H2×W2 and N×(MK)×1×H2×W2, respectively. N is the number of pyramids and equal to the number of marker points, M is the number of layers in the pyramid, and K is the number of layers in layer M that are input to the convolutional encoder.

[0072] Step 2.2: Convert the original smaller-scale layer I... CNN The input is fed into a 7×7 convolutional layer of the convolutional encoder to obtain feature map F1, which has dimensions of N×K×C1×H3×W3.

[0073] Step 2.3: Max pool the feature map F1 and then input it into the first convolutional block in the convolutional encoder to obtain the feature map F2, which has dimensions of N×K×C1×H4×W4.

[0074] Step 2.4: Input feature map F2 into the second convolutional block in the convolutional encoder to obtain feature map F3, whose dimensions are N×K×C2×H5×W5;

[0075] Step 2.5: Input feature map F3 into the third convolutional block in the convolutional encoder to obtain feature map F4, whose dimensions are N×K×C3×H6×W6;

[0076] Step 2.6: Input feature map F4 into the fourth convolutional block in the convolutional encoder to obtain feature map F5, whose dimensions are N×K×C4×H7×W7;

[0077] Step 2.7: Perform average pooling on feature map F5 to obtain feature vector F. CNN Its dimension size is This indicates that the vector length is equal to the number of channels, C4.

[0078] Step 2.8: Convert the original larger-scale layer I... ST The image patch embedding module of the Swin Transformer encoder is input to obtain the embedding vector F. PE Its dimension size is in This means the vector length is equal to the product of the height H4 and the width W4 in feature map F2. This indicates that the vector length is equal to the number of channels, C1.

[0079] Step 2.9: Embed the vector F PE The input is fed into the first layer of the Swing Transformer encoder to obtain the feature vector F6, which has a dimension of [missing information].

[0080] Step 2.10: Input feature vector F6 into the second coding block of the SwinTransformer encoder to obtain feature vector F7, whose dimension is... in This indicates that the vector length is equal to the product of the height H5 and the width W5 in feature map F3. This indicates that the vector length is equal to the number of channels, C2.

[0081] Step 2.11: Input feature vector F7 into the third coding block of the Swing Transformer encoder to obtain feature vector F8, whose dimension is... in This indicates that the vector length is equal to the product of the height H6 and the width W6 in feature map F4. This indicates that the vector length is equal to the number of channels, C3.

[0082] Step 2.12: Input feature vector F8 into the fourth coding block of the SwingTransformer encoder to obtain feature vector F9, whose dimension is... in This indicates that the vector length is equal to the product of the height H7 and the width W7 in feature map F5. This indicates that the vector length is equal to the number of channels, C4.

[0083] Step 2.13: Swap the third and fourth dimensions of the feature vector F9, and perform average pooling on the fourth dimension to obtain the feature vector F. ST Its dimension size is

[0084] Step 2.14, for the feature vector F CNN and F ST Performing a concatenation operation on the second dimension yields F. concat Its dimension size is

[0085] Where C1 to C4 are the number of channels, H2 to H7 are the height of the image, and W2 to W7 are the width of the image.

[0086] It should be noted that in this application, H and W represent the height and width of the image, and their subscripts represent different images; L represents the vector length, and its subscripts represent different vectors, which will not be elaborated further below.

[0087] Step S3: Obtain the feature vector F concat The input is fed into a convolutional embedding layer for embedding operations, and the dimensionality is reduced to obtain the feature vector F. embedding .

[0088] The feature vector F described in this application concat The input is fed into a convolutional embedding layer for embedding operations, and the dimensionality is reduced to obtain the feature vector F. embedding ,include:

[0089] Step 3.1, in the feature vector F concat Expanding the second dimension yields the feature vector F. flatten Its dimensions are in The representative performs an expansion operation on F. concat M of length The concatenation of eigenvectors;

[0090] Step 3.2: Convert the feature vector F flatten The input is fed into a convolutional embedding layer, and then convolved using grouped one-dimensional convolutional layers to obtain the dimensionality-reduced feature vector F. embedding Its dimension is N×L1, where L1 is the vector length.

[0091] Step S4: Obtain the feature vector F embedding The input is fed into the graph attention module to obtain the feature vector F. attention .

[0092] The feature vector F obtained as described in this application embedding The input is fed into the graph attention module to obtain the feature vector F. attention ,like Figure 5 As shown, it includes:

[0093] Step 4.1: Convert the feature vector F embedding By performing fully connected operations on three linear layers Q, K, and V, we obtain vector F. Q F K F V Each of them has a dimension of N×L2, where the vector length L2 is the number of output nodes of the linear layers Q, K, and V;

[0094] Step 4.2, for vector F Q and the transposed F K Perform a dot product operation to obtain the attention matrix H. matrix Its dimensions are N×N;

[0095] Step 4.3: Transfer the attention matrix H matrix The input is standardized by the batch processing normalization layer, then activated using the following formula and normalized to obtain the attention heatmap H. attention Its dimensions are N×N;

[0096]

[0097] Step 4.4: Transfer the attention heatmap H attention sum vector F V Perform a dot product operation to obtain the attention feature F between each marker point. H Its dimension is N×L2;

[0098] Step 4.5: Transfer attention features F H A fully connected operation is performed through a linear layer R to restore the attention feature vector to a dimension of N×L1. Then, the attention feature vector is combined with F. embedding Performing a residual connection operation yields the feature vector F. attention Its dimension is N×L1.

[0099] Step S5: Transfer the feature vector F attention The input is fed into a three-layer MLP for regression to obtain the relative offset. And according to and Update the position of the marker. New ones are obtained in succession A Gaussian pyramid centered on the central point is constructed, and multiple iterations are performed to obtain the final positions of each marker.

[0100] The feature vector F described in this application attention The input is fed into a three-layer MLP for regression to obtain the relative offset. And according to and Update the position of the marker. Get new A Gaussian pyramid centered on the central point is constructed, and multiple iterations are performed to obtain the final positions of each marker. include:

[0101] Step 5.1: Convert the feature vector F attention The input is fed into a fully connected linear layer P1 and activated by a ReLU function to obtain the feature vector F. MLP1 Its dimension is N×L3, where the vector length L3 is the number of output nodes of the linear layer P1;

[0102] Step 5.2: Convert the feature vector F MLP1The input is fed into a fully connected linear layer P2 and activated by a ReLU function to obtain the feature vector F. MLP2 Its dimension is N×L4, where the vector length L4 is the number of output nodes of the linear layer P2;

[0103] Step 5.3: Transfer the feature vector F MLP2 The input is fed into a fully connected linear layer P3, and regression is performed to obtain the relative offset. Its dimension is N×2;

[0104] Step 5.4: Update according to the formula below. and with A new multi-scale Gaussian pyramid I is constructed around the center and used as the input for the next iteration. After 10 iterations... The final positions of each marker:

[0105]

[0106] Where t represents the current iteration number, and its value ranges from 1 to 10.

[0107] The method and technical framework described in this application, such as Figure 3 As shown, in other embodiments, the number of hybrid encoder and hybrid decoder network layers can be modified.

[0108] The convolutional embedding layer includes N grouped one-dimensional convolutions with a kernel size of 384 and a stride of 3. Each type of marker point is embedded using a separate one-dimensional convolutional layer to achieve dimensionality reduction. The feature embedding effect of a single marker point is as follows: Figure 4 As shown, the length L1 of the embedded vector is calculated using the following formula.

[0109]

[0110] The third MLP layer, such as Figure 6 The diagrams show the operations for a fully connected layer, an activation layer, and dropout, as well as the operations for a fully connected layer, an activation layer, and dropout.

[0111] It should be noted that in this application, N is the number of marker points, M is the number of layers in the Gaussian pyramid, K is the number of layers in the Gaussian pyramid input to the convolutional encoder, C is the number of channels, H is the height of the image, W is the width of the image, L is the vector length, and the subscripts of the letters indicate the sequence number, which is used to distinguish the dimensions of different feature maps and feature vectors.

[0112] This application constructs a Gaussian pyramid to help extract features from lateral lesions at different scales and designs a hybrid encoder to address the problem that convolutional network layers are not good at extracting high-level global features. Convolutional embedding layers composed of grouped one-dimensional convolutional layers are used instead of fully connected layers for feature embedding and dimensionality reduction, significantly reducing model parameters. Furthermore, through a graph attention module, the model can combine feature information between different landmarks to extract global contextual information, enhancing the model's generalization ability when processing lateral lesions with inconspicuous texture information.

[0113] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for detecting landmarks in lateral cephalometric radiographs based on hybrid encoders and convolutional embedding, characterized in that, The method for detecting landmarks in lateral cephalometric radiographs based on hybrid encoders and convolutional embeddings includes: Calculate the prior distribution of the marker points based on the annotations in the known dataset to obtain the initial predicted positions of the marker points. Preprocessing was performed on the original lateral cephalometric radiograph to obtain... The multi-scale Gaussian pyramid centered on ; Gauss Pyramid After downsampling each layer to a uniform size, the samples are input into the hybrid encoder to obtain feature vectors. and ,Will and Concatenation yields the feature vector ; eigenvectors The input is fed into a convolutional embedding layer for embedding operations, and the dimensionality is reduced to obtain the feature vector. ; The obtained feature vector The input is fed into the graph attention module to obtain the feature vector. ; eigenvectors The input is fed into a three-layer MLP for regression to obtain the relative offset. and according to and Update the position of the marker. to obtain new A Gaussian pyramid centered on the central point is constructed, and multiple iterations are performed to obtain the final positions of each marker. ; Wherein, the feature vector The input is fed into a convolutional embedding layer for embedding operations, and the dimensionality is reduced to obtain the feature vector. ,include: Step 3.1, in the feature vector Expanding the second dimension yields the feature vector. Its dimensions are ,in The representative expands the operation to M lengths are The concatenation of eigenvectors, The number of pyramids is equal to the number of marker points. This represents the number of layers in the pyramid. Step 3.2: Convert the feature vector The input is fed into a convolutional embedding layer, and then convolved using grouped one-dimensional convolutional layers to obtain the dimensionality-reduced feature vector. Its dimensions are , The length of the vector; The convolutional embedding layer includes N grouped one-dimensional convolutions with a kernel size of 384 and a stride of 3. Each type of marker point is embedded using a separate one-dimensional convolutional layer to achieve dimensionality reduction. The length of the dimensionality-reduced vector is... The calculation method is as follows: 。 2. The method for detecting landmark points in lateral cephalometric radiographs based on hybrid encoders and convolutional embedding as described in claim 1, characterized in that, The prior distribution of the marker points is calculated based on the annotations in the known dataset to obtain the initial predicted positions of the marker points. Preprocessing was performed on the original lateral cephalometric radiograph to obtain... The multi-scale Gaussian pyramid centered on ,include: Step 1.1: Calculate the prior distribution of marker points based on the annotations in the known dataset to obtain the average position. Standard deviation ; Step 1.2, according to and Obtain random initial positions that follow a normal distribution ; Step 1.3: Apply Gaussian convolution to the original lateral cephalometric radiograph to obtain a Gaussian blurred image. By cropping regions of different sizes from the center, the number of marker points can be obtained. indivual Multi-scale Gaussian pyramid As input, the dimensions of each layer of the image are respectively ,in The height of the first layer of the pyramid image. The width of the first layer of the pyramid image. The corresponding number is Layer image, values ​​from 1 to .

3. The method for detecting landmark points in lateral cephalometric radiographs based on hybrid encoders and convolutional embedding as described in claim 1, characterized in that, The Gauss Pyramid After downsampling each layer to a uniform size, the samples are input into the hybrid encoder to obtain feature vectors. and ,Will and Concatenation yields the feature vector ,include: Step 2.1: Determine the size of the Gaussian pyramid. Divided into two types of layers and Each layer of the image is downsampled to a uniform size, with dimensions of [dimensions to be filled in]. and , The number of pyramids is equal to the number of marker points. This represents the number of layers in the pyramid. for The number of layers input into the convolutional encoder; Step 2.2: Convert the original smaller-scale layer... The input is fed into the convolutional layer of the convolutional encoder to obtain the feature map. Its dimension size is ; Step 2.3: Transfer the feature map Max pooling is performed, and then the input is fed into the first convolutional block in the convolutional encoder to obtain the feature map. Its dimension size is ; Step 2.4: Transfer the feature map The input is fed into the second convolutional block in the convolutional encoder to obtain the feature map. Its dimension size is ; Step 2.5: Transfer the feature map The input is fed into the third convolutional block in the convolutional encoder to obtain the feature map. Its dimension size is ; Step 2.6: Transfer the feature map The input is fed into the fourth convolutional block in the convolutional encoder to obtain the feature map. Its dimension size is ; Step 2.7: Transfer the feature map Perform average pooling to obtain the feature vector. Its dimension size is , This indicates that the vector length is equal to the number of channels. ; Step 2.8: Convert the original larger-scale layer... The image patch embedding module of the Swin Transformer encoder is input to obtain the embedding vector. Its dimension size is ,in This indicates that the vector length is equal to the feature map. Medium to high and width The product of This indicates that the vector length is equal to the number of channels. ; Step 2.9: Embed the vector The input is fed into the first layer of the Swing Transformer encoder to obtain the feature vector. Its dimension size is ; Step 2.10: Transfer the feature vector The input is fed into the second coding block of the Swing Transformer encoder to obtain the feature vector. Its dimension size is ,in This indicates that the vector length is equal to the feature map. Medium to high and width The product of This indicates that the vector length is equal to the number of channels. ; Step 2.11: Transfer the feature vector The feature vector is obtained by inputting it into the third coding block of the Swing Transformer encoder. Its dimension size is ,in This indicates that the vector length is equal to the feature map. Medium to high and width The product of This indicates that the vector length is equal to the number of channels. ; Step 2.12: Transfer the feature vector The feature vector is obtained by inputting it into the fourth coding block of the Swing Transformer encoder. Its dimension size is ,in This indicates that the vector length is equal to the feature map. Medium to high and width The product of This indicates that the vector length is equal to the number of channels. ; Step 2.13: Exchange feature vectors The third and fourth dimensions are then used, and average pooling is applied to the fourth dimension to obtain the feature vector. Its dimension size is ; Step 2.14: Process the feature vectors and Perform the splicing operation on the second dimension to obtain... Its dimension size is ; in, For the number of channels, The height of the image. The width of the image.

4. The method for detecting landmark points in lateral cephalometric radiographs based on hybrid encoders and convolutional embedding as described in claim 1, characterized in that, The obtained feature vector The input is fed into the graph attention module to obtain the feature vector. ,include: Step 4.1: Convert the feature vector By performing fully connected operations on three linear layers Q, K, and V respectively, a vector is obtained. , , All of their dimensions are , where the vector length This represents the number of output nodes for the linear layers Q, K, and V. Step 4.2, for vectors and transposed Perform a dot product operation to obtain the attention matrix. Its dimensions are ; Step 4.3: Apply the attention matrix The input is standardized in a batch processing layer, then activated and normalized using the following formula to obtain the attention heatmap. Its dimensions are ; ； Step 4.4: Create an attention heatmap sum vector Perform a dot product operation to obtain the attention features between each marker point. Its dimensions are ; Step 4.5: Focus on attentional features A fully connected operation is performed using a linear layer R to restore the dimension to . The attention feature vector, and then the attention feature vector and The feature vector is obtained by performing a residual connection operation. Its dimensions are .

5. The method for detecting landmark points in lateral cephalometric radiographs based on a hybrid encoder and convolutional embedding as described in claim 4, characterized in that, The feature vector The input is fed into a three-layer MLP for regression to obtain the relative offset. and according to and Update the position of the marker. to obtain new A Gaussian pyramid centered on the central point is constructed, and multiple iterations are performed to obtain the final positions of each marker. ,include: Step 5.1: Convert the feature vector Input to linear layer Perform a fully connected operation, activate the network using the ReLU function, and then dropout to obtain the feature vector. Its dimensions are , where the vector length For linear layers The number of output nodes; Step 5.2: Convert the feature vector Input to linear layer Perform a fully connected operation, activate the network using the ReLU function, and then dropout to obtain the feature vector. Its dimensions are , where the vector length For linear layers The number of output nodes; Step 5.3: Transfer the feature vector Input to linear layer Perform a fully connected operation and regression to obtain the relative offset. Its dimensions are ; Step 5.4: Update according to the formula below. and with Construct a new multi-scale Gaussian pyramid at the center This is used as the input for the next iteration, and after 10 iterations... The final positions of each marker: ； Where t represents the current iteration number, and its value ranges from 1 to 10.

Images