A method for detecting key points in medical images

Abstract

This application relates to the field of medical image processing, and more particularly to a method for keypoint detection in medical images. The method includes: acquiring raw medical image data; augmenting the images to generate a training dataset; constructing a detection model; embedding an improved CBAM attention module after the shallow feature extraction layer of the backbone network of the detection model; the improved CBAM module including a channel attention submodule and a spatial attention submodule; inputting the augmented training dataset into the detection model; optimizing the network parameters; generating a trained keypoint detection model; inputting the medical image to be detected into the trained model; and outputting the keypoint coordinates and segmented regions. This method overcomes the difficulty in identifying small targets and weakly salient regions in medical images, enabling the capture of incisal edges in mouth opening measurement, the extraction of keypoints in neck extension assessment, the locking of cartilage structures in nail-chin distance calculation, and the multi-region segmentation and grading judgment in the improved McLeod grading to be completed within a clinically acceptable timeframe.

Description

Technical Field

[0001] This application relates to the field of medical image processing, and more particularly to a method for detecting key points in medical images. Background Technology

[0002] Computer vision technology is increasingly widely used in medical image processing, especially in the localization and quantitative analysis of key anatomical structures. However, existing neural networks generally suffer from insufficient feature perception capabilities when dealing with challenges unique to medical images, such as noise interference, low contrast, and varying target scales, making it difficult to effectively focus on small targets and weakly salient regions. While traditional segmentation networks (such as UNet) can capture local details, their complex computational architecture leads to low inference efficiency and significant fluctuations in output results, failing to meet the stringent requirements of real-time clinical diagnosis.

[0003] In specific medical scenarios, this deficiency becomes even more apparent:

[0004] Mouth opening measurement requires simultaneous identification of facial geometric features, the position of the incisal edges of the upper and lower incisors, and hand posture information. Existing models lack the ability to collaboratively perceive multi-scale targets, making it difficult to accurately locate the micro-structures of the oral cavity.

[0005] The assessment of neck extension relies on the accurate extraction of key facial points such as the bridge of the nose and the tip of the mandible, as well as the midpoint of the clavicle. Conventional networks are not stable enough in capturing low-contrast anatomical landmarks of the neck.

[0006] The calculation of the thyroid-mental distance requires accurate location of weakly salient areas such as the thyroid cartilage notch and the chin apex. Existing methods are easily affected by soft tissue interference in the neck, leading to positioning deviation.

[0007] The improved McFarland grading requires simultaneous segmentation of the hard palate, soft palate, uvula, and palatopharyngeal arch regions and quantification of their visibility. The traditional two-stage network (detection first, segmentation later) serial processing mechanism introduces significant delays, and the segmentation results fluctuate significantly due to the propagation of initial detection errors.

[0008] Current solutions have significant limitations: while mainstream object detection frameworks (such as the YOLO series) offer advantages in efficient inference, they lack native attention guidance mechanisms tailored to the characteristics of medical images; general attention modules (such as CBAM), although capable of enhancing feature representation, have fixed structures that are not adapted to the multi-scale characteristics of medical images, limiting their ability to focus on lesion edges and minute structures. Therefore, a technical solution that balances real-time performance, robustness, and adaptability to medical scenarios is urgently needed to overcome existing bottlenecks. Summary of the Invention

[0009] To address the aforementioned problems, this application provides a method for detecting key points in medical images. The method provided by this application includes the following steps:

[0010] The raw medical image data is acquired, and the images are augmented using data augmentation techniques to generate a training dataset.

[0011] A detection model is built based on the YOLO network architecture. An improved CBAM attention module is embedded after the shallow feature extraction layer of the backbone network of the detection model. The improved CBAM module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule is used to perform dual pooling aggregation operation, and the spatial attention submodule is used to perform dilated convolution operation. The channel attention submodule is implemented by a deformable two-level attention mechanism and dynamically adjusts the feature map sampling position through learnable offsets to generate channel weights that focus on key regions.

[0012] The enhanced training dataset is input into the detection model to optimize the network parameters and generate the trained keypoint detection model.

[0013] The medical image to be detected is input into the trained model, and the key point coordinates and segmentation regions are output. For the mouth opening detection scenario, the key points of the incisal edges of the upper and lower incisors on the face and the key points of the index and ring fingers of the hand are identified simultaneously, and the ratio of the mouth opening distance to the distance of the three fingers is calculated. For the modified McLaurel grading scenario, the hard palate, soft palate, uvula and palatopharyngeal arch regions are segmented, and the grading results are determined based on the visibility and area ratio of each region.

[0014] The dual-pooling aggregation operation includes simultaneously performing global average pooling and global max pooling on the input feature map, concatenating the two pooling results and inputting them into a two-layer perceptron with shared weights to generate a channel weight vector.

[0015] The dilated convolution operation involves using a convolution kernel with a dilation rate greater than 1 to extract spatial features from the input feature map and generate a spatial weight matrix.

[0016] Specifically, in the channel attention submodule, the two-layer perceptron includes a hidden layer, the number of neurons in the hidden layer is 1 / 16 of the number of input channels, and the ReLU activation function is used for feature nonlinear transformation.

[0017] Specifically, the dilated convolution operation of the spatial attention submodule employs three sets of parallel branches with dilation rates of 6, 12, and 18, respectively. The three sets of output feature maps are then fused through channel concatenation to generate a spatial weight matrix.

[0018] Specifically, the learnable offset is generated through a 3×3 convolutional layer, satisfying the following conditions:

[0019] Δ p k =Con 3×3 ( F )

[0020] Where Δ p k Represents the first feature map k The coordinate offset of each position, Con 3×3 This represents a 3×3 convolution operation. F For the input feature map, the sampling position of the feature map is dynamically distorted by the offset, so that the channel weights are focused on the edge region of the lesion.

[0021] Specifically, the deformable two-level attention mechanism includes a content-aware branch, which generates a feature weight matrix M through 1×1 convolution. c The sampled features from the position offset branch are subjected to a Hadamard product operation to achieve the following formula:

[0022] F out =M c ⊙ Bilinear(F in , p +Δ p ),

[0023] in, F out This represents the reconstructed feature map output by the deformable two-level attention mechanism, ⊙ denotes element-wise multiplication, and Bilinear denotes bilinear interpolation sampling. p This is the original coordinate grid.

[0024] Specifically, the shallow feature extraction layer of the backbone network refers to the end of the C2f module, and the improved CBAM module is directly connected to the output of the C2f module.

[0025] Specifically, in the improved McFarland grading scenario, the determination of grading results based on the visibility and area proportion of each region includes:

[0026] When the visible pixel percentage of the soft palate region is less than 30%, it is judged as invisible. When the area of ​​the uvula region is less than 50 pixels, it is judged as invisible. The classification results of 1-4 levels are determined based on the combination of the visibility of the four regions: hard palate, soft palate, uvula and palatopharyngeal arch.

[0027] Specifically, this also includes scenarios involving the calculation of the neck tilt angle:

[0028] Identify key points on the bridge of the nose and the tip of the chin, based on the angle between the vector of the line connecting these two points and the head's reference axis. i Calculate the tilt angle, where i = arccos(( v 1 v 2) / (| v 1| | v 2|)),v 1 is the reference axis vector. v 2 is the vector of the line connecting the bridge of the nose and the tip of the mandible.

[0029] Specifically, the data augmentation technique is a combination of Mosaic augmentation and Mixup augmentation:

[0030] Mosaic enhancement randomly crops and stitches four images into a single composite image; Mixup enhancement is performed according to... l Weights ∈ [0.2, 0.8] are used to linearly superimpose pixels from two images, where... l It follows a Beta(0.5,0.5) distribution.

[0031] This application has the following technical effects:

[0032] It effectively overcomes the difficulty of identifying small targets and weakly salient regions in medical images. By enhancing feature perception capabilities, it significantly improves the localization accuracy of key anatomical landmarks and greatly optimizes computational efficiency. This enables precise capture of the incisal edges of the upper and lower incisors in mouth opening measurement, stable extraction of key facial and neck points in neck extension assessment, reliable locking of microcartilage structures in nail-chin distance calculation, and multi-region segmentation and grading judgment in modified McLeod grading to be completed within a clinically acceptable real-time range. It provides a robust and efficient solution for complex medical diagnostic scenarios. Attached Figure Description

[0033] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts.

[0034] Figure 1 This is a flowchart of a medical image key point detection method according to an embodiment of this application. Detailed Implementation

[0035] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0036] Keypoint detection in medical images has long been hampered by the inherent limitations of traditional neural networks. Existing models are significantly inadequate in extracting shallow features when faced with noise, low-contrast environments, and multi-scale anatomical structures unique to medical images. This makes it difficult to accurately perceive subtle anatomical landmarks such as tooth incisal edges and cartilage notches. While conventional solutions like UNet segmentation networks can capture local details, their complex encoding and decoding structures introduce enormous computational burdens, often leading to failure in real-time diagnostic scenarios due to inference delays exceeding clinically tolerable thresholds. More concerningly, current two-stage detection frameworks (localization followed by measurement) require sequential execution of face detection, hand recognition, and keypoint regression in tasks such as mouth opening assessment. Errors are amplified at each stage, ultimately leading to systematic biases in clinical indicators such as neck tilt angle or thyromental distance. This deficiency is particularly critical in the modified McElligott grading task—when the visible area of ​​the soft palate or uvula approaches a critical value due to image blurring, traditional networks lack sensitivity to pixel-level subtle changes, easily leading to misclassification and affecting the accuracy of airway risk assessment.

[0037] To overcome the aforementioned bottlenecks, this invention constructs a lightweight detection architecture based on attention enhancement and proposes a keypoint detection method for medical images. The core of this architecture lies in modifying the shallow feature extraction layer of the YOLO backbone network: a customized CBAM attention module is embedded at the output of the C2f module. This module operates collaboratively through dual paths of channel attention and spatial attention. The channel path employs a dual-pooling aggregation strategy, simultaneously extracting global average pooling and global max pooling features. Channel weight vectors are generated through a two-layer perceptron with shared weights, where the number of hidden layer neurons is set to 1 / 16 of the number of input channels to balance expressive power and computational overhead. The spatial path innovatively employs three sets of parallel dilated convolution branches with fixed dilation rates of 6, 12, and 18. This multi-scale receptive field design allows the network to simultaneously capture the microscopic texture of the thyroid cartilage notch and the macroscopic morphology of the uvula. When the dilated convolution dilation rate deviates from the [6, 12, 18] combination, the measurement error of the thyromental distance increases significantly because a single receptive field cannot cover the multi-scale characteristics of the cartilage structure. More importantly, a deformable two-level attention mechanism is introduced into the channel path, which dynamically generates the offset Δ of the feature map sampling position through 3×3 convolution. p k =Con 3×3 ( F This allows the network to adaptively distort the feature sampling grid, concentrating computational resources on key areas such as the incisal edges of the upper and lower incisors that are easily overwhelmed by noise.

[0038] In the specific implementation process, such as Figure 1 As shown, follow these steps:

[0039] The raw medical image data is acquired, and the images are augmented using data augmentation techniques to generate a training dataset.

[0040] A detection model is built based on the YOLO network architecture. An improved CBAM attention module is embedded after the shallow feature extraction layer of the backbone network of the detection model. The improved CBAM module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule is used to perform dual pooling aggregation operation, and the spatial attention submodule is used to perform dilated convolution operation. The channel attention submodule is implemented by a deformable two-level attention mechanism, and the feature map sampling position is dynamically adjusted through learnable offsets to generate channel weights that focus on key regions.

[0041] The enhanced training dataset is input into the detection model, the network parameters are optimized, and the trained keypoint detection model is generated.

[0042] The medical image to be detected is input into the trained model, and the key point coordinates and segmentation regions are output. For the mouth opening detection scenario, the key points of the incisal edges of the upper and lower incisors on the face and the key points of the index and ring fingers of the hand are identified simultaneously, and the ratio of the mouth opening distance to the distance of the three fingers is calculated. For the modified McLaurel grading scenario, the hard palate, soft palate, uvula and palatopharyngeal arch regions are segmented, and the grading results are determined based on the visibility and area ratio of each region.

[0043] The dual-pooling aggregation operation includes simultaneously performing global average pooling and global max pooling on the input feature map, concatenating the two pooling results and inputting them into a two-layer perceptron with shared weights to generate a channel weight vector.

[0044] The dilated convolution operation involves using a convolution kernel with a dilation rate greater than 1 to extract spatial features from the input feature map and generate a spatial weight matrix.

[0045] In this embodiment, specifically, the medical images are first preprocessed. A Mosaic and Mixup hybrid enhancement strategy is adopted: four original images are randomly selected, cropped, and stitched together to form a composite image, while simultaneously... l Weights ∈ [0.2, 0.8] are used to linearly superimpose pixels from two images, where... l The dataset follows a Beta (0.5, 0.5) distribution. This strategy effectively simulates common limb occlusion and optical distortions encountered in clinical imaging while expanding the dataset to several thousand layers. During training, a transfer learning strategy is employed. The backbone network is initialized with pre-trained weights from the COCO dataset. The first 150 layers are frozen and trained for 50 epochs before the entire network is unfrozen. The learning rate is decreased stepwise from 0.01 to 0.001, and the batch size is set to 16 to accommodate GPU memory limitations.

[0046] In this embodiment, the two-layer perceptron in the channel attention submodule contains a hidden layer with the number of neurons being 1 / 16 of the number of input channels, and the ReLU activation function is used for feature nonlinear transformation.

[0047] In this embodiment, the dilated convolution operation of the spatial attention submodule adopts three sets of parallel branches with dilation rates of 6, 12 and 18, respectively. The feature maps output by the three sets are merged by channel splicing to generate a spatial weight matrix.

[0048] In this embodiment, the learnable offset is generated through a 3×3 convolutional layer, specifically satisfying the following:

[0049] Δ p k =Con 3×3 ( F )

[0050] Where Δ p k Represents the first feature map k The coordinate offset of each position, Con 3×3 This represents a 3×3 convolution operation. F For the input feature map, the sampling position of the feature map is dynamically distorted by the offset, so that the channel weights are focused on the edge region of the lesion.

[0051] In this embodiment, the deformable two-level attention mechanism includes a content-aware branch, which generates a feature weight matrix M through 1×1 convolution. c The sampled features from the position offset branch are subjected to a Hadamard product operation to achieve the following formula:

[0052] F out =M c ⊙ Bilinear(F in , p +Δ p ),

[0053] in, F out This represents the reconstructed feature map output by the deformable two-level attention mechanism, ⊙ denotes element-wise multiplication, and Bilinear denotes bilinear interpolation sampling. p This is the original coordinate grid.

[0054] In this embodiment, the shallow feature extraction layer of the backbone network specifically refers to the end of the C2f module, and the improved CBAM module is directly connected to the output of the C2f module.

[0055] In this embodiment, in the improved McFarland grading scenario, determining the grading result based on area proportion includes:

[0056] When the visible pixel percentage of the soft palate region is less than 30%, it is judged as invisible. When the area of ​​the uvula region is less than 50 pixels, it is judged as invisible. The classification results of 1-4 levels are determined based on the combination of the visibility of the four regions: hard palate, soft palate, uvula and palatopharyngeal arch.

[0057] This embodiment also includes a scenario for calculating the neck tilt angle:

[0058] Identify key points on the bridge of the nose and the tip of the chin, and calculate the tilt angle based on the angle θ between the vector connecting these two points and the head reference axis, where θ = arccos((v1) v2) / (|v1| |v2|)), v1 is the reference axis vector, and v2 is the vector of the line connecting the bridge of the nose and the tip of the chin.

[0059] For other detection scenarios, when measuring mouth opening, the network simultaneously outputs the coordinates of the incisal edges of the upper and lower incisors (P1, P2) and the coordinates of the tips of the index and ring fingers (Q1, Q2, Q3), which are then calculated using formula D. oral =|P1-P2| to calculate the oral cavity opening distance, D finger =(|Q1-Q2|+|Q2-Q3|) / 2 to calculate the width of the three fingers, and the final mouth opening ratio R=D oral / D finger .

[0060] During neck tilt assessment, a head orientation vector is constructed using the bridge of the nose (A) and the chin (B). v 2, with reference axis vector v 1. Calculate the included angle. i =arccos(( v 1· v 2) / (| v 1|·| v 2|)).

[0061] In calculating the thyroid-mental distance, multi-branch dilated convolution significantly improves the robustness of recognition of the thyroid cartilage notch.

[0062] In the improved McFarland grading, when the visible pixel ratio of the soft palate region is less than 30%, it is judged as invisible, and when the area of ​​the uvula region is less than 50 pixels, it is considered as missing, providing an accurate basis for the area ratio calculation.

[0063] Obviously, the embodiments described above are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0064] It should be understood that when the terms "first," "second," etc., are used in the claims, description, and drawings of this application, they are only used to distinguish different objects and not to describe a specific order. The terms "comprising" and "including" used in the description and claims of this application indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

Claims

1. A method for detecting key points in medical images, characterized in that, Includes the following steps: The raw medical image data is acquired, and the images are augmented using data augmentation techniques to generate a training dataset. A detection model is built based on the YOLO network architecture. An improved CBAM attention module is embedded after the shallow feature extraction layer of the backbone network of the detection model. The improved CBAM module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule is used to perform dual pooling aggregation operation, and the spatial attention submodule is used to perform dilated convolution operation. The channel attention submodule is implemented by a deformable two-level attention mechanism and dynamically adjusts the feature map sampling position through learnable offsets to generate channel weights that focus on key regions. The enhanced training dataset is input into the detection model to optimize the network parameters and generate the trained keypoint detection model. The medical image to be detected is input into the trained model, and the key point coordinates and segmentation regions are output. For the mouth opening detection scenario, the key points of the incisal edges of the upper and lower incisors on the face and the key points of the index and ring fingers of the hand are identified simultaneously, and the ratio of the mouth opening distance to the distance of the three fingers is calculated. For the modified McLaurel grading scenario, the hard palate, soft palate, uvula and palatopharyngeal arch regions are segmented, and the grading results are determined based on the visibility and area ratio of each region. The dual-pooling aggregation operation includes simultaneously performing global average pooling and global max pooling on the input feature map, concatenating the two pooling results and inputting them into a two-layer perceptron with shared weights to generate a channel weight vector. The dilated convolution operation involves using a convolution kernel with a dilation rate greater than 1 to extract spatial features from the input feature map and generate a spatial weight matrix. The deformable two-level attention mechanism includes a content-aware branch, which generates a feature weight matrix M through 1×1 convolution. c The sampled features from the position offset branch are subjected to a Hadamard product operation to achieve the following formula: F out =M c ⊙ Bilinear(F in , p +D p ), in, F out This represents the reconstructed feature map output by the deformable two-level attention mechanism, ⊙ denotes element-wise multiplication, and Bilinear denotes bilinear interpolation sampling. p This is the original coordinate grid.

2. The medical image key point detection method according to claim 1, characterized in that, In the channel attention submodule, the two-layer perceptron includes a hidden layer. The number of neurons in the hidden layer is 1 / 16 of the number of input channels, and the ReLU activation function is used for feature nonlinear transformation.

3. The medical image key point detection method according to claim 1, characterized in that, The dilated convolution operation of the spatial attention submodule employs three sets of parallel branches with dilation rates of 6, 12, and 18, respectively. The feature maps output by the three sets are then fused through channel concatenation to generate a spatial weight matrix.

4. The medical image key point detection method according to claim 1, characterized in that, The learnable offset is generated through a 3×3 convolutional layer, specifically satisfying the following: Δ p k =With 3×3 ( F ) Where Δ p k Represents the first feature map k The coordinate offset of each position, Con 3×3 This represents a 3×3 convolution operation. F For the input feature map, the sampling position of the feature map is dynamically distorted by the offset, so that the channel weights are focused on the edge region of the lesion.

5. The medical image key point detection method according to claim 1, characterized in that, The shallow feature extraction layer of the backbone network specifically refers to the end of the C2f module, and the improved CBAM module is directly connected to the output of the C2f module.

6. The medical image key point detection method according to claim 1, characterized in that, In the improved McFarland grading scenario, the grading results are determined based on the visibility and area proportion of each region, including: When the visible pixel percentage of the soft palate region is less than 30%, it is judged as invisible. When the area of ​​the uvula region is less than 50 pixels, it is judged as invisible. The classification results of 1-4 levels are determined based on the combination of the visibility of the four regions: hard palate, soft palate, uvula and palatopharyngeal arch.

7. The medical image key point detection method according to claim 1, characterized in that, This also includes scenarios involving neck tilt angle calculation: Identify key points on the bridge of the nose and the tip of the chin, based on the angle between the vector of the line connecting these two points and the head's reference axis. θ Calculate the tilt angle, where θ = arccos(( v 1 v 2) / (| v 1| | v 2|)), v 1 is the reference axis vector. v 2 is the vector of the line connecting the bridge of the nose and the tip of the mandible.

8. The medical image key point detection method according to claim 1, characterized in that, The data augmentation technique is specifically a combination of Mosaic augmentation and Mixup augmentation: Mosaic enhancement randomly crops and stitches four images into a single composite image; Mixup enhancement is performed according to... λ Weights ∈ [0.2, 0.8] are used to linearly superimpose pixels from two images, where... λ It follows a Beta(0.5,0.5) distribution.

Images