Medical image key point detection method and system based on attention mechanism

Abstract

The invention provides a medical image key point detection method and system based on an attention mechanism in the technical field of computer vision, and the method comprises the steps: S1, building a key point detection model based on an encoder and a decoder, and setting a loss function, an optimizer function and a hyper-parameter of the key point detection model; s2, a large number of historical medical images are acquired to construct a data set; s3, dividing the data set into a training set, a verification set and a test set based on a preset proportion, and sequentially training, verifying and testing the key point detection model through the training set, the verification set and the test set; and S4, deploying the key point detection model passing the test, detecting the key points of the medical image through the deployed key point detection model to obtain a key point detection result, and continuously iterating and optimizing the key point detection model based on the key point detection result. The method has the advantage that the detection precision of the key points of the medical image is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and system for detecting key points in medical images based on an attention mechanism. Background Art

[0002] Medical imaging is dedicated to using various imaging technologies to acquire, process and display the internal structure and function of the human body. These medical images are one of the most important diagnostic and treatment methods in modern medicine, and can provide doctors with detailed information about diseases, abnormalities and organ status, thereby supporting diagnosis, treatment and monitoring of patients' disease progression.

[0003] Medical image applications rely on keypoint identification, which requires detecting and then identifying key points in medical images. Traditionally, keypoint detection in medical images relies on visual inspection by doctors, which undoubtedly relies heavily on their expertise and requires significant time and specialized human resources. Consequently, the need for automated keypoint detection in medical images has arisen.

[0004] In recent years, with the rapid development of deep learning technology, it has become a mainstream method for image processing. However, when applied to keypoint detection in medical images, the following problems exist: 1. Unlike natural RGB images, medical images are generally subject to noise and artifacts, and some pixel values ​​are higher than 255. This leads to instability in the image feature extraction process, which in turn affects the accuracy of keypoint detection. 2. Keypoints in medical images are also part of the human body structure, and their texture and morphology are similar to those of the surrounding environment. They are easily confused with similarity features, ultimately affecting the accuracy of locating the region of interest. 3. The relatively fixed presentation format of medical images can lead to poor inductive bias in attention calculation.

[0005] Therefore, how to provide a medical image key point detection method and system based on the attention mechanism to improve the accuracy of medical image key point detection has become a technical problem that needs to be solved urgently. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a medical image key point detection method and system based on the attention mechanism, so as to improve the accuracy of medical image key point detection.

[0007] In a first aspect, the present invention provides a method for detecting key points in medical images based on an attention mechanism, comprising the following steps:

[0008] Step S1: Create a key point detection model based on the encoder, decoder, and prediction output module, and set the loss function, optimizer function, and hyperparameters of the key point detection model;

[0009] The encoder and decoder are jump-connected; the encoder consists of five layers of sequentially connected encoding modules; the encoding module consists of sequentially connected multi-scale cross-attention units and downsampling units; the decoder consists of five layers of sequentially connected decoding modules; the decoding module consists of sequentially connected bilinear interpolation units and convolution units;

[0010] The encoding module is used to extract multi-scale local features and global features from the input medical image and fuse them into fused features. During the feature extraction process, the medical image is downsampled to obtain medical sub-images. The decoding module is used to decode each medical sub-image based on the fused features. During the decoding process, each medical sub-image is upsampled to generate three upsampled feature maps for each key point. The prediction output module is used to output key point detection results including key point heat maps and the number of key points based on each upsampled feature map.

[0011] Step S2: obtaining a large number of historical medical images, preprocessing and annotating each of the historical medical images, and then constructing a data set;

[0012] Step S3: dividing the data set into a training set, a validation set, and a test set based on a preset ratio, and sequentially training, validating, and testing the key point detection model using the training set, validation set, and test set;

[0013] Step S4: deploy the key point detection model that has passed the test, detect the key points of the medical image using the deployed key point detection model to obtain the key point detection results, and continuously iterate and optimize the key point detection model based on the key point detection results.

[0014] Furthermore, in step S1, the multi-scale cross attention unit is composed of a conditional position encoding sub-unit, a multi-scale global attention sub-unit and a convolutional feedforward neural sub-unit;

[0015] The conditional position encoding subunit is used to encode the input medical image through two-dimensional deep convolution to obtain a position code; the multi-scale global attention subunit is used to extract multi-scale local features and global features from the position code; the convolutional feedforward neural subunit is used to fuse the multi-scale local features and global features to obtain a fused feature.

[0016] Furthermore, the step S2 is specifically as follows:

[0017] A large number of historical medical images are obtained, each of the historical medical images is preprocessed including at least image denoising, image enhancement, image cropping, image rotation and image alignment, key points of each of the preprocessed historical medical images are annotated, and a dataset is constructed based on the annotated historical medical images.

[0018] Furthermore, the step S3 is specifically as follows:

[0019] Dividing the dataset into a training set, a validation set, and a test set based on a ratio of 6:2:2, training the key point detection model using the training set, and continuously optimizing the loss function, optimizer function, and hyperparameters during the training process until a preset convergence condition is met;

[0020] The trained key point detection model is verified using the validation set to determine whether the detection accuracy is greater than a preset accuracy threshold. If not, the verification fails, and the training set is expanded to continue training. If so, the verification passes, and:

[0021] The verified key point detection model is tested using the test set to determine whether the confidence is greater than a preset confidence threshold. If not, the test fails and the training set is expanded to continue training; if so, the test passes and training ends.

[0022] Furthermore, the step S4 is specifically as follows:

[0023] Deploy the key point detection model that has passed the test and set an API interface for calling the key point detection model; obtain a real-time medical image to be detected, pre-process the real-time medical image, and then input the key point detection model through the API interface to obtain the key point detection result;

[0024] The key point detection model is continuously iterated and optimized based on the feedback annotation of the key point detection results, a detection log is generated based on the key point detection results, real-time medical images and detection time, and the detection log is encrypted, stored and distributedly backed up.

[0025] In a second aspect, the present invention provides a medical image key point detection system based on an attention mechanism, comprising the following modules:

[0026] A key point detection model creation module is used to create a key point detection model based on the encoder, decoder and prediction output module, and set the loss function, optimizer function and hyperparameters of the key point detection model;

[0027] The encoder and decoder are jump-connected; the encoder consists of five layers of sequentially connected encoding modules; the encoding module consists of sequentially connected multi-scale cross-attention units and downsampling units; the decoder consists of five layers of sequentially connected decoding modules; the decoding module consists of sequentially connected bilinear interpolation units and convolution units;

[0028] The encoding module is used to extract multi-scale local features and global features from the input medical image and fuse them into fused features. During the feature extraction process, the medical image is downsampled to obtain medical sub-images. The decoding module is used to decode each medical sub-image based on the fused features. During the decoding process, each medical sub-image is upsampled to generate three upsampled feature maps for each key point. The prediction output module is used to output key point detection results including key point heat maps and the number of key points based on each upsampled feature map.

[0029] A data set construction module is used to obtain a large number of historical medical images, pre-process and annotate each of the historical medical images, and then construct a data set;

[0030] A key point detection model training module is used to divide the data set into a training set, a validation set, and a test set based on a preset ratio, and train, validate, and test the key point detection model in sequence using the training set, validation set, and test set;

[0031] The medical image key point detection module is used to deploy the key point detection model that has passed the test, detect the key points of the medical image through the deployed key point detection model, obtain the key point detection results, and continuously iterate and optimize the key point detection model based on the key point detection results.

[0032] Furthermore, in the key point detection model creation module, the multi-scale cross attention unit is composed of a conditional position encoding sub-unit, a multi-scale global attention sub-unit and a convolutional feedforward neural sub-unit;

[0033] The conditional position encoding subunit is used to encode the input medical image through two-dimensional deep convolution to obtain a position code; the multi-scale global attention subunit is used to extract multi-scale local features and global features from the position code; the convolutional feedforward neural subunit is used to fuse the multi-scale local features and global features to obtain a fused feature.

[0034] Furthermore, the dataset construction module is specifically used to:

[0035] A large number of historical medical images are obtained, each of the historical medical images is preprocessed including at least image denoising, image enhancement, image cropping, image rotation and image alignment, key points of each of the preprocessed historical medical images are annotated, and a dataset is constructed based on the annotated historical medical images.

[0036] Furthermore, the key point detection model training module is specifically used to:

[0037] Dividing the dataset into a training set, a validation set, and a test set based on a ratio of 6:2:2, training the key point detection model using the training set, and continuously optimizing the loss function, optimizer function, and hyperparameters during the training process until a preset convergence condition is met;

[0038] The trained key point detection model is verified using the validation set to determine whether the detection accuracy is greater than a preset accuracy threshold. If not, the verification fails, and the training set is expanded to continue training. If so, the verification passes, and:

[0039] The verified key point detection model is tested using the test set to determine whether the confidence is greater than a preset confidence threshold. If not, the test fails and the training set is expanded to continue training; if so, the test passes and training ends.

[0040] Furthermore, the medical image key point detection module is specifically used to:

[0041] Deploy the key point detection model that has passed the test and set an API interface for calling the key point detection model; obtain a real-time medical image to be detected, pre-process the real-time medical image, and then input the key point detection model through the API interface to obtain the key point detection result;

[0042] The key point detection model is continuously iterated and optimized based on the feedback annotation of the key point detection results, a detection log is generated based on the key point detection results, real-time medical images and detection time, and the detection log is encrypted, stored and distributedly backed up.

[0043] The advantages of the present invention are:

[0044] 1. Create a key point detection model through the encoder, decoder and prediction output module, set the loss function, optimizer function and hyperparameters of the key point detection model, then obtain a large number of historical medical images for preprocessing and annotation to build a data set, divide the data set into training set, validation set and test set, train, verify and test the key point detection model in turn through the training set, validation set and test set, deploy the key point detection model that passes the test, detect the key points of medical images through the deployed key point detection model, obtain the key point detection results, and continuously iterate and optimize the key point detection model based on the key point detection results; that is, perform medical image key point detection through the pre-trained key point detection model, and the encoder and decoder of the key point detection model are jump-connected to form a U-shaped network structure; the encoder is composed of five layers of encoding modules connected in sequence; the encoding module is composed of multi-scale cross-attention units and downsampling units connected in sequence; the decoder is composed of five layers of decoding modules connected in sequence; decoding The module consists of bilinear interpolation units and convolution units connected in sequence; the encoding module is used to extract multi-scale local features and global features from the input medical image and fuse them into fused features, and downsample the medical image to obtain a medical sub-image during the feature extraction process; the decoding module is used to decode each medical sub-image based on the fused features, upsample each medical sub-image during the decoding process, and generate three upsampled feature maps for each key point; the prediction output module is used to output the key point detection results including the key point heat map and the number of key points based on each upsampled feature map; that is, multi-scale local features and global features are extracted in layers to effectively learn the global information and local information at different scales, stabilize the feature extraction process, reduce the information loss in the downsampling process and the jump connection process, alleviate the poor inductive bias problem during attention calculation, and combine the generation of three upsampled feature maps for each key point and converge them to alleviate the deviation problem in a single upsampled feature map, and ultimately greatly improve the accuracy of medical image key point detection.

[0045] 2. By setting the decoder to consist of five layers of decoding modules connected in sequence, and the decoding module consists of bilinear interpolation units and convolution units connected in sequence, while ensuring the detection accuracy, the model parameters are effectively reduced, which facilitates the lightweight deployment of the key point detection model.

[0046] 3. By adopting multi-scale cross-attention units in the encoder and combining local and global feature extraction, we can effectively capture key point features of different sizes in medical images (such as tiny lesions or large-scale anatomical landmarks) and improve the detection accuracy of multi-scale targets (key points).

[0047] 4. By using skip connections between the encoder and decoder, we can avoid the loss of details during the downsampling process, promote the fusion of underlying high-resolution features and high-level semantic features, and improve the accuracy of key point positioning.

[0048] 5. Position encoding is generated through two-dimensional deep convolution. Compared with traditional fixed position encoding, it can adapt to the structural differences of different medical images (such as organ morphological changes) and enhance the model's perception of complex anatomical positions.

[0049] 6. Through preprocessing operations such as denoising, rotation, and alignment, the diversity and complexity of real medical images are simulated, the model's robustness to noise and deformation is improved, and the risk of overfitting is reduced.

[0050] 7. By dividing the data set into 6:2:2 ratios and setting dual thresholds for accuracy and confidence, we ensure that the model meets clinical reliability requirements during training, validation, and testing. By automatically expanding the training set when validation or testing fails, we achieve data-driven iterative optimization.

[0051] 8. Through the encrypted storage and distributed backup of detection logs, medical privacy data can be effectively protected and post-event traceability can be supported.

[0052] 9. The decoder generates three upsampled feature maps for each key point, and predicts the key point heat map by superimposing multi-scale features to reduce missed detections and false detections. This is especially suitable for scenes with dense key points (such as spinal vertebrae positioning).

[0053] 10. Through multi-scale attention mechanism, dynamic position encoding, strict data optimization process and secure deployment mechanism, the pain points of medical image key point detection such as scale sensitivity, low positioning accuracy and high data privacy risk are solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0055] Figure 1 This is a flowchart of a medical image key point detection method based on an attention mechanism of the present invention.

[0056] Figure 2 It is a structural diagram of a medical image key point detection system based on an attention mechanism of the present invention.

[0057] Figure 3 It is a structural diagram of the key point detection model of the present invention.

[0058] Figure 4 It is a structural diagram of the multi-scale cross-attention unit of the present invention.

[0059] Figure 5It is a structural diagram of the multi-scale global attention sub-unit of the present invention.

[0060] Figure 6 It is a schematic diagram of the key point heat map integration of the present invention. DETAILED DESCRIPTION

[0061] The technical solution in the embodiment of the present application has the following general idea: medical image key point detection is performed through a pre-trained key point detection model, and the encoder and decoder of the key point detection model are jump-connected to form a U-shaped network structure; the encoder is composed of five layers of encoding modules connected in sequence; the decoder is composed of five layers of decoding modules connected in sequence; the encoding module is used to extract multi-scale local features and global features from the input medical image and fuse them into fusion features, and downsample the medical image to obtain a medical sub-image during the feature extraction process; the decoding module is used to decode each medical sub-image based on the fusion features, and each medical sub-image is decoded during the decoding process. Upsampling, generating three upsampling feature maps for each key point; the prediction output module is used to output the key point detection results including the key point heat map and the number of key points based on each upsampling feature map; that is, multi-scale local features and global features are extracted in layers to effectively learn global information and local information at different scales, stabilize the feature extraction process, reduce information loss in the downsampling process and the jump connection process, alleviate the poor inductive bias problem during attention calculation, and combine the generation of three upsampling feature maps for each key point and convergence to alleviate the deviation problem of a single upsampling feature map, thereby improving the accuracy of key point detection in medical images.

[0062] Please refer to Figures 1 to 6 As shown, a preferred embodiment of a medical image key point detection method based on an attention mechanism of the present invention includes the following steps:

[0063] Step S1: Create a key point detection model based on the encoder, decoder, and prediction output module, and set the loss function, optimizer function, and hyperparameters of the key point detection model;

[0064] The encoder and decoder are jump-connected, i.e., U-Net is used as the basic framework of the model; the encoder consists of five layers of sequentially connected encoding modules; the encoding modules consist of sequentially connected multi-scale cross-attention units and downsampling units; the decoder consists of five layers of sequentially connected decoding modules; the decoding modules consist of sequentially connected bilinear interpolation units and convolution units;

[0065] By using skip connections between the encoder and decoder, we can avoid detail loss during downsampling, promote the fusion of underlying high-resolution features and high-level semantic features, and improve the accuracy of key point positioning.

[0066] By adopting multi-scale cross-attention units in the encoder and combining local and global feature extraction, we can effectively capture key point features of different sizes in medical images (such as tiny lesions or large-scale anatomical landmarks) and improve the detection accuracy of multi-scale targets (key points).

[0067] By setting the decoder to consist of five layers of decoding modules connected in sequence, and the decoding module consisting of bilinear interpolation units and convolution units connected in sequence, while ensuring the detection accuracy, the model parameters are effectively reduced, facilitating the lightweight deployment of the key point detection model.

[0068] The encoding module is used to extract multi-scale local features and global features from the input medical image and fuse them into fused features. During the feature extraction process, the medical image is downsampled to obtain medical sub-images. The decoding module is used to decode each medical sub-image based on the fused features. During the decoding process, each medical sub-image is upsampled to generate three upsampled feature maps for each key point. The prediction output module is used to output key point detection results including key point heat maps and the number of key points based on each upsampled feature map.

[0069] The decoder generates three upsampled feature maps for each key point, and predicts the key point heat map by superimposing multi-scale features to reduce missed detections and false detections. This is especially suitable for scenes with dense key points (such as spinal vertebrae positioning).

[0070] The medical image is input into the encoder for layered feature extraction. Each layer combines multi-scale cross-attention to improve the feature extraction capability and simultaneously reduce the image resolution. It is then sent to the decoder at the same layer for upsampling, gradually restoring the size of the medical image, and outputting the predicted key point heat map.

[0071] Step S2: obtaining a large number of historical medical images, preprocessing and annotating each of the historical medical images, and then constructing a data set;

[0072] Step S3: dividing the data set into a training set, a validation set, and a test set based on a preset ratio, and sequentially training, validating, and testing the key point detection model using the training set, validation set, and test set;

[0073] Step S4: deploy the key point detection model that has passed the test, detect the key points of the medical image using the deployed key point detection model to obtain the key point detection results, and continuously iterate and optimize the key point detection model based on the key point detection results.

[0074] Through multi-scale attention mechanism, dynamic position encoding, strict data optimization process and secure deployment mechanism, the pain points of medical image key point detection such as scale sensitivity, low positioning accuracy and high data privacy risk are solved.

[0075] In step S1, the multi-scale cross attention unit is composed of a conditional position encoding sub-unit, a multi-scale global attention sub-unit and a convolutional feedforward neural sub-unit;

[0076] The conditional position encoding subunit is used to encode the input medical image through two-dimensional deep convolution to obtain a position code; the multi-scale global attention subunit is used to extract multi-scale local features and global features from the position code; and the convolutional feedforward neural subunit is used to fuse the multi-scale local features and global features to obtain a fused feature. The multi-scale local features and global features can be derived from the texture or topological structure of the image.

[0077] Position encoding is generated through two-dimensional deep convolution. Compared with traditional fixed position encoding, it can adapt to the structural differences of different medical images (such as organ morphological changes) and enhance the model's perception of complex anatomical positions.

[0078] Although spatial relationships between key points provide key clues to infer their precise locations, and global attention can effectively model these relationships, it also comes with high computational costs; in addition, global attention exhibits weak inductive bias because the features it extracts are too simple, so this paper introduces multi-scale cross-attention units to address these problems.

[0079] The formula of the conditional position encoding subunit is:

[0080] C = X + DWC (X);

[0081] Where X represents the input medical image; DWC() represents a two-dimensional depthwise convolution with a kernel size of 3, a stride of 1, and a zero padding of 1; C represents the positional encoding.

[0082] The multi-scale global attention subunit consists of multi-scale convolution and cross-scale cross attention; multi-scale convolution is used to extract multi-scale local features and reduce the spatial and channel dimensions of the input features. To achieve these goals, depth-wise separable convolution (DWC) with different kernel sizes and stride configurations is first used to reduce the spatial size and enhance the diversity of local features, and then point convolution (PWC) is used to reduce the number of output channels to one-quarter of the input channel dimension. Specifically, the kernel size of DWC is set to 4, 8, and 12, with corresponding strides of 2, 4, and 6; the kernel size and stride of PWC are both set to 1. The process definition of multi-scale convolution is as follows:

[0083] I i =PWC i (DWC i (C));

[0084] Among them, I irepresents the output of PWC; i∈{0,1,2}, represents the kernel size configuration of multi-scale convolution; i=0, i=1 and i=2 correspond to kernel sizes of 4, 8 and 12 respectively.

[0085] Cross-scale attention performs global attention in the spatial dimension. Its structure is similar to a recurrent neural network, because the current computational unit takes the output features of the previous computational unit as its query. In each unit, the keys and values ​​used for attention calculation are derived from convolutions of different scales. Thanks to multi-scale convolution, the computational complexity of global attention is greatly reduced, and the diversity of the generated global features is also greatly enhanced. The process of cross-scale attention is defined as follows:

[0086] K i ,V i =Split(Conv(I i ));

[0087]

[0088] M = C + Concat(Q0, Q1, Q2, Q3);

[0089] Among them, Q i , K i and V i Represent the query, key and value matrices used for attention calculation respectively; Split() represents the split operation along the channel dimension; Conν() represents the standard 1×1 convolution with a stride of 1; Softmax() represents the normalized exponential function; T represents transpose; d represents the feature dimension; Q0 represents the initial query matrix; Concat() represents the splicing operation along the channel dimension; M represents the output of cross-scale cross attention.

[0090] The convolutional feedforward neural subunit fuses the connection features from the multi-scale global attention subunit and consists of three convolutional layers: two 1×1 convolutional layers and one 3×3 DWC layer with a stride of 1. The first 1×1 convolutional layer F1 is used to expand the number of channels, while the second 1×1 convolutional layer F2 is used to restore the number of channels to match the number of input feature channels. The 3×3 DWC layer further enhances the local adaptation of the fused features. The process definition of the convolutional feedforward neural subunit is as follows:

[0091] O=M+F2(F3(GeLU(F1(M)));

[0092] Among them, GeLU() represents the GeLU activation function; O represents the output of the convolutional feedforward neural subunit.

[0093] The process definition of the decoding module is as follows:

[0094] Y out=DoubleConv(concat(Bilinear(X D ),X E ));

[0095] Among them, X D and X E Represent the features of the decoder and encoder respectively; Bilinear() represents bilinear difference; DoubleConv() represents two consecutive 3×3 convolutions with a stride of 1.

[0096] The prediction output module uses a 1×1 convolutional layer to generate three upsampled feature maps for each key point, and calculates three losses based on the output of the three upsampled feature maps to aggregate multiple prediction results to improve detection accuracy; since this method is similar to the integrated learning mechanism, it can also be called an integrated prediction head.

[0097] The step S2 is specifically as follows:

[0098] A large number of historical medical images are obtained, each of the historical medical images is preprocessed including at least image denoising, image enhancement, image cropping, image rotation and image alignment, key points of each of the preprocessed historical medical images are annotated, and a dataset is constructed based on the annotated historical medical images.

[0099] Through preprocessing operations such as denoising, rotation, and alignment, the diversity and complexity of real medical images are simulated, the model's robustness to noise and deformation is improved, and the risk of overfitting is reduced.

[0100] The step S3 is specifically as follows:

[0101] Dividing the dataset into a training set, a validation set, and a test set based on a ratio of 6:2:2, training the key point detection model using the training set, continuously optimizing the loss function, the optimizer function, and the hyperparameters during the training process, and finding a parameter combination that optimizes the model performance of the key point detection model until a preset convergence condition is met;

[0102] The trained key point detection model is verified using the validation set to determine whether the detection accuracy is greater than a preset accuracy threshold. If not, the verification fails, and the training set is expanded to continue training. If so, the verification passes, and:

[0103] The verified key point detection model is tested using the test set to determine whether the confidence is greater than a preset confidence threshold. If not, the test fails and the training set is expanded to continue training; if so, the test passes and training ends.

[0104] By dividing the data set into 6:2:2 ratios and setting dual thresholds for accuracy and confidence, we ensure that the model meets clinical reliability requirements during the training, validation, and testing stages. By automatically expanding the training set when validation or testing fails, we achieve data-driven iterative optimization.

[0105] The step S4 is specifically as follows:

[0106] Deploy the key point detection model that has passed the test and set an API interface for calling the key point detection model; obtain a real-time medical image to be detected, pre-process the real-time medical image, and then input the key point detection model through the API interface to obtain the key point detection result;

[0107] The key point detection model is continuously iterated and optimized based on the feedback annotation of the key point detection results, a detection log is generated based on the key point detection results, real-time medical images and detection time, and the detection log is encrypted, stored and distributedly backed up.

[0108] Through the encrypted storage and distributed backup of detection logs, medical privacy data can be effectively protected and support post-event traceability.

[0109] The encryption of the detection log is specifically as follows:

[0110] After scrambling the R, G, and B components of the real-time medical images in the test log using a preset scrambling rule, the test log is encrypted into primary encrypted data using the RSA algorithm. This primary encrypted data is then split based on a 7:3 ratio and the order of the data is reversed to create secondary encrypted data. This secondary encrypted data is then encrypted into an encrypted log using the 3DES algorithm. This multi-level encryption and transformation of the test log significantly enhances the security of the test log storage.

[0111] A preferred embodiment of a medical image key point detection system based on an attention mechanism of the present invention includes the following modules:

[0112] A key point detection model creation module is used to create a key point detection model based on the encoder, decoder and prediction output module, and set the loss function, optimizer function and hyperparameters of the key point detection model;

[0113] The encoder and decoder are jump-connected, i.e., U-Net is used as the basic framework of the model; the encoder consists of five layers of sequentially connected encoding modules; the encoding modules consist of sequentially connected multi-scale cross-attention units and downsampling units; the decoder consists of five layers of sequentially connected decoding modules; the decoding modules consist of sequentially connected bilinear interpolation units and convolution units;

[0114] By using skip connections between the encoder and decoder, we can avoid detail loss during downsampling, promote the fusion of underlying high-resolution features and high-level semantic features, and improve the accuracy of key point positioning.

[0115] By adopting multi-scale cross-attention units in the encoder and combining local and global feature extraction, we can effectively capture key point features of different sizes in medical images (such as tiny lesions or large-scale anatomical landmarks) and improve the detection accuracy of multi-scale targets (key points).

[0116] By setting the decoder to consist of five layers of decoding modules connected in sequence, and the decoding module consisting of bilinear interpolation units and convolution units connected in sequence, while ensuring the detection accuracy, the model parameters are effectively reduced, facilitating the lightweight deployment of the key point detection model.

[0117] The encoding module is used to extract multi-scale local features and global features from the input medical image and fuse them into fused features. During the feature extraction process, the medical image is downsampled to obtain medical sub-images. The decoding module is used to decode each medical sub-image based on the fused features. During the decoding process, each medical sub-image is upsampled to generate three upsampled feature maps for each key point. The prediction output module is used to output key point detection results including key point heat maps and the number of key points based on each upsampled feature map.

[0118] The decoder generates three upsampled feature maps for each key point, and predicts the key point heat map by superimposing multi-scale features to reduce missed detections and false detections. This is especially suitable for scenes with dense key points (such as spinal vertebrae positioning).

[0119] The medical image is input into the encoder for layered feature extraction. Each layer combines multi-scale cross-attention to improve the feature extraction capability and simultaneously reduce the image resolution. It is then sent to the decoder at the same layer for upsampling, gradually restoring the size of the medical image, and outputting the predicted key point heat map.

[0120] A data set construction module is used to obtain a large number of historical medical images, pre-process and annotate each of the historical medical images, and then construct a data set;

[0121] A key point detection model training module is used to divide the data set into a training set, a validation set, and a test set based on a preset ratio, and train, validate, and test the key point detection model in sequence using the training set, validation set, and test set;

[0122] The medical image key point detection module is used to deploy the key point detection model that has passed the test, detect the key points of the medical image through the deployed key point detection model, obtain the key point detection results, and continuously iterate and optimize the key point detection model based on the key point detection results.

[0123] Through multi-scale attention mechanism, dynamic position encoding, strict data optimization process and secure deployment mechanism, the pain points of medical image key point detection such as scale sensitivity, low positioning accuracy and high data privacy risk are solved.

[0124] In the key point detection model creation module, the multi-scale cross attention unit is composed of a conditional position encoding sub-unit, a multi-scale global attention sub-unit and a convolutional feedforward neural sub-unit;

[0125] The conditional position encoding subunit is used to encode the input medical image through two-dimensional deep convolution to obtain a position code; the multi-scale global attention subunit is used to extract multi-scale local features and global features from the position code; and the convolutional feedforward neural subunit is used to fuse the multi-scale local features and global features to obtain a fused feature. The multi-scale local features and global features can be derived from the texture or topological structure of the image.

[0126] Position encoding is generated through two-dimensional deep convolution. Compared with traditional fixed position encoding, it can adapt to the structural differences of different medical images (such as organ morphological changes) and enhance the model's perception of complex anatomical positions.

[0127] Although spatial relationships between key points provide key clues to infer their precise locations, and global attention can effectively model these relationships, it also comes with high computational costs; in addition, global attention exhibits weak inductive bias because the features it extracts are too simple, so this paper introduces multi-scale cross-attention units to address these problems.

[0128] The formula of the conditional position encoding subunit is:

[0129] C = X + DWC (X);

[0130] Where X represents the input medical image; DWC() represents a two-dimensional depthwise convolution with a kernel size of 3, a stride of 1, and a zero padding of 1; C represents the positional encoding.

[0131] The multi-scale global attention subunit consists of multi-scale convolution and cross-scale cross attention; multi-scale convolution is used to extract multi-scale local features and reduce the spatial and channel dimensions of the input features. To achieve these goals, depth-wise separable convolution (DWC) with different kernel sizes and stride configurations is first used to reduce the spatial size and enhance the diversity of local features, and then point convolution (PWC) is used to reduce the number of output channels to one-quarter of the input channel dimension. Specifically, the kernel size of DWC is set to 4, 8, and 12, with corresponding strides of 2, 4, and 6; the kernel size and stride of PWC are both set to 1. The process definition of multi-scale convolution is as follows:

[0132] I i =PWC i (DWC i (C));

[0133] Among them, I i represents the output of PWC; i∈{0,1,2}, represents the kernel size configuration of multi-scale convolution; i=0, i=1 and i=2 correspond to kernel sizes of 4, 8 and 12 respectively.

[0134] Cross-scale attention performs global attention in the spatial dimension. Its structure is similar to a recurrent neural network, because the current computational unit takes the output features of the previous computational unit as its query. In each unit, the keys and values ​​used for attention calculation are derived from convolutions of different scales. Thanks to multi-scale convolution, the computational complexity of global attention is greatly reduced, and the diversity of the generated global features is also greatly enhanced. The process of cross-scale attention is defined as follows:

[0135] K i ,V i =Split(Conv(I i ));

[0136]

[0137] M = C + Concat(Q0, Q1, Q2, Q3);

[0138] Among them, Q i , K i and V i Represent the query, key, and value matrices used for attention calculation respectively; Split() represents the split operation along the channel dimension; Conv() represents the standard 1×1 convolution with a stride of 1; Softmax() represents the normalized exponential function; T represents transpose; d represents the feature dimension; Q0 represents the initial query matrix; Concat() represents the splicing operation along the channel dimension; M represents the output of cross-scale cross attention.

[0139] The convolutional feedforward neural subunit fuses the connection features from the multi-scale global attention subunit and consists of three convolutional layers: two 1×1 convolutional layers and one 3×3 DWC layer with a stride of 1. The first 1×1 convolutional layer F1 is used to expand the number of channels, while the second 1×1 convolutional layer F2 is used to restore the number of channels to match the number of input feature channels. The 3×3 DWC layer further enhances the local adaptation of the fused features. The process definition of the convolutional feedforward neural subunit is as follows:

[0140] Q = M + F2(F3(GeLU(F1(M)));

[0141] Among them, GeLU() represents the GeLU activation function; O represents the output of the convolutional feedforward neural subunit.

[0142] The process definition of the decoding module is as follows:

[0143] Y out =DoubleConv(ConCat(Bilinear(X D ),X E ));

[0144] Among them, X D and X E Represent the features of the decoder and encoder respectively; Bilinear() represents bilinear difference; DoubleConv() represents two consecutive 3×3 convolutions with a stride of 1.

[0145] The prediction output module uses a 1×1 convolutional layer to generate three upsampled feature maps for each key point, and calculates three losses based on the output of the three upsampled feature maps to aggregate multiple prediction results to improve detection accuracy; since this method is similar to the integrated learning mechanism, it can also be called an integrated prediction head.

[0146] The dataset construction module is specifically used for:

[0147] A large number of historical medical images are obtained, each of the historical medical images is preprocessed including at least image denoising, image enhancement, image cropping, image rotation and image alignment, key points of each of the preprocessed historical medical images are annotated, and a dataset is constructed based on the annotated historical medical images.

[0148] Through preprocessing operations such as denoising, rotation, and alignment, the diversity and complexity of real medical images are simulated, the model's robustness to noise and deformation is improved, and the risk of overfitting is reduced.

[0149] The key point detection model training module is specifically used for:

[0150] Dividing the dataset into a training set, a validation set, and a test set based on a ratio of 6:2:2, training the key point detection model using the training set, continuously optimizing the loss function, the optimizer function, and the hyperparameters during the training process, and finding a parameter combination that optimizes the model performance of the key point detection model until a preset convergence condition is met;

[0151] The trained key point detection model is verified using the validation set to determine whether the detection accuracy is greater than a preset accuracy threshold. If not, the verification fails, and the training set is expanded to continue training. If so, the verification passes, and:

[0152] The verified key point detection model is tested using the test set to determine whether the confidence is greater than a preset confidence threshold. If not, the test fails and the training set is expanded to continue training; if so, the test passes and training ends.

[0153] By dividing the data set into 6:2:2 ratios and setting dual thresholds for accuracy and confidence, we ensure that the model meets clinical reliability requirements during the training, validation, and testing stages. By automatically expanding the training set when validation or testing fails, we achieve data-driven iterative optimization.

[0154] The medical image key point detection module is specifically used for:

[0155] Deploy the key point detection model that has passed the test and set an API interface for calling the key point detection model; obtain a real-time medical image to be detected, pre-process the real-time medical image, and then input the key point detection model through the API interface to obtain the key point detection result;

[0156] The key point detection model is continuously iterated and optimized based on the feedback annotation of the key point detection results, a detection log is generated based on the key point detection results, real-time medical images and detection time, and the detection log is encrypted, stored and distributedly backed up.

[0157] Through the encrypted storage and distributed backup of detection logs, medical privacy data can be effectively protected and support post-event traceability.

[0158] The encryption of the detection log is specifically as follows:

[0159] After scrambling the R, G, and B components of the real-time medical images in the test log using a preset scrambling rule, the test log is encrypted into primary encrypted data using the RSA algorithm. This primary encrypted data is then split based on a 7:3 ratio and the order of the data is reversed to create secondary encrypted data. This secondary encrypted data is then encrypted into an encrypted log using the 3DES algorithm. This multi-level encryption and transformation of the test log significantly enhances the security of the test log storage.

[0160] In summary, the advantages of the present invention are:

[0161] 1. Create a key point detection model through the encoder, decoder and prediction output module, set the loss function, optimizer function and hyperparameters of the key point detection model, then obtain a large number of historical medical images for preprocessing and annotation to build a data set, divide the data set into training set, validation set and test set, train, verify and test the key point detection model in turn through the training set, validation set and test set, deploy the key point detection model that passes the test, detect the key points of medical images through the deployed key point detection model, obtain the key point detection results, and continuously iterate and optimize the key point detection model based on the key point detection results; that is, perform medical image key point detection through the pre-trained key point detection model, and the encoder and decoder of the key point detection model are jump-connected to form a U-shaped network structure; the encoder is composed of five layers of encoding modules connected in sequence; the encoding module is composed of multi-scale cross-attention units and downsampling units connected in sequence; the decoder is composed of five layers of decoding modules connected in sequence; decoding The module consists of bilinear interpolation units and convolution units connected in sequence; the encoding module is used to extract multi-scale local features and global features from the input medical image and fuse them into fused features, and downsample the medical image to obtain a medical sub-image during the feature extraction process; the decoding module is used to decode each medical sub-image based on the fused features, upsample each medical sub-image during the decoding process, and generate three upsampled feature maps for each key point; the prediction output module is used to output the key point detection results including the key point heat map and the number of key points based on each upsampled feature map; that is, multi-scale local features and global features are extracted in layers to effectively learn the global information and local information at different scales, stabilize the feature extraction process, reduce the information loss in the downsampling process and the jump connection process, alleviate the poor inductive bias problem during attention calculation, and combine the generation of three upsampled feature maps for each key point and converge them to alleviate the deviation problem in a single upsampled feature map, and ultimately greatly improve the accuracy of medical image key point detection.

[0162] 2. By setting the decoder to consist of five layers of decoding modules connected in sequence, and the decoding module consists of bilinear interpolation units and convolution units connected in sequence, while ensuring the detection accuracy, the model parameters are effectively reduced, which facilitates the lightweight deployment of the key point detection model.

[0163] 3. By adopting multi-scale cross-attention units in the encoder and combining local and global feature extraction, we can effectively capture key point features of different sizes in medical images (such as tiny lesions or large-scale anatomical landmarks) and improve the detection accuracy of multi-scale targets (key points).

[0164] 4. By using skip connections between the encoder and decoder, we can avoid the loss of details during the downsampling process, promote the fusion of underlying high-resolution features and high-level semantic features, and improve the accuracy of key point positioning.

[0165] 5. Position encoding is generated through two-dimensional deep convolution. Compared with traditional fixed position encoding, it can adapt to the structural differences of different medical images (such as organ morphological changes) and enhance the model's perception of complex anatomical positions.

[0166] 6. Through preprocessing operations such as denoising, rotation, and alignment, the diversity and complexity of real medical images are simulated, the model's robustness to noise and deformation is improved, and the risk of overfitting is reduced.

[0167] 7. By dividing the data set into 6:2:2 ratios and setting dual thresholds for accuracy and confidence, we ensure that the model meets clinical reliability requirements during training, validation, and testing. By automatically expanding the training set when validation or testing fails, we achieve data-driven iterative optimization.

[0168] 8. Through the encrypted storage and distributed backup of detection logs, medical privacy data can be effectively protected and post-event traceability can be supported.

[0169] 9. The decoder generates three upsampled feature maps for each key point, and predicts the key point heat map by superimposing multi-scale features to reduce missed detections and false detections. This is especially suitable for scenes with dense key points (such as spinal vertebrae positioning).

[0170] 10. Through multi-scale attention mechanism, dynamic position encoding, strict data optimization process and secure deployment mechanism, the pain points of medical image key point detection such as scale sensitivity, low positioning accuracy and high data privacy risk are solved.

[0171] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting key points in medical images based on an attention mechanism, characterized by: The steps include: Step S1: Create a key point detection model based on the encoder, decoder, and prediction output module, and set the loss function, optimizer function, and hyperparameters of the key point detection model; The encoder and decoder are jump-connected; the encoder consists of five layers of sequentially connected encoding modules; the encoding module consists of sequentially connected multi-scale cross-attention units and downsampling units; the decoder consists of five layers of sequentially connected decoding modules; the decoding module consists of sequentially connected bilinear interpolation units and convolution units; The encoding module is used to extract multi-scale local features and global features from the input medical image and fuse them into fused features. During the feature extraction process, the medical image is downsampled to obtain medical sub-images. The decoding module is used to decode each medical sub-image based on the fused features. During the decoding process, each medical sub-image is upsampled to generate three upsampled feature maps for each key point. The prediction output module is used to output key point detection results including key point heat maps and the number of key points based on each upsampled feature map. Step S2: obtaining a large number of historical medical images, preprocessing and annotating each of the historical medical images, and then constructing a data set; Step S3: dividing the data set into a training set, a validation set, and a test set based on a preset ratio, and sequentially training, validating, and testing the key point detection model using the training set, validation set, and test set; Step S4: deploy the key point detection model that has passed the test, detect the key points of the medical image using the deployed key point detection model to obtain the key point detection results, and continuously iterate and optimize the key point detection model based on the key point detection results.

2. The method for detecting key points in medical images based on an attention mechanism according to claim 1, wherein: In step S1, the multi-scale cross attention unit is composed of a conditional position encoding sub-unit, a multi-scale global attention sub-unit and a convolutional feedforward neural sub-unit; The conditional position encoding subunit is used to encode the input medical image through two-dimensional depth convolution to obtain a position code; the multi-scale global attention subunit is used to extract multi-scale local features and global features from the position code; The convolutional feedforward neural subunit is used to fuse multi-scale local features and global features to obtain fused features.

3. The method for detecting key points in medical images based on an attention mechanism according to claim 1, wherein: The step S2 is specifically as follows: A large number of historical medical images are obtained, each of the historical medical images is preprocessed including at least image denoising, image enhancement, image cropping, image rotation and image alignment, key points of each of the preprocessed historical medical images are annotated, and a dataset is constructed based on the annotated historical medical images.

4. The method for detecting key points in medical images based on an attention mechanism according to claim 1, wherein: The step S3 is specifically as follows: Dividing the dataset into a training set, a validation set, and a test set based on a ratio of 6:2:2, training the key point detection model using the training set, and continuously optimizing the loss function, optimizer function, and hyperparameters during the training process until a preset convergence condition is met; The trained key point detection model is verified using the validation set to determine whether the detection accuracy is greater than a preset accuracy threshold. If not, the verification fails, and the training set is expanded to continue training. If so, the verification passes, and: The verified key point detection model is tested using the test set to determine whether the confidence is greater than a preset confidence threshold. If not, the test fails and the training set is expanded to continue training; if so, the test passes and training ends.

5. The method for detecting key points in medical images based on an attention mechanism according to claim 1, wherein: The step S4 is specifically as follows: Deploy the key point detection model that has passed the test and set an API interface for calling the key point detection model; obtain a real-time medical image to be detected, pre-process the real-time medical image, and then input the key point detection model through the API interface to obtain the key point detection result; The key point detection model is continuously iterated and optimized based on the feedback annotation of the key point detection results, a detection log is generated based on the key point detection results, real-time medical images and detection time, and the detection log is encrypted, stored and distributedly backed up.

6. A medical image keypoint detection system based on an attention mechanism, characterized by: Includes the following modules: A key point detection model creation module is used to create a key point detection model based on the encoder, decoder and prediction output module, and set the loss function, optimizer function and hyperparameters of the key point detection model; The encoder and decoder are jump-connected; the encoder consists of five layers of sequentially connected encoding modules; the encoding module consists of sequentially connected multi-scale cross-attention units and downsampling units; the decoder consists of five layers of sequentially connected decoding modules; the decoding module consists of sequentially connected bilinear interpolation units and convolution units; The encoding module is used to extract multi-scale local features and global features from the input medical image and fuse them into fused features. During the feature extraction process, the medical image is downsampled to obtain medical sub-images. The decoding module is used to decode each medical sub-image based on the fused features. During the decoding process, each medical sub-image is upsampled to generate three upsampled feature maps for each key point. The prediction output module is used to output key point detection results including key point heat maps and the number of key points based on each upsampled feature map. A data set construction module is used to obtain a large number of historical medical images, pre-process and annotate each of the historical medical images, and then construct a data set; A key point detection model training module is used to divide the data set into a training set, a validation set, and a test set based on a preset ratio, and train, validate, and test the key point detection model in sequence using the training set, validation set, and test set; The medical image key point detection module is used to deploy the key point detection model that has passed the test, detect the key points of the medical image through the deployed key point detection model, obtain the key point detection results, and continuously iterate and optimize the key point detection model based on the key point detection results.

7. The medical image key point detection system based on the attention mechanism according to claim 6, characterized in that: In the key point detection model creation module, the multi-scale cross attention unit is composed of a conditional position encoding sub-unit, a multi-scale global attention sub-unit and a convolutional feedforward neural sub-unit; The conditional position encoding subunit is used to encode the input medical image through two-dimensional depth convolution to obtain a position code; the multi-scale global attention subunit is used to extract multi-scale local features and global features from the position code; The convolutional feedforward neural subunit is used to fuse multi-scale local features and global features to obtain fused features.

8. The medical image key point detection system based on the attention mechanism according to claim 6, characterized in that: The dataset construction module is specifically used for: A large number of historical medical images are obtained, each of the historical medical images is preprocessed including at least image denoising, image enhancement, image cropping, image rotation and image alignment, key points of each of the preprocessed historical medical images are annotated, and a dataset is constructed based on the annotated historical medical images.

9. The medical image key point detection system based on the attention mechanism according to claim 6, characterized in that: The key point detection model training module is specifically used for: Dividing the dataset into a training set, a validation set, and a test set based on a ratio of 6:2:2, training the key point detection model using the training set, and continuously optimizing the loss function, optimizer function, and hyperparameters during the training process until a preset convergence condition is met; The trained key point detection model is verified using the validation set to determine whether the detection accuracy is greater than a preset accuracy threshold. If not, the verification fails, and the training set is expanded to continue training. If so, the verification passes, and: The verified key point detection model is tested using the test set to determine whether the confidence is greater than a preset confidence threshold. If not, the test fails and the training set is expanded to continue training; if so, the test passes and training ends.

10. The medical image key point detection system based on the attention mechanism according to claim 6, characterized in that: The medical image key point detection module is specifically used for: Deploy the key point detection model that has passed the test and set an API interface for calling the key point detection model; obtain a real-time medical image to be detected, pre-process the real-time medical image, and then input the key point detection model through the API interface to obtain the key point detection result; The key point detection model is continuously iterated and optimized based on the feedback annotation of the key point detection results, a detection log is generated based on the key point detection results, real-time medical images and detection time, and the detection log is encrypted, stored and distributedly backed up.

Images