Medical image key point detection method and system based on improved U-shaped network

Abstract

The invention discloses a medical image key point detection method and system based on an improved U-shaped network, and the method comprises the steps: collecting image data, carrying out the standardization, denoising and enhancement, employing a coding module to extract local and global features through convolution and Transform double branches, carrying out the fusion, setting an attention module at a jump connection part, carrying out the processing of a weighted feature, and carrying out the detection of a key point of a medical image. The decoding module outputs key point heat map positioning through deconvolution and multi-scale up-sampling in combination with the features; the system is composed of a data acquisition module, a preprocessing module, a coding module, an attention module, a decoding module, a key point detection module and a feedback display module, image processing to result visualization is achieved, and the detection precision and efficiency are improved. The method has the beneficial effects that the new double-path structure improves the deep-level feature extraction capability and enhances the recognition capability under complex background interference; the attention mechanism is improved, and the detail detection precision is improved; multi-scale feature fusion is carried out, and the attention of target areas of different sizes is optimized; the comprehensive advantages are remarkable, and medical diagnosis can be achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and particularly to a method and system for detecting key points of medical images based on an improved U-shaped network. Background Art

[0002] Medical image analysis, as an important part of modern medical diagnosis, plays an indispensable role in aspects such as early disease diagnosis, surgical planning, and treatment effect evaluation. Among them, the key point detection technology is a crucial link in medical image analysis and is widely used in fields such as brain tumor localization in brain images and fracture detection in the skeletal system.

[0003] Currently, clinically, measuring medical parameters of image key points mainly relies on manual marking by doctors. This process not only consumes a large amount of time and energy, prolongs the diagnosis cycle, but also due to the experience and subjective factors of individual doctors, the marking results are different, reducing the accuracy and consistency of diagnosis, and greatly wasting medical resources. Therefore, the research on automatic key point detection algorithms based on deep learning is particularly urgent.

[0004] However, most of the existing medical image key point detection methods are based on the architecture of traditional convolutional neural networks (CNNs). Such methods have high requirements for the quality of image data and have poor anti-noise ability when facing medical images with problems such as noise and blur. In complex scenarios, traditional CNNs also face problems of insufficient local feature extraction and difficulty in multi-scale information fusion. Due to the presence of blurred tissue boundaries and tissue occlusion in medical images, the features of the occluded part cannot be effectively extracted, seriously affecting the effect of key point detection. In addition, during the feature extraction process, the convolutional network is limited to a local area and only focuses on a small part of the input image, lacking an understanding of the global context information of the image, making it difficult for the model to capture important global information and accurately detect the positions of key points.

[0005] Although some existing methods attempt to introduce an attention mechanism or improve the network structure, they still cannot effectively solve the above problems. For example, in some methods, the design of the attention mechanism is not perfect, and it is unable to adaptively adjust the weights of features at different scales, resulting in a low correct rate of key point detection in areas with more details. And in other methods, the improvement of the network structure does not fully consider the characteristics of medical images, so that the performance of the model is limited when processing complex medical images. Summary of the Invention

[0006] The purpose of the present invention is to provide a medical image key point detection method and system based on an improved U-shaped network, to solve problems such as poor anti-noise ability, insufficient extraction and fusion of local and global features, and imperfect attention mechanism in existing methods, improve the accuracy and efficiency of medical image key point detection, shorten the detection time, realize convenient output on multiple terminals, and provide more reliable technical support for medical diagnosis.

[0007] To achieve the above object, the present invention adopts the following technical means: A medical image key point detection method based on an improved U-shaped network, comprising the following sequential steps: Obtain the image data to be processed, standardize the obtained data, unify the image size and gray level parameters, use a denoising algorithm to remove noise, and use a data augmentation technique to obtain the preprocessed image data; Use the pre-constructed and trained improved U-shaped network to perform feature processing on the preprocessed image data. The U-shaped network includes an encoding module and a decoding module. The encoding module adopts a dual-branch structure of convolution and Transformer. The convolutional branch sequentially extracts local features of the input feature map through multiple Conv Blocks, and the Transformer branch uses the attention mechanism to obtain global features through multiple Transformer Blocks. The two branches achieve the coupling and exchange of features through specific processing; an attention mechanism module is provided at the skip connection between the encoding module and the decoding module. The attention mechanism module extracts features at different levels through three convolutional layers, obtains a weighted feature map through specific processing, and transmits the fused features to the decoding module; The decoding module gradually restores the image resolution through a deconvolution structure, and at the same time receives the weighted features transmitted by the skip connection; perform upsampling with different magnification factors on the features of each layer of the decoding module to achieve multi-scale feature fusion; the final output layer generates a key point heat map through a convolutional operation, and obtains the key point position coordinates through post-processing.

[0008] Preferably, the denoising algorithm includes at least one of Gaussian filtering, wavelet transform or non-local mean filtering.

[0009] Preferably, the two branches achieve the coupling and exchange of features through Reshaping and Interpolating processing and Reshaping and Avgpooling processing.

[0010] Preferably, the Reshaping and Interpolating process is used to adjust the size of the feature map output by the Transformer branch to adapt it to the input dimension of the convolutional branch; the Reshaping and Avgpooling process is used to adjust the feature map structure output by the convolutional branch to adapt it to the attention calculation of the Transformer branch.

[0011] Preferably, the attention mechanism module specifically includes: Three convolutional layers, Conv layer1, Conv layer2, and Conv layer3, which extract features at different levels respectively; Perform Reshape&transpose processing on the output of Conv layer1; Perform Reshape processing on the output of Conv layer2 and multiply the feature maps with the result of Reshape&transpose processing, followed by Softmax processing; Multiply the result of Softmax processing with the Reshape processing result of Conv layer3; After multiplying the results and performing Reshape processing, add them to the original input feature map to obtain a weighted feature map, and transmit the fused features to the decoding module.

[0012] Preferably, Conv layer1, Conv layer2, and Conv layer3 adopt different convolution operations respectively to extract features at different levels.

[0013] A medical image keypoint detection system based on an improved U-net includes the following interconnected modules: Data acquisition module: Connected to a medical imaging device to obtain image data to be processed; Preprocessing module: Receives the data transmitted by the data acquisition module and normalizes, denoises, and enhances the image data; Encoding module: Comprising a dual-path structure of convolution and Transformer, receives the data from the preprocessing module, extracts local features and global features respectively, and realizes the coupling and interchange of the features of the dual-path structure through Reshaping and Interpolating processing and Reshaping and Avgpooling processing; Attention mechanism module: Applies the attention mechanism to the feature maps of different scales transmitted by the encoding module respectively to extract important region features and suppress unimportant region features; Decoding module: Feature splice the important region features extracted by each attention mechanism module with the corresponding deconvolved features respectively. At the same time, perform upsampling by 8 times, 4 times, and 2 times respectively for each layer and then fuse them to achieve multi-scale feature fusion and fine feature restoration, conduct key point detection, generate key point heat maps, and obtain key point position information through post-processing; Key point detection module: Convert the feature map finally obtained by the decoding module into heat maps of multiple key points through convolution operations, and then locate the coordinate positions of each key point through non-maximum suppression; Feedback and display module: Receive the coordinates measured by the key point detection module and calculate relevant medical parameters. Use the parameter measurement system built by PyQt to visually display the positions of the key points and the corresponding parameter values, and output the results to the user.

[0014] Preferably, the data acquisition module supports medical images in DICOM and NIfTI formats.

[0015] Preferably, the denoising algorithm of the preprocessing module adopts an adaptive selection strategy, and automatically selects Gaussian filtering, wavelet transform or non-local mean filtering according to the image type.

[0016] Preferably, the feedback and display module supports multi-terminal output, including computer displays and tablet devices.

[0017] Advantages of the present invention: 1. The present invention innovatively adopts a dual-path structure to enhance the recognition ability in complex scenarios. In the encoding stage of the U-shaped network, a dual-path structure of convolution and Transformer is innovatively adopted. The convolutional network focuses on extracting local features, and the Transformer network is responsible for capturing global features. Through the coupling and exchange of features, the deep fusion of local and global features is achieved. This innovation breaks the limitations of traditional methods in feature extraction, greatly enhancing the model's recognition ability in complex image scenarios such as noise, blur, and tissue occlusion, ensuring accurate extraction of key features in various complex situations.

[0018] 2. The present invention improves the attention mechanism to enhance the detection accuracy of details. At the skip connection of the U-shaped network, an improved attention mechanism is introduced. By adaptively weighting the features in the skip connection, the model can more flexibly focus on important information, reduce redundancy, and improve network performance. When processing image regions with more details, it can significantly improve the correct rate of key point detection, enabling the model to grasp image details more accurately and effectively avoiding detection errors caused by improper feature fusion.

[0019] 3. The multi-scale feature fusion of the present invention optimizes the attention to the target area. In the decoding stage of the U-shaped network, to solve the problem of feature information loss during the upsampling process, a multi-scale feature map fusion strategy based on attention is adopted. After upsampling each layer of feature maps by 8 times, 4 times, and 2 times respectively, they are combined with the attention mechanism to enhance the network's attention to the target areas of interest, greatly improving the ability to extract detailed information, thereby significantly improving the accuracy of key point detection and providing strong support for the accurate analysis of medical images.

[0020] 4. The present invention has significant comprehensive advantages and empowers medical diagnosis. The innovation not only greatly improves the detection accuracy but also significantly shortens the detection time. Clinical experiments show that in various medical image detection scenarios such as brain tumors, bone fractures, and abdominal organs, the detection time is greatly shortened and the detection accuracy is significantly improved. At the same time, the system supports multi-terminal output, is easy to operate, greatly improves the efficiency and accuracy of medical diagnosis, provides a more reliable diagnostic tool for clinicians, and is expected to promote the technological progress in the field of medical image diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0022] Figure 1 It is a flowchart of the method of the present invention. <>

[0023] Figure 2 It is a flowchart of the improved attention mechanism of the present invention.

[0024] Figure 3 It is a system block diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0026] In the description of the present invention, it should be noted that unless otherwise clearly specified and limited, the terms "installation", "connection", and "coupling" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations. Embodiment 1

[0027] As Figure 1-2 shown, a medical image key point detection method based on an improved U-shaped network includes the following ordered steps: Step 1: Data acquisition and preprocessing: Obtain the image data to be processed through a medical imaging device, standardize the obtained data, unify the image size and gray scale parameters, use a denoising algorithm to remove noise, and the denoising algorithm includes at least one of Gaussian filtering, wavelet transform, or non-local mean filtering, and use data augmentation techniques such as rotation, translation, scaling, and flipping to obtain the preprocessed image data; Step 2: Use the pre-constructed and trained improved U-shaped network to perform feature processing on the preprocessed image data: The U-shaped network includes an encoding module and a decoding module. The encoding module adopts a dual-branch structure of convolution and Transformer, where: The convolutional branch sequentially extracts the local features of the input feature map through multiple Conv Blocks; The Transformer branch uses the attention mechanism to obtain global features through multiple Transformer Blocks; The two branches realize the coupling and exchange of features through Reshaping and Interpolating processing and Reshaping and Avgpooling processing. Among them, Reshaping and Interpolating processing is used to adjust the size of the feature map output by the Transformer branch to adapt to the input dimension of the convolutional branch; Reshaping and Avgpooling processing is used to adjust the feature map structure output by the convolutional branch to adapt to the attention calculation of the Transformer branch; An attention mechanism module is provided at the skip connection between the encoding module and the decoding module. The attention mechanism module includes: Three convolutional layers, Conv layer1, Conv layer2, and Conv layer3, extract features at different levels. Among them, Conv layer1, Conv layer2, and Conv layer3 adopt different convolution operations to extract features at different levels; Perform Reshape&transpose processing on the output of Conv layer1; Perform Reshape processing on the output of Conv layer2, multiply the feature maps with the result of Reshape&transpose processing, and perform Softmax processing; Multiply the result of Softmax processing with the Reshape processing result of Conv layer3; After performing Reshape processing on the multiplication result, add it to the original input feature map to obtain a weighted feature map, and transfer the fused features to the decoding module; Step 3: Image reconstruction and key point detection: The decoding module gradually restores the image resolution through a transposed convolution structure and simultaneously receives the weighted features transmitted through skip connections; Perform upsampling by 8 times, 4 times, and 2 times on the features of each layer of the decoding module respectively to achieve multi-scale feature fusion; The final output layer generates a key point heat map through convolution operations, and obtains the key point position coordinates through post-processing.

[0028] As Figure 3 shown, the medical image key point detection system based on the improved U-shaped network includes the following interconnected modules: Data acquisition module: Connected to medical imaging equipment, supports reading of multiple medical image formats such as DICOM and NIfTI, can automatically identify the image format and perform data transmission to obtain the image data to be processed.

[0029] Preprocessing module: Receives the data transmitted by the data acquisition module, adopts denoising algorithms such as Gaussian filtering, wavelet transform, or non-local mean filtering, automatically selects the optimal denoising method according to the image type, and performs denoising, normalization, and data augmentation operations on the image data.

[0030] Encoding module: Includes a dual-path structure of convolution and Transformer, receives the data from the preprocessing module, extracts local features and global features respectively, and realizes the coupling and interchange of the features of the dual-path structure through Reshaping and Interpolating processing and Reshaping andAvgpooling processing.

[0031] Attention mechanism module: Apply the attention mechanism to the feature maps of different scales transmitted by the encoding module respectively, extract the features of important regions, and suppress the features of unimportant regions.

[0032] Decoding module: Concatenate the features of important regions extracted by each attention mechanism module with the corresponding deconvolved features respectively. At the same time, perform upsampling by 8 times, 4 times, and 2 times respectively for each layer and then fuse them to achieve multi-scale feature fusion and fine feature restoration, thereby improving the model's ability to capture image details and the accuracy of feature expression, perform key point detection, generate key point heat maps, perform binary processing on the heat maps using a threshold segmentation algorithm, and obtain the key point position information through post-processing.

[0033] Key point detection module: Convert the feature map finally obtained by the decoding module into heat maps of multiple key points through convolution operations, and then locate the coordinate positions of each key point through non-maximum suppression.

[0034] Feedback and display module: Receive the coordinates measured by the key point detection module and calculate relevant medical parameters, support multi-terminal output, including devices such as computer displays and mobile tablets, use the parameter measurement system built by PyQt to visually display the positions of key points and the corresponding parameter values, and output the results to users.

[0035] Training method and optimization strategy of U-shaped network model Loss function: Adopt a composite loss function that combines heat map loss and regression loss. The heat map loss measures the difference between the heat map predicted by the network and the true heat map. By minimizing this loss, the positioning accuracy of the network is enhanced. The regression loss ensures the accurate prediction of the key point positions.

[0036] Optimization algorithm: Use the Adam optimization algorithm to train the network, utilize the adaptive learning rate adjustment strategy to accelerate the training process and improve the convergence speed. At the same time, adopt the early stopping method to avoid overfitting.

[0037] Data augmentation: Augment the training data through operations such as rotation, translation, scaling, and flipping to increase the generalization ability of the model and ensure good performance on different medical image data. Embodiment 2

[0038] Key point detection of brain tumor images The neurosurgery department of a certain hospital collected 100 cases of MRI images of brain tumors and analyzed them with the detection system of the present invention. The data acquisition module automatically identifies the DICOM format of the MRI images and transmits the data to the preprocessing module. The preprocessing module selects wavelet transform to remove noise according to the image type, and at the same time performs normalization and data augmentation processing on the images.

[0039] The convolutional network branch of the encoding module extracts local features of the tumor, and the Transformer branch obtains global features of the brain image to achieve the coupling and interchange of features. At the skip connection of the U-shaped network, the attention mechanism module extracts features of important regions of the feature map. In the decoding stage, the attention-based multi-scale feature map fusion module enhances the attention to the tumor region. The detection module generates a heatmap of key points, and through threshold segmentation and post-processing, the tumor boundary and key position points are determined.

[0040] When clinicians compared the detection results with the manually marked results, they found that the detection system of the present invention not only shortened the detection time from an average of 30 minutes to 5 minutes, but also improved the detection accuracy from 65% to 85% in the case of blurred tumor boundaries. Example 3

[0041] Key point detection for skeletal fracture images In a trauma center, 80 X-ray images suspected of fractures were collected. The data acquisition module transmitted the image data in NIfTI format to the preprocessing module, which used Gaussian filtering to remove noise and simultaneously performed normalization and data augmentation.

[0042] The two branches of the encoding module extract local and global features respectively and fuse them. The attention mechanism module extracts features of important regions of the feature map at the skip connection. The detection module processes the generated heatmap to accurately locate the fracture position.

[0043] Statistically, the detection accuracy of the detection system of the present invention for fracture positions reaches 90%, which is 20% higher than that of traditional methods. Moreover, the system can complete the detection of a single image within 3 minutes, greatly improving the emergency diagnosis efficiency. Example 4

[0044] Key point detection for abdominal organ images A certain hospital's gastroenterology department collected 120 abdominal CT images for detecting lesions of organs such as the liver and spleen. The data acquisition module transmitted the data after identifying the CT image format, and the preprocessing module used non-local means filtering to remove noise and performed normalization and data augmentation on the images.

[0045] The encoding module fuses local and global features, and the attention mechanism module highlights the features of important regions of the feature map. The detection module determines the key points of the organ boundary and the lesion position.

[0046] Clinical verification results show that the detection accuracy of the detection system of the present invention for organs such as the liver and spleen is stable above 88%. When processing images with partial occlusion of organs, the detection accuracy is 15% higher than that of traditional methods. At the same time, the multi-terminal display function of the system facilitates doctors to view the detection results anytime and anywhere, improving the convenience of diagnosis.

[0047] It is obvious to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, in any aspect, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any reference signs in the claims should not be construed as limiting the claims involved.

[0048] In addition, it should be understood that although this specification is described in terms of embodiments, not every embodiment only contains an independent technical solution. This narrative manner of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A medical image key point detection method based on an improved U-shaped network, characterized in that, It includes the following sequential steps: Obtain the image data to be processed, standardize the obtained data, unify the image size and gray-scale parameters, use a denoising algorithm to remove noise, and apply data augmentation techniques to obtain the preprocessed image data; Use the pre-constructed and trained improved U-shaped network to perform feature processing on the preprocessed image data. The U-shaped network includes an encoding module and a decoding module. The encoding module adopts a dual-branch structure of convolution and Transformer. The convolutional branch sequentially extracts local features of the input feature map through multiple Conv Blocks, and the Transformer branch uses the attention mechanism through multiple TransformerBlocks to obtain global features. The two branches achieve feature coupling and exchange through specific processing; An attention mechanism module is set at the skip connection between the encoding module and the decoding module. This attention mechanism module extracts features at different levels through three convolutional layers, and obtains a weighted feature map through specific processing, and transmits the fused features to the decoding module; The decoding module gradually restores the image resolution through a deconvolution structure, and at the same time receives the weighted features transmitted by the skip connection; Perform upsampling at different magnification factors on the features of each layer of the decoding module to achieve multi-scale feature fusion; The final output layer generates a key-point heat map through a convolution operation, and obtains the key-point position coordinates through post-processing.

2. The medical image key point detection method based on the improved U-shaped network according to claim 1, wherein The denoising algorithm includes at least one of Gaussian filtering, wavelet transform, or non-local mean filtering.

3. The medical image key point detection method based on the improved U-shaped network according to claim 1, characterized in that, The two branches achieve feature coupling and exchange through Reshaping and Interpolating processing and Reshaping and Avgpooling processing.

4. The method for detecting key points of medical images based on the improved U-shaped network according to claim 3, wherein The Reshaping and Interpolating processing is used to adjust the size of the feature map output by the Transformer branch to adapt to the input dimension of the convolutional branch; The Reshaping and Avgpooling processing is used to adjust the feature map structure output by the convolutional branch to adapt to the attention calculation of the Transformer branch.

5. The medical image key point detection method based on the improved U-shaped network according to claim 1, wherein The attention mechanism module specifically includes: Three convolutional layers Conv layer1, Conv layer2, Conv layer3, which extract features at different levels respectively; Perform Reshape & transpose processing on the output of Conv layer1; Perform Reshape processing on the output of Conv layer2 and multiply it with the result of Reshape & transpose processing for the feature map, and perform Softmax processing; Multiply the result of Softmax processing with the Reshape processing result of Conv layer3; After performing Reshape processing on the multiplication result, add it to the original input feature map to obtain a weighted feature map, and transmit the fused features to the decoding module.

6. The medical image key point detection method based on the improved U-shaped network according to claim 5, characterized in that, The Conv layer1, Conv layer2, and Conv layer3 respectively adopt different convolution operations to extract features at different levels.

7. A medical image key point detection system based on an improved U-shaped network, characterized in that, It includes the following interconnected modules: Data acquisition module: Connected to medical imaging equipment to obtain the image data to be processed; Preprocessing module: Receiving the data transmitted by the data acquisition module, standardizing, denoising, and data enhancing the image data; Encoding module: Comprising a dual-path structure of convolution and Transformer, receiving the data from the preprocessing module, respectively extracting local features and global features, and realizing the coupling and interchange of the dual-path structure features through Reshaping and Interpolating processing and Reshaping and Avgpooling processing; Attention mechanism module: Applying the attention mechanism to the feature maps of different scales transmitted by the encoding module respectively to extract the important region features and suppress the unimportant region features; Decoding module: Feature splicing the important region features extracted by each attention mechanism module with the corresponding deconvolved features respectively, and at the same time performing upsampling by 8 times, 4 times, and 2 times respectively for each layer and then fusing them to achieve multi-scale feature fusion and fine feature restoration, performing key point detection, generating key point heat maps, and obtaining the key point position information through post-processing; Key point detection module: Converting the feature map finally obtained by the decoding module into heat maps of multiple key points through convolution operations, and then locating the coordinate positions of each key point through non-maximum suppression; Feedback and display module: Receiving the coordinates measured by the key point detection module and calculating the relevant medical parameters, using the parameter measurement system built by PyQt to visually display the positions of the key points and the corresponding parameter values, and outputting the results to the user.

8. The medical image key point detection system based on the improved U-shaped network according to claim 7, characterized in that, The data acquisition module supports medical images in DICOM and NIfTI formats.

9. The medical image key point detection system based on the improved U-shaped network according to claim 7, characterized in that, The denoising algorithm of the preprocessing module adopts an adaptive selection strategy, automatically selecting Gaussian filtering, wavelet transform, or non-local mean filtering according to the image type.

10. The medical image key point detection system based on the improved U-shaped network according to claim 7, characterized in that, The feedback and display module supports multi-terminal output, including computer displays and tablet devices.

Images