Hip joint image enhancement processing method based on deep learning

Abstract

The invention provides a hip joint image enhancement processing method based on deep learning, and relates to the field of image enhancement. A hip joint image enhancement model is provided, a bidirectional circulation mechanism and a token shift mechanism are combined to construct a feature extraction module, a global and local dependency relationship of a hip joint image is captured, an incremental fusion strategy is introduced to construct an iterative feature enhancement module, and features of different levels are gradually fused through multi-stage iteration. And learning parameters are designed to dynamically adjust the fusion proportion of new and old features, and a hip joint image enhancement model is trained through a staged training strategy, so that the model not only can learn low-level pixel information, but also can capture high-level structure and semantic features, and the adaptability to different lesion areas of the hip joint is enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of image enhancement, and in particular relates to a hip joint image enhancement processing method based on deep learning. Background Art

[0002] Accurate diagnosis of hip joint diseases is highly dependent on high-quality medical images. However, affected by the resolution of imaging equipment, patient position and complexity of lesion tissue, hip joint images often have problems such as low contrast, blurred edges, and noise interference, which makes it difficult to identify subtle lesions. Although traditional image enhancement methods can improve global contrast, they lack specificity for local anatomical structures, and can easily cause over-enhancement of normal tissue or loss of details in the lesion area. They also rely on manual parameter adjustment and are unable to meet the needs of efficient and automated clinical processing.

[0003] In recent years, deep learning technology, especially methods such as convolutional neural networks and generative adversarial networks, has made remarkable progress in the field of medical image enhancement. However, its application in the field of hip joints still faces significant challenges. Most existing models are based on general convolutional neural network designs, which are insufficient in modeling the complex structures unique to the hip joint and make it difficult to balance the extraction of global dependencies and local details. In addition, most image enhancement methods adopt static feature fusion strategies and lack a dynamic mechanism for progressive optimization of multi-stage features, which results in limited model generalization, especially unstable enhancement effects on lesion areas with large morphological differences.

[0004] To address the above problems, a hip joint image enhancement processing method based on deep learning was proposed. Through multi-scale adaptive contrast enhancement and a bidirectional loop mechanism combined with gradient information and multi-scale context, the bone edge and lesion texture were accurately enhanced while noise interference was suppressed. An iterative incremental fusion module was designed, and learnable parameters were introduced to dynamically adjust the weights of new and old features, thereby achieving progressive enhancement of multi-level features and significantly improving the adaptability to complex lesions. Summary of the invention

[0005] The present invention provides a hip joint image enhancement processing method based on deep learning, aiming to propose a hip joint image enhancement model, combine a bidirectional loop mechanism and a token shift mechanism to construct a feature extraction module, capture the global and local dependencies of the hip joint image, introduce an incremental fusion strategy to construct an iterative feature enhancement module, gradually fuse features of different levels through multi-stage iterations, and design learnable parameters to dynamically adjust the fusion ratio of new and old features. The hip joint image enhancement model is trained through a staged training strategy, so that the model can not only learn low-level pixel information, but also capture high-level structural and semantic features, thereby enhancing the adaptability to different lesion areas of the hip joint.

[0006] The present invention aims to propose a hip joint image enhancement model and provide a hip joint image enhancement processing method based on deep learning, which includes the following steps.

[0007] S1. Construct a hip joint image dataset, collect normal and pathological hip joint images, annotate the region of interest for each hip joint image, and create a hip joint image dataset.

[0008] S2. Preprocess the hip joint image dataset, including multi-scale adaptive local contrast enhancement operation and image registration operation based on transformation invariant features, and divide the preprocessed hip joint image dataset into a training set and a test set.

[0009] S3. A feature extraction module is constructed by combining the bidirectional loop mechanism and the token shift mechanism. The bidirectional loop mechanism captures the global dependencies of hip joint images, and the token shift mechanism is used to enhance the local dependencies of hip joint images.

[0010] S4. Introduce an incremental fusion strategy to construct an iterative feature enhancement module. The incremental fusion strategy gradually fuses hip joint image features at different levels through multi-stage iterations. In each stage, the output features of the previous stage are weightedly fused with the attention features of the current stage, and learnable parameters are designed. The fusion ratio of new and old features of hip joint images is adjusted to gradually optimize the global and local dependencies of hip joint images.

[0011] S5. Construct a hip joint image enhancement model, which includes an input, a feature extraction module, an iterative feature enhancement module and an output.

[0012] S6. Training and testing of hip joint image enhancement model. The model is trained in stages using the preprocessed hip joint image training set. After the training is completed, the preprocessed hip joint image test set is input into the model to obtain enhanced hip joint images.

[0013] Preferably, in step S1, a hip joint image dataset is constructed, and normal and diseased hip joint images are collected, wherein diseased hip joints include hip arthritis, hip fractures and hip dysplasia. The region of interest is annotated for each hip joint image, and the region of interest includes the acetabulum, femoral head and joint space. A hip joint image dataset is produced, which includes the constructed annotated hip joint images and the corresponding annotation files.

[0014] Preferably, in step S2, the hip joint image dataset is preprocessed, and the specific steps are: S21. For the multi-scale adaptive local contrast enhancement operation, the local contrast of the hip joint image is enhanced by multi-scale feature extraction and adaptive mechanism. The specific formula is: ; In the formula, is the position of the hip joint in the input image The pixel value of is in scale The features of the local image block calculated below, and It is a scale The local mean and standard deviation under It is a scale The corresponding weights are used to control the importance of different scales. It is dynamically adjusted according to the difference in pixel values ​​within the local window. is the gradient of the input image at the hip joint, and are adjustment factors that control the intensity of local contrast enhancement and edge enhancement, is a multi-scale set containing different local window sizes; Weight The mathematical model is: ; In the formula, is the pixel value in the local window, is the position of the hip joint in the input image The pixel value of S22. For the image registration operation based on transformation invariant features, the features in the hip joint image that are invariant to various geometric transformations are extracted to accurately match the joint area in the hip joint image, especially for the lesion area with large morphological differences. The specific formula is: ; In the formula, is the transformation function, which represents the transformation from the hip joint image To hip image The geometric transformation relationship of and is the hip joint image to be matched, The hip joint image The feature point locations, is the total number of feature points extracted from the hip joint image, is the Euclidean distance, Among all possible transformation functions , select a transformation function that minimizes the expression.

[0015] Preferably, in step S2, the multi-scale mechanism in the multi-scale adaptive local contrast enhancement operation can capture local features of different sizes of the hip joint image, ensure the clarity of the key areas of the hip joint image, and effectively avoid over-enhancement or under-enhancement by adaptively and dynamically adjusting the contrast and edge enhancement intensity; the image registration operation based on transformation invariant features ensures the alignment of the joint areas between different hip joint images by minimizing the feature point matching error, thereby reducing the error accumulation in subsequent processing; the registration operation reduces the geometric differences between hip joint images, making the hip joint image enhancement model more stable when processing hip joint images of different angles or resolutions.

[0016] Preferably, in step S3, the feature extraction module is constructed by: S31. Input preprocessed hip joint image , , , and 1 are the height, width and channel of the preprocessed hip joint image, respectively. Through a 3×3 convolution operation, Projection to shallow features , , yes The channel will Flatten to a one-dimensional sequence , , is the total number of tokens for hip joint images, Through the multi-directional token shift layer, the local features of the hip joint image are extracted and the context range of each token is expanded to obtain the feature , Through the linear projection layer, we get the receiving matrix , the bond matrix , value matrix , , , , , and are the weights of the linear projection layer; S32. Calculate the global attention through a bidirectional loop mechanism, where the calculation formula of the bidirectional attention is: ; The calculation formula of the cyclic attention is: ; In the formula, is a circular offset operation; The results of bidirectional and recurrent attention are fused by weighted summation to obtain global attention. , the specific calculation formula is: ; In the formula, is a weight hyperparameter used to balance bidirectional and recurrent attention; By gated attention to the receiving matrix Processing is performed to adjust the receiving strength of each token to obtain the weighted features , the specific calculation formula is: ; In the formula, is the weight matrix, is an element-wise multiplication operation.

[0017] Preferably, in step S3, for the feature extraction module, the bidirectional loop mechanism captures the global dependencies between different regions in the hip joint image by calculating the global attention, so that the hip joint image enhancement model can simultaneously consider the relationship between each position and all other positions in the hip joint image, enhance the understanding of the global structure, and perform more stably when processing complex lesion areas; the token shift mechanism expands the contextual scope of each token, i.e., image block, through multi-directional token shifting, and enhances the local features of the hip joint image; the feature extraction module gradually extracts from low-level features to high-level features through shallow convolution and token shift operations, ensuring the comprehensiveness and hierarchy of the hip joint image features. Through the bidirectional loop mechanism and gated attention, the model can dynamically adjust the receiving intensity of each token to ensure that the features of key areas are fully enhanced.

[0018] Preferably, in step S4, the iterative feature enhancement module is constructed by: S41, multi-stage feature initialization, input feature extraction module output features , , generating the initial query through a linear projection layer ,key ,value , , , , , and is the weight matrix; S42, iterative attention calculation and fusion, define the number of iterations as , for each iteration , the calculation formula of attention weight is: ; S43. Design of learnable parameters , , adjust the fusion ratio of new and old features of hip joint images, The specific formula is: ; In the formula, is a learnable scale factor that controls the coupling strength of curvature and entropy, Updated by gradient descent, is the local curvature strength of the hip joint input image, is the local information entropy, is the adaptive threshold, According to the local curvature strength and local information entropy The median of is dynamically adjusted; Learnable Scale Factor The mathematical model is: ; In the formula, is the learning rate, is the total loss of the hip image enhancement model, is the gradient value; Local curvature strength The mathematical model is: ; In the formula, is the curvature of the hip joint input image, is the gradient magnitude of the hip joint input image; Local information entropy The mathematical model is: ; In the formula, So The sliding window centered on is the pixel in the window Grayscale probability distribution of ; Adaptive Threshold The mathematical model is: ; In the formula, is the learning rate, which controls the threshold adjustment step size, is the median calculation; S44, weighted fusion of the output features of the previous stage and the attention features of the current stage, and use Dynamically adjust the fusion ratio. The specific calculation formula is: ; The fused features are transformed nonlinearly. The specific calculation formula is: ; In the formula, is the projection weight matrix; S45. After each iteration, the initial features are connected through skip connections. Add it to the current fusion feature. ,go through After iterations, the final fusion feature is obtained , after a 3×3 convolution operation, the enhanced hip joint image is generated , the specific calculation formula is: ; In the formula, is a convolution operation with a convolution kernel of 3×3. is the preprocessed hip joint input image.

[0019] Preferably, in step S4, for the iterative feature enhancement module, the hip joint image features at different levels are gradually fused through multi-stage iterations, and the fusion ratio of new and old features is dynamically adjusted with learnable parameters, which can significantly improve the detail clarity of the hip joint image and the visibility of the lesion area. Dynamic adjustment of the fusion ratio reduces the excessive dependence of the hip joint image enhancement model on specific features. Through progressive optimization and dynamic fusion strategies, the expressiveness and generalization performance of the hip joint image enhancement model are significantly enhanced, while overfitting is avoided, providing higher quality hip joint enhanced images for clinical diagnosis.

[0020] Preferably, in step S5, for the hip joint image enhancement model, the input layer receives the preprocessed hip joint image training set, enters the feature extraction module, uses a bidirectional loop mechanism and a multi-directional token shift mechanism to capture the global and local dependencies of the hip joint image, enters the iterative feature enhancement module, adopts an incremental fusion strategy, and continuously fuses hip joint image features at different levels through multi-stage iterations, and the output layer outputs the enhanced hip joint image.

[0021] Preferably, in step S6, the specific steps of hip joint image enhancement model training and testing are: S61, the first stage uses pixel loss Training the hip joint image enhancement model, the pixel loss The calculation formula is: ; Where N is the total number of pixels in the hip joint image; S62, the second stage uses perceptual loss Training the hip image enhancement model, the perceptual loss The calculation formula is: ; In the formula, It is The characteristics of the iteration, is the weight coefficient, is the square of the L2 norm in the feature space; The total loss of the hip joint image enhancement model is , , and is the balance weight factor, , ; S63, calculating the quality of the enhanced hip joint image, and using the structural similarity index to evaluate the performance of the model, wherein the calculation formula of the structural similarity index is: ; In the formula, is the preprocessed hip joint image , This is the enhanced hip joint image , yes The average value of pixels, yes The average value of pixels, is an image and The covariance of yes The variance of yes The variance of and is a constant to avoid the denominator being zero.

[0022] Preferably, in step S6, for the hip joint image enhancement model, a staged training strategy is used through the joint optimization of pixel loss and perceptual loss, which can significantly improve the performance of the hip joint image model. In the first stage, pixel loss is used to ensure that the hip joint enhanced image is highly consistent with the true label at the pixel level, thereby improving the overall clarity. In the second stage, perceptual loss is used to optimize high-level semantic features, enhance detail retention capabilities, and avoid excessive smoothing or distortion. This method not only improves the training efficiency and stability of the hip joint image enhancement model, but also significantly improves the visual quality and diagnostic value of the hip joint enhanced image, and is particularly suitable for precise enhancement of complex lesion areas.

[0023] Compared with the prior art, the present invention has the following technical effects: The technical solution provided by the present invention proposes a hip joint image enhancement model, combines a bidirectional loop mechanism and a token shift mechanism to construct a feature extraction module, captures the global and local dependencies of the hip joint image, and significantly improves the model's feature expression capabilities for complex anatomical structures and lesion areas; introduces an incremental fusion strategy to construct an iterative feature enhancement module, gradually fuses features of different levels through multi-stage iterations, and designs learnable parameters to dynamically adjust the fusion ratio of new and old features, thereby enhancing the flexibility and adaptability of the model and avoiding overfitting; trains the model through a phased training strategy, so that the model can not only learn low-level pixel information, but also capture high-level structural and semantic features, significantly improving the detail clarity and diagnostic value of the hip joint enhanced image, and enhancing the adaptability to different lesion areas of the hip joint. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flowchart of hip joint image data processing provided by the present invention.

[0025] Figure 2 It is a structural diagram of the feature extraction module provided by the present invention.

[0026] Figure 3 It is a structural diagram of the iterative feature enhancement module provided by the present invention.

[0027] Figure 4 This is a rendering of the hip joint image before enhancement provided by the present invention.

[0028] Figure 5 This is an effect diagram of the hip joint image enhancement provided by the present invention. DETAILED DESCRIPTION

[0029] The present invention aims to propose a hip joint image enhancement processing method based on deep learning, proposes a hip joint image enhancement model, combines a bidirectional loop mechanism and a token shift mechanism to construct a feature extraction module, captures the global and local dependencies of the hip joint image, and significantly improves the model's feature expression ability for complex anatomical structures and lesion areas; introduces an incremental fusion strategy to construct an iterative feature enhancement module, gradually fuses features of different levels through multi-stage iterations, and designs learnable parameters to dynamically adjust the fusion ratio of new and old features, thereby enhancing the flexibility and adaptability of the model and avoiding overfitting; trains the model through a phased training strategy, so that the model can not only learn low-level pixel information, but also capture high-level structural and semantic features, significantly improving the detail clarity and diagnostic value of the hip joint enhanced image, and enhancing the adaptability to different lesion areas of the hip joint.

[0030] See also Figure 1 As shown, a hip joint image enhancement processing method based on deep learning in an embodiment of the present application.

[0031] S1. Construct a hip joint image dataset, collect normal and pathological hip joint images, annotate the region of interest for each hip joint image, and create a hip joint image dataset.

[0032] Furthermore, in step S1, a hip joint image dataset is constructed, and normal and diseased hip joint images are collected from hospitals, medical imaging databases and public datasets, where diseased hip joints include hip arthritis, hip fractures and hip dysplasia. The professional medical image annotation tool LabelMe is used to annotate the region of interest of each hip joint image, including the acetabulum, femoral head and joint space. The annotation results are saved in the XML standard format and correspond to the original hip joint images to produce a hip joint image dataset, which includes the constructed hip joint images after annotation and the corresponding annotation files.

[0033] S2. Preprocess the hip joint image dataset, including multi-scale adaptive local contrast enhancement operation and image registration operation based on transformation invariant features, and divide the preprocessed hip joint image dataset into a training set and a test set.

[0034] Furthermore, in step S2, the specific steps for preprocessing the hip joint image dataset are as follows.

[0035] S21. For the multi-scale adaptive local contrast enhancement operation, the local contrast of the hip joint image is enhanced by multi-scale feature extraction and adaptive mechanism. The specific formula is: ; In the formula, is the position of the hip joint in the input image The pixel value of is in scale The features of the local image block calculated below, It is a scale The local mean under the condition, the initial value is set to 0 during the implementation process, and the value range is 0 to 1. It is a scale The standard deviation under the condition is set to 2 as the initial value during implementation, and the value range is 0.5 to 5. It is a scale The corresponding weights are used to control the importance of different scales. The initial value is set to 0.5 during implementation, and the value range is 0 to 1. is the gradient of the input image at the hip joint, It is an adjustment factor that controls the local contrast enhancement. Its value is 0.5 during implementation. It is an adjustment factor that controls the strength of edge enhancement. Its value is 1 during implementation. is a multi-scale set containing different local window sizes. ; Weight The mathematical model is: ; In the formula, is the pixel value in the local window, is the position of the hip joint in the input image The pixel value of During implementation, the local mean The calculation formula is: ; Where M is the total number of pixels in the local window; During implementation, the standard deviation The calculation formula is: ; S22. For the image registration operation based on transformation invariant features, the features in the hip joint image that are invariant to various geometric transformations are extracted to accurately match the joint area in the hip joint image, especially for the lesion area with large morphological differences. The specific formula is: ; In the formula, is the transformation function, which represents the transformation from the hip joint image To hip image The geometric transformation relationship includes translation, rotation and scaling. and is the hip joint image to be matched, The hip joint image feature point locations, using the SIFT algorithm from the image and Extract feature points , is the total number of feature points extracted from the hip joint image, is the Euclidean distance, Among all possible transformation functions , select a transformation function that minimizes the expression.

[0036] S3. A feature extraction module is constructed by combining the bidirectional loop mechanism and the token shift mechanism. The bidirectional loop mechanism captures the global dependencies of hip joint images, and the token shift mechanism is used to enhance the local dependencies of hip joint images.

[0037] Further, in step S3, if Figure 2 As shown, the specific steps of the feature extraction module are as follows.

[0038] S31. Input preprocessed hip joint image , , 512, 512 and 1 are the height, width and channel of the preprocessed hip joint image respectively. Through the 3×3 convolution operation, Projection to shallow features , , 64 is The channel will Flatten to a one-dimensional sequence , , is the total number of tokens for hip joint images, Through the multi-directional token shift layer, the local features of the hip joint image are extracted and the context range of each token is expanded to obtain the feature , Through the linear projection layer, we get the receiving matrix , the bond matrix , value matrix , , , , , and They are the weights of the linear projection layer, and the dimensions are set to ; S32. Calculate the global attention through a bidirectional loop mechanism, where the calculation formula of the bidirectional attention is: ; The calculation formula of the cyclic attention is: ; In the formula, is a circular offset operation; The results of bidirectional and recurrent attention are fused by weighted summation to obtain global attention. , the specific calculation formula is: ; In the formula, is a weight hyperparameter used to balance bidirectional and recurrent attention, and is set to 0.7 in the implementation; By gated attention to the receiving matrix Processing is performed to adjust the receiving strength of each token to obtain the weighted features , the specific calculation formula is: ; In the formula, is a weight matrix, which ranges from 0 to 1 during implementation. is an element-wise multiplication operation.

[0039] S4. Introduce an incremental fusion strategy to construct an iterative feature enhancement module. The incremental fusion strategy gradually fuses hip joint image features at different levels through multi-stage iterations. In each stage, the output features of the previous stage are weightedly fused with the attention features of the current stage, and learnable parameters are designed. The fusion ratio of new and old features of hip joint images is adjusted to gradually optimize the global and local dependencies of hip joint images.

[0040] Further, in step S4, if Figure 3 As shown, the specific steps of the iterative feature enhancement module are as follows.

[0041] S41, multi-stage feature initialization, input feature extraction module output features , , generating the initial query through a linear projection layer ,key ,value , , , , , and is the weight matrix, and the dimensions are set to ; S42, iterative attention calculation and fusion, define the number of iterations as , for each iteration , the calculation formula of attention weight is: ; S43. Design of learnable parameters Adjust the fusion ratio of the new and old features of the hip joint image. The initial value is set to 0.5 during the implementation process, and the value range is 0 to 1. The specific formula is: ; In the formula, is a learnable scale factor that controls the coupling strength between curvature and entropy. The initial value is set to 1 during implementation. is the local curvature strength of the hip joint input image, is the local information entropy, is the adaptive threshold, and the initial value is set to 0 during implementation; Learnable Scale Factor The mathematical model is: ; In the formula, is the learning rate, and the initial value is set to 0.001 during implementation. is the total loss of the hip image enhancement model, is the gradient value; Local curvature strength The mathematical model is: ; In the formula, is the curvature of the hip joint input image, is the gradient magnitude of the hip joint input image; Local information entropy The mathematical model is: ; In the formula, So The sliding window centered on is the pixel in the window Grayscale probability distribution of ; Adaptive Threshold The mathematical model is: ; In the formula, is the learning rate, which controls the threshold adjustment step size. The initial value is set to 0.001 during the implementation. is the median calculation; S44, weighted fusion of the output features of the previous stage and the attention features of the current stage, and use Dynamically adjust the fusion ratio. The specific calculation formula is: ; The fused features are transformed nonlinearly. The specific calculation formula is: ; In the formula, is the projection weight matrix, which ranges from 0 to 1 during implementation; S45. After each iteration, the initial features are connected through skip connections. Add it to the current fusion feature. ,go through After iterations, the final fusion feature is obtained , after a 3×3 convolution operation, the enhanced hip joint image is generated , the specific calculation formula is: ; In the formula, is a convolution operation with a convolution kernel of 3×3. is the preprocessed hip joint input image.

[0042] S5. Construct a hip joint image enhancement model, which includes an input, a feature extraction module, an iterative feature enhancement module and an output.

[0043] Furthermore, in step S5, for the hip joint image enhancement model, the input layer receives the preprocessed hip joint image training set, enters the feature extraction module, uses a bidirectional loop mechanism and a multi-directional token shift mechanism to capture the global and local dependencies of the hip joint image, enters the iterative feature enhancement module, adopts an incremental fusion strategy, and continuously fuses hip joint image features at different levels through multi-stage iterations. The output layer outputs the enhanced hip joint image.

[0044] Furthermore, in step S5, for the hip joint image enhancement model, it is written in Python language, using the Pytorch framework, the optimizer uses SGD, the initial learning rate is set to 0.01, the momentum is set to 0.9, the weight decay is set to 0.0001, the training batch is set to 32, and the number of training rounds is initially set to 50. After the training is completed, the structural similarity index is used to evaluate the performance of the model.

[0045] S6. Training and testing of hip joint image enhancement model. The model is trained in stages using the preprocessed hip joint image training set. After the training is completed, the preprocessed hip joint image test set is input into the model to obtain enhanced hip joint images.

[0046] Furthermore, in step S6, the specific steps of training and testing the hip joint image enhancement model are as follows.

[0047] S61, the first stage uses pixel loss Training the hip joint image enhancement model, the pixel loss The calculation formula is: ; Where N is the total number of pixels in the hip joint image; S62, the second stage uses perceptual loss Training the hip image enhancement model, the perceptual loss The calculation formula is: ; In the formula, It is The characteristics of the iteration, is the weight coefficient, which ranges from 0 to 1 during implementation. is the square of the L2 norm in the feature space; The total loss of the hip joint image enhancement model is , , and is the balance weight factor. The initial value is set to 1, and the value range is 0.5 to 1.5. The initial value is set to 0.1, and the value range is 0.1 to 0.5; S63, calculating the quality of the enhanced hip joint image, and using the structural similarity index to evaluate the performance of the model, wherein the calculation formula of the structural similarity index is: ; In the formula, is the preprocessed hip joint image , This is the enhanced hip joint image , yes The average value of pixels, yes The average value of pixels, during implementation and The value range is from 0 to 1. is an image and The covariance of yes The variance of yes The variance of and is a constant to avoid the denominator being zero. and The value of .

[0048] Furthermore, in step S6, if Figure 4 As shown, Figure 4 The effect of hip joint image enhancement is shown in Figure 2. Figure 5 As shown, Figure 5 The image shows the effect of hip joint image enhancement after being processed by the hip joint image enhancement model.

[0049] The above are only preferred embodiments of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A hip joint image enhancement processing method based on deep learning, characterized in that: The following steps are involved: S1. Construct a hip joint image dataset, collect normal and pathological hip joint images, annotate the region of interest for each hip joint image, and create a hip joint image dataset; S2, preprocessing the hip joint image dataset, including a multi-scale adaptive local contrast enhancement operation and an image registration operation based on transformation invariant features, and dividing the preprocessed hip joint image dataset into a training set and a test set; S3, combining the bidirectional loop mechanism and the token shift mechanism to build a feature extraction module, the bidirectional loop mechanism captures the global dependency of the hip joint image, and the token shift mechanism is used to enhance the local dependency of the hip joint image; S4. Introduce an incremental fusion strategy to construct an iterative feature enhancement module. The incremental fusion strategy gradually fuses hip joint image features at different levels through multi-stage iterations. In each stage, the output features of the previous stage are weightedly fused with the attention features of the current stage, and learnable parameters are designed. Adjust the fusion ratio of new and old features of hip joint images and gradually optimize the global and local dependencies of hip joint images; S5, constructing a hip joint image enhancement model, wherein the hip joint image enhancement model comprises an input, a feature extraction module, an iterative feature enhancement module and an output; S6. Hip joint image enhancement model training and testing. The model is trained in stages using the preprocessed hip joint image training set. After the training is completed, the preprocessed hip joint image test set is input into the model to obtain the enhanced hip joint image.

2. According to the method for hip joint image enhancement based on deep learning in claim 1, it is characterized in that: In step S2, the hip joint image dataset is preprocessed, and the specific steps are as follows: S21. For the multi-scale adaptive local contrast enhancement operation, the local contrast of the hip joint image is enhanced by multi-scale feature extraction and adaptive mechanism. The specific formula is: ； In the formula, is the position of the hip joint in the input image The pixel value of is in scale The features of the local image block calculated below, and It is a scale The local mean and standard deviation under It is a scale The corresponding weights are used to control the importance of different scales. is the gradient of the input image at the hip joint, and are adjustment factors that control the intensity of local contrast enhancement and edge enhancement, is a multi-scale set containing different local window sizes; Weight The mathematical model is: ； In the formula, is the pixel value in the local window, is the position of the hip joint in the input image The pixel value of S22. For the image registration operation based on transformation invariant features, the features in the hip joint image that are invariant to various geometric transformations are extracted to accurately match the joint area in the hip joint image, especially for the lesion area with large morphological differences. The specific formula is: ； In the formula, is the transformation function, which represents the transformation from the hip joint image To hip image The geometric transformation relationship of and is the hip joint image to be matched, The hip joint image The feature point locations, is the total number of feature points extracted from the hip joint image, is the Euclidean distance, Among all possible transformation functions , select a transformation function that minimizes the expression.

3. According to the hip joint image enhancement processing method based on deep learning as described in claim 2, it is characterized in that: In the step S3, the method for constructing the feature extraction module is as follows: S31. Input preprocessed hip joint image , , , and 1 are the height, width and channel of the preprocessed hip joint image, respectively. Through a 3×3 convolution operation, Projection to shallow features , , yes The channel will Flatten to a one-dimensional sequence , , is the total number of tokens for hip joint images, Through the multi-directional token shift layer, the local features of the hip joint image are extracted and the context range of each token is expanded to obtain the feature , Through the linear projection layer, we get the receiving matrix , the bond matrix , value matrix , , , , , and are the weights of the linear projection layer; S32. Calculate the global attention through a bidirectional loop mechanism, where the calculation formula of the bidirectional attention is: ； The calculation formula of the recurrent attention is: ； In the formula, is a circular offset operation; The results of bidirectional and recurrent attention are fused by weighted summation to obtain global attention. , the specific calculation formula is: ； In the formula, is a weight hyperparameter used to balance bidirectional and recurrent attention; By gated attention to the receiving matrix Processing is performed to adjust the receiving strength of each token to obtain the weighted features , the specific calculation formula is: ； In the formula, is the weight matrix, is an element-wise multiplication operation.

4. A hip joint image enhancement processing method based on deep learning according to claim 3, characterized in that: In the step S4, the method for constructing the iterative feature enhancement module is: S41, multi-stage feature initialization, input feature extraction module output features , , generating the initial query through a linear projection layer ,key ,value , , , , , and is the weight matrix; S42, iterative attention calculation and fusion, define the number of iterations as , for each iteration , the calculation formula of attention weight is: ； S43. Design of learnable parameters , , adjust the fusion ratio of new and old features of hip joint images, The specific formula is: ； In the formula, is a learnable scale factor that controls the coupling strength of curvature and entropy, is the local curvature strength of the hip joint input image, is the local information entropy, is the adaptive threshold; Learnable Scale Factor The mathematical model is: ； In the formula, is the learning rate, is the total loss of the hip image enhancement model, is the gradient value; Local curvature strength The mathematical model is: ； In the formula, is the curvature of the hip joint input image, is the gradient magnitude of the hip joint input image; Local information entropy The mathematical model is: ； In the formula, So The sliding window centered on is the pixel in the window Grayscale probability distribution of ; Adaptive Threshold The mathematical model is: ； In the formula, is the learning rate, which controls the threshold adjustment step size, is the median calculation; S44, weighted fusion of the output features of the previous stage and the attention features of the current stage, and use Dynamically adjust the fusion ratio. The specific calculation formula is: ； The fused features are transformed nonlinearly. The specific calculation formula is: ； In the formula, is the projection weight matrix; S45. After each iteration, the initial features are connected through skip connections. Add it to the current fusion feature. ,go through After iterations, the final fusion feature is obtained , after a 3×3 convolution operation, the enhanced hip joint image is generated , the specific calculation formula is: ； In the formula, is a convolution operation with a convolution kernel of 3×3. is the preprocessed hip joint input image.

5. A hip joint image enhancement processing method based on deep learning according to claim 4, characterized in that: In the S5 step, for the hip joint image enhancement model, the input layer receives the preprocessed hip joint image training set, enters the feature extraction module, uses a bidirectional loop mechanism and a multi-directional token shift mechanism to capture the global and local dependencies of the hip joint image, enters the iterative feature enhancement module, adopts an incremental fusion strategy, and continuously fuses hip joint image features at different levels through multi-stage iterations. The output layer outputs the enhanced hip joint image.

6. A hip joint image enhancement processing method based on deep learning according to claim 5, characterized in that: In step S6, the specific steps of training and testing the hip joint image enhancement model are as follows: S61, the first stage uses pixel loss Training the hip joint image enhancement model, the pixel loss The calculation formula is: ； Where N is the total number of pixels in the hip joint image; S62, the second stage uses perceptual loss Training the hip image enhancement model, the perceptual loss The calculation formula is: ； In the formula, It is The characteristics of the iteration, is the weight coefficient, is the square of the L2 norm in the feature space; The total loss of the hip joint image enhancement model is , , and is the balance weight factor, , ; S63, calculating the quality of the enhanced hip joint image, and using the structural similarity index to evaluate the performance of the model, wherein the calculation formula of the structural similarity index is: ； In the formula, is the preprocessed hip joint image , This is the enhanced hip joint image , yes The average value of pixels, yes The average value of pixels, is an image and The covariance of yes The variance of yes The variance of and is a constant to avoid the denominator being zero.

Images