Fracture detection method in X-ray image based on progressive alignment and feature fusion network

Abstract

The invention provides a fracture detection method in an X-ray image based on a progressive alignment and feature fusion network, and relates to the technical field of fracture target detection in an X-ray scene. Through combination of ductility of windmill type convolution and background suppression capability of a dual-channel attention mechanism, the problems of insensitivity of a model to long-distance features and attention offset caused by noise interference under an X-ray image are effectively solved. And meanwhile, a content awareness guiding sampling module is designed, so that the loss of pixel-level small fractures in the forward propagation process is made up, and the information recovery capability is improved. A low-rank module is introduced, the modeling capability of the network on linear fracture is enhanced through a unique low-rank decomposition and asymmetric convolution branch fusion mode, and the model parameter quantity and the calculation complexity are reduced while the high semantic expression capability is kept.

Description

Technical Field

[0001] This invention relates to the technical field of fracture target detection in X-ray scenes, and more particularly to a fracture detection method in X-ray images based on progressive alignment and feature fusion networks. Background Technology

[0002] Currently, X-ray fracture detection faces complex and variable imaging conditions. Different equipment parameters, exposure intensities, and imaging angles can lead to significant differences in image contrast, resulting in noise, artifacts, or localized blurring, thus affecting the clarity of the skeletal structure. These factors make deep learning models susceptible to interference during feature extraction and fracture recognition, making it difficult to accurately locate the fracture region. In fracture detection tasks, subtle fracture features are typically weak, with blurred boundaries and are easily affected by noise, causing them to gradually attenuate or even be lost during network forward propagation, severely impacting the sensitivity and accuracy of the detection system. Furthermore, when there is structural overlap or dense distribution in the skeletal region, different tissues may occlude and overlap in grayscale, further increasing the detection difficulty. Therefore, existing X-ray fracture detection methods still suffer from insufficient robustness, low recognition rate of subtle fractures, and poor recall in dense areas under complex imaging environments. Improvements are needed through techniques such as feature enhancement and noise suppression to enhance the accuracy and stability of the detection system. Summary of the Invention

[0003] To address the technical problems mentioned in the background section, this invention provides a fracture detection method for X-ray images based on a progressive alignment and feature fusion network. To comprehensively evaluate the network's detection capabilities for various mask types, this application uses multiple datasets to evaluate the model. In the PAF-Net network, an innovative progressive directional convolution is proposed. By combining the extensibility of windmill-shaped convolution with the background suppression capabilities of a dual-channel attention mechanism, this effectively solves the problems of model insensitivity to long-distance features and attention shift caused by noise interference in X-ray images. Simultaneously, a content-aware guided sampling module is designed to compensate for the loss of pixel-level small fractures during forward propagation, improving information recovery capabilities. A low-rank module is introduced, enhancing the network's modeling ability for linear fractures through a unique low-rank decomposition and asymmetric convolution branch fusion method, reducing the number of model parameters and computational complexity while maintaining high semantic expressiveness.

[0004] The technical means employed in this invention are as follows: A fracture detection method in X-ray images based on progressive alignment and feature fusion networks includes the following steps: Step 1: Obtain X-ray fracture images to construct a dataset, and divide the X-ray fracture image dataset into a training set, a validation set, and a test set according to the proportions; Step 2: Input the X-ray fracture images into the PAF-Net network in batches of 16 images; after processing by the progressively oriented convolutional module in the shallow backbone structure, the enhanced shallow feature map S1 is obtained; Step 3: Input the enhanced shallow feature map S1 obtained in Step 2 into the deep backbone structure of the PAF-Net network. Sort the multi-level features of the backbone according to the semantic hierarchy of the network output to obtain four sets of multi-scale feature maps S2, S3, S4 and S5. Step 4: Input the feature map S5 from step 3 into the content-aware guided sampling module. By predicting the spatial position of each sampling point in the feature map, adaptively resample the fine-grained structure lost due to downsampling in the deep features based on the generated offset, and output the restored high-resolution feature map S6. Step 5: Perform Concat fusion on the high-resolution feature map S6 from step 4 and the feature map S3 from step 3 to obtain the fused feature map F1; Step 6: Input the feature map F1 from step 5 into the low-rank module to extract compact semantic features, and obtain feature map F2; Step 7: The feature map F2 from step 6 is passed through the content-aware guided sampling module. The spatial resolution of the feature map is restored through dynamic offset prediction and adaptive upsampling to obtain the reconstructed high-resolution feature map F3. Step 8: Align the feature map F3 from step 7 with the feature map S2 from step 3 in terms of spatial scale, and then perform feature-level stitching and fusion along the channel dimension to obtain feature map F4; Step 9: Perform semantic compression and feature reconstruction on the feature map F4 from step 8 using a low-rank module to obtain feature map F5; Step 10: Input the feature maps S4 and S5 from step 3 into the feature extraction module composed of conventional convolutional layers and C2f structure, and fuse and enhance the two sets of feature maps to obtain two sets of enhanced multi-scale fused feature maps F6 and F7. Step S11: Input the F5, F6, and F7 feature maps obtained in steps 9 and 10 into the detection head; perform predictions on the feature maps at each scale, and finally output the fracture detection results through multi-scale parallel inference.

[0005] Furthermore, the progressive directional convolution module is used to calculate the channel importance distribution and spatial response distribution of the input features, and its calculation process is as follows: ; in, This represents the feature map after windmill convolution; the windmill convolution is a directional convolution operation that splits the convolution kernel in different directions and executes them in parallel. This indicates a dual-channel attention module; This represents the final output of the progressive directional convolution module; DSAM is used to perform bi-branch attention weighting on the input feature map. Its calculation method involves generating channel weight coefficients and spatial weight coefficients based on the input feature map, and then applying these two types of attention weights to the original feature map to obtain the enhanced attention feature map. ; in, and These represent the weight factors corresponding to the channel branch and the spatial branch, respectively. Indicates the input feature map, This represents a submodule that performs feature description and weight calculation on the input feature map along the channel dimension. This represents a submodule that performs feature description and weight calculation on the input feature map in the spatial dimension.

[0006] Furthermore, the calculation method of the content-aware guided sampling module is as follows: adaptively calculate the pixel sampling offset and sampling weight coefficient based on the input feature map, and perform content-aware resampling of the original feature map based on the sampling offset and sampling weight to generate enhanced resampled features: ; ; ; ; ; Where X represents the input feature map of the previous layer. This indicates that the input feature map is upsampled by a factor of s. This represents the result after upsampling the input feature map. This represents the offset field predicted from the original feature map. This represents the result after upsampling the input prediction offset field. This represents the sigmoid function. This represents the normalized offset field. Represents the basic sampling network. This represents the adaptive sampling network resulting from the fusion of the base sampling network and the normalized offset field. GridSample represents the content-aware sampling operation performed on the input feature map using bilinear interpolation based on the sampling grid coordinates. This represents the output feature map after adaptive offset guided sampling.

[0007] Furthermore, the PAF-Net includes a Backbone network and a Head detection layer; wherein, the shallow structure of the Backbone includes two high-resolution feature extraction stages, P1 and P2, and a progressive directional convolution module is embedded in each stage.

[0008] Furthermore, the deep backbone structure includes three feature extraction stages, P3, P4, and P5, connected sequentially in depth.

[0009] Furthermore, the detection head includes: a convolutional branch for bounding box regression, a confidence branch for target existence prediction, and a classification branch for fracture category identification.

[0010] Compared with the prior art, the present invention has the following advantages: 1. To address the issues of attention bias and incomplete region modeling in the skeletal region caused by background noise interference in X-ray images, we introduce progressive directional convolution (PDC) based on windmill convolution and dual-channel attention mechanism to filter image noise under X-ray and expand the skeletal modeling region.

[0011] 2. To address the issue of information loss during forward propagation, we designed Content-Aware Guided Sampling (CAGS) to promptly compensate for the loss of information and improve the ability to recover and reconstruct information.

[0012] 3. To address the computational redundancy issue of conventional C2f modules, we designed a low-rank module (LR) to reduce redundant computations in the model, while also improving the modeling capability for enhanced features of linear fractures. Based on the above reasons, this invention has been widely promoted in the field of fracture detection, and is especially suitable for complex scenarios where the target is difficult to detect. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the model of the present invention.

[0015] Figure 2 This is a comparison chart showing the effectiveness of the method of the present invention with other methods for detecting fractures in X-ray images. Detailed Implementation

[0016] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0018] like Figure 1 As shown, this invention provides a fracture detection method in X-ray images based on a progressive alignment and feature fusion network, comprising the following steps: Step 1: Obtain X-ray fracture images to construct a dataset, and divide the X-ray fracture image dataset into a training set, a validation set, and a test set according to the proportions; Step 2: Input the X-ray fracture images into the PAF-Net network in batches of 16 images each; after processing by the progressively oriented convolutional module in the shallow backbone structure, the enhanced shallow feature map S1 is obtained. The PAF-Net consists of a backbone network and a head detection layer. The shallow structure of the backbone includes two high-resolution feature extraction stages, P1 and P2, and embeds progressively oriented convolutional modules within them.

[0019] Step 3: Input the enhanced shallow feature map S1 obtained in Step 2 into the deep backbone structure of the PAF-Net network. The deep backbone includes three feature extraction stages, P3, P4 and P5, which are connected in sequence according to depth. The multi-level features of the backbone are sorted according to the semantic level output by the network to obtain four sets of multi-scale feature maps S2, S3, S4 and S5.

[0020] Step 4: Input the feature map S5 from Step 3 into the content-aware guided sampling module. By predicting the spatial position of each sampling point in the feature map, the module adaptively resamples the fine-grained structures lost due to downsampling in the deep features using the generated offset, and outputs the restored high-resolution feature map S6. The calculation method of the content-aware guided sampling module is as follows: adaptively calculate the pixel sampling offset and sampling weight coefficient according to the input feature map, and perform content-aware resampling of the original feature map according to the sampling offset and sampling weight to generate enhanced resampled features.

[0021] ; ; ; ; ; Where X represents the input feature map of the previous layer. This indicates that the input feature map is upsampled by a factor of s. This represents the result after upsampling the input feature map. This represents the offset field predicted from the original feature map. This represents the result after upsampling the input prediction offset field. This represents the sigmoid function. This represents the normalized offset field. Represents the basic sampling network. This represents the adaptive sampling network resulting from the fusion of the base sampling network and the normalized offset field. GridSample represents the content-aware sampling operation performed on the input feature map using bilinear interpolation based on the sampling grid coordinates. This represents the output feature map after adaptive offset guided sampling.

[0022] Step 5: Perform Concat fusion on the high-resolution feature map S6 from step 4 and the feature map S3 from step 3 to obtain the fused feature map F1; Step 6: Input the feature map F1 from step 5 into the low-rank module to extract compact semantic features, and obtain feature map F2; Step 7: The feature map F2 from step 6 is passed through the content-aware guided sampling module. The spatial resolution of the feature map is restored through dynamic offset prediction and adaptive upsampling to obtain the reconstructed high-resolution feature map F3. Step 8: Align the feature map F3 from step 7 with the feature map S2 from step 3 in terms of spatial scale, and then perform feature-level stitching and fusion along the channel dimension to obtain feature map F4; Step 9: Perform semantic compression and feature reconstruction on the feature map F4 from step 8 using the low-rank module to obtain feature map F5; Step 10: Input the feature maps S4 and S5 from step 3 into the feature extraction module, which consists of a conventional convolutional layer and a C2f structure, and fuse the two sets of feature maps and perform further feature enhancement to obtain two sets of enhanced multi-scale fused feature maps F6 and F7. Step S11: Input the F5, F6, and F7 feature maps obtained in steps 9 and 10 into the detection head; the detection head includes a convolution branch for bounding box regression, a confidence branch for target existence prediction, and a classification branch for fracture category identification, and performs predictions on feature maps at each scale, and finally outputs fracture detection results through multi-scale parallel inference.

[0023] Preferably, the progressive directional convolution module is used to calculate the channel importance distribution and spatial response distribution of the input features, and the calculation process is as follows: ; in, This represents the feature map after windmill convolution; the windmill convolution is a directional convolution operation that splits the convolution kernel in different directions and executes them in parallel. (Two-branch attention mechanism) refers to a two-channel attention module; This represents the final output of PDC (Progressive Directional Convolution). DSAM (Dual-Branch Attention Mechanism) is used to apply dual-branch attention weights to the input feature map. Its calculation method involves generating channel weight coefficients and spatial weight coefficients based on the input feature map, and then applying these two types of attention weights to the original feature map to obtain the enhanced attention feature map. ; in, and These represent the weight factors corresponding to the channel branch and the spatial branch, respectively. Indicates the input feature map, This refers to the submodule that performs feature description and weight calculation on the channel dimension of the input feature map. This refers to the submodule that performs feature description and weight calculation on the input feature map in the spatial dimension.

[0024] Example 1 like Figure 2As shown in the diagram, this invention provides a comparison with other algorithms for fracture target detection. The experimental results demonstrate that, despite interference from different scenes and lighting conditions, this invention can accurately capture and classify targets. This invention exhibits higher accuracy and recall. Furthermore, it significantly outperforms other networks in capturing and recalling small and blurry targets. Based on these observations, we conclude that, in complex contexts, this invention possesses stronger target capture and classification capabilities, demonstrating higher accuracy and comprehensiveness. This is significant in the field of deep learning object detection because it can better handle complex real-world situations, providing a more reliable and stable solution for fracture target detection tasks.

[0025] This embodiment compares the experimental results of different algorithms using two objective metrics: AP and AP50. AP represents average precision, an indicator of model performance obtained by maintaining a balance between precision and recall at different confidence thresholds. AP50 represents the average precision at a confidence threshold of 50%. This invention uses a network based on progressive alignment and feature fusion, employs PDC to suppress background noise and expand the modeling region, uses CAGS to suppress the loss of weak information, and utilizes LR to compress semantic information and enhance the ability to model linear cracks. Therefore, this invention shows significant improvements in AP and AP50 metrics compared to the latest algorithms, as shown in Table 1. The method in this application outperforms other object detection algorithms.

[0026] Table 1 Comparison data of each model

[0027] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0028] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0029] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0030] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0031] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0032] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0033] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fracture detection method in X-ray images based on progressive alignment and feature fusion networks, characterized in that, Includes the following steps: Step 1: Obtain X-ray fracture images to construct a dataset, and divide the X-ray fracture image dataset into a training set, a validation set, and a test set according to the proportions; Step 2: Input the X-ray fracture images into the PAF-Net network in batches of 16 images; After processing by the progressive directional convolution module in the shallow backbone structure, the enhanced shallow feature map S1 is obtained; Step 3: Input the enhanced shallow feature map S1 obtained in Step 2 into the deep backbone structure of the PAF-Net network. Sort the multi-level features of the backbone according to the semantic hierarchy of the network output to obtain four sets of multi-scale feature maps S2, S3, S4 and S5. Step 4: Input the feature map S5 from step 3 into the content-aware guided sampling module. By predicting the spatial position of each sampling point in the feature map, adaptively resample the fine-grained structure lost due to downsampling in the deep features based on the generated offset, and output the restored high-resolution feature map S6. Step 5: Perform Concat fusion on the high-resolution feature map S6 from step 4 and the feature map S3 from step 3 to obtain the fused feature map F1; Step 6: Input the feature map F1 from step 5 into the low-rank module to extract compact semantic features, and obtain feature map F2; Step 7: The feature map F2 from step 6 is passed through the content-aware guided sampling module. The spatial resolution of the feature map is restored through dynamic offset prediction and adaptive upsampling to obtain the reconstructed high-resolution feature map F3. Step 8: Align the feature map F3 from step 7 with the feature map S2 from step 3 in terms of spatial scale, and then perform feature-level stitching and fusion along the channel dimension to obtain feature map F4; Step 9: Perform semantic compression and feature reconstruction on the feature map F4 from step 8 using a low-rank module to obtain feature map F5; Step 10: Input the feature maps S4 and S5 from step 3 into the feature extraction module composed of conventional convolutional layers and C2f structure, and fuse and enhance the two sets of feature maps to obtain two sets of enhanced multi-scale fused feature maps F6 and F7. Step S11: Input the F5, F6, and F7 feature maps obtained in steps 9 and 10 into the detection head; perform predictions on the feature maps at each scale, and finally output the fracture detection results through multi-scale parallel inference.

2. The fracture detection method in X-ray images based on progressive alignment and feature fusion networks according to claim 1, characterized in that, The progressive directional convolution module is used to calculate the channel importance distribution and spatial response distribution of the input features. The calculation process is as follows: ； in, This represents the feature map after windmill convolution; This indicates a dual-channel attention module; This represents the final output of the progressive directional convolution module; DSAM is used to perform bi-branch attention weighting on the input feature map. Its calculation method involves generating channel weight coefficients and spatial weight coefficients based on the input feature map, and then applying these two types of attention weights to the original feature map to obtain the enhanced attention feature map. ； in, and These represent the weight factors corresponding to the channel branch and the spatial branch, respectively. Indicates the input feature map, This represents a submodule that performs feature description and weight calculation on the input feature map along the channel dimension. This represents a submodule that performs feature description and weight calculation on the input feature map in the spatial dimension.

3. The fracture detection method in X-ray images based on progressive alignment and feature fusion networks according to claim 1, characterized in that, The calculation method of the content-aware guided sampling module is as follows: It adaptively calculates the pixel sampling offset and sampling weight coefficients based on the input feature map, and performs content-aware resampling of the original feature map according to the sampling offset and sampling weights to generate enhanced resampled features. ； ； ； ； ； Where X represents the input feature map of the previous layer. This indicates that the input feature map is upsampled by a factor of s. This represents the result after upsampling the input feature map. This represents the offset field predicted from the original feature map. This represents the result after upsampling the input prediction offset field. This represents the sigmoid function. This represents the normalized offset field. Represents the basic sampling network. This represents the adaptive sampling network resulting from the fusion of the base sampling network and the normalized offset field. GridSample represents the content-aware sampling operation performed on the input feature map using bilinear interpolation based on the sampling grid coordinates. This represents the output feature map after adaptive offset guided sampling.

4. The fracture detection method in X-ray images based on progressive alignment and feature fusion networks according to claim 1, characterized in that, The PAF-Net includes a backbone network and a head detection layer. The shallow structure of the backbone includes two high-resolution feature extraction stages, P1 and P2, with progressive directional convolution modules embedded in each stage.

5. The fracture detection method in X-ray images based on progressive alignment and feature fusion networks according to claim 1, characterized in that, The deep backbone structure includes three feature extraction stages: P3, P4, and P5, which are connected in sequence according to depth.

6. The fracture detection method in X-ray images based on progressive alignment and feature fusion networks according to claim 1, characterized in that, The detection head includes: a convolutional branch for bounding box regression, a confidence branch for target existence prediction, and a classification branch for fracture category identification.

Images