Wrist Image Fracture Recognition and Detection Method and System Based on Improved YOLOv8 Network

Abstract

The present invention discloses a method and system for wrist image fracture recognition and detection based on an improved YOLOv8 network. The method includes: obtaining a dataset of wrist fracture images to be processed and performing data preprocessing to obtain wrist fracture images; based on a fast network backbone structure, introducing a bidirectional feature pyramid network structure integrated with a global-to-local spatial aggregation module and a dynamic cross-stage deformable convolutional network module to construct an improved YOLOv8 recognition network model; performing image recognition processing on the wrist fracture images based on the improved YOLOv8 recognition network model to obtain wrist image fracture recognition results. The present invention can enhance the ability to capture wrist fracture features in complex backgrounds and the efficiency of feature extraction, thereby improving the accuracy of wrist fracture image recognition. As a method and system for wrist image fracture recognition and detection based on an improved YOLOv8 network, the present invention can be widely applied to the field of image recognition technology.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method and system for wrist image fracture recognition and detection based on an improved YOLOv8 network. Background Art

[0002] Medical image diagnosis methods based on artificial intelligence have been gradually widely applied in the field of fracture diagnosis. More and more researchers use convolutional neural networks (CNNs) for fracture detection and classification. At present, automated diagnosis based on convolutional neural networks mainly inputs wrist fracture X-ray images into the model, and then extracts local and global features from the images through multi-layer convolution and pooling operations. Finally, a fully connected layer or a Softmax activation function is used to classify the features and output the prediction results. However, the related technologies still have certain limitations. First, the fracture areas in wrist fracture images are usually small and irregular in shape. Traditional convolution operations are limited by the receptive field and it is difficult to accurately capture the features of small targets. Second, the complex background and low contrast in X-ray images further increase the detection difficulty. The existing feature fusion modules are insufficient in aggregating detailed information, resulting in the model being prone to missed detection or false detection under complex image conditions. Finally, existing models usually adopt relatively complex feature extraction modules, with a large amount of calculations and a lack of lightweight design, and the inference speed is slow, making it difficult to meet the real-time requirements. Summary of the Invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a method and system for wrist image fracture recognition and detection based on an improved YOLOv8 network, which can enhance the ability to capture wrist fracture features under complex backgrounds and the efficiency of feature extraction, thereby improving the accuracy of wrist fracture image recognition.

[0004] The first technical solution adopted by the present invention is: A method for wrist image fracture recognition and detection based on an improved YOLOv8 network, comprising the following steps:

[0005] Obtain a dataset of wrist fracture images to be processed and perform data preprocessing to obtain wrist fracture images;

[0006] Based on a fast network backbone structure, introduce a bidirectional feature pyramid network structure integrating a global-to-local spatial aggregation module and a dynamic cross-stage deformable convolutional network module to construct an improved YOLOv8 recognition network model;

[0007] Perform image recognition processing on the wrist fracture images based on the improved YOLOv8 recognition network model to obtain the wrist image fracture recognition results.

[0008] Further, the step of obtaining a dataset of wrist fracture images to be processed and performing data preprocessing to obtain wrist fracture images specifically includes:

[0009] Obtain a dataset of wrist fracture images to be processed;

[0010] Perform data cleaning and annotation processing on the dataset of wrist fracture images to be processed to obtain a dataset of wrist fracture images with annotations;

[0011] Perform data augmentation and division processing on the dataset of wrist fracture images with annotations to obtain wrist fracture images.

[0012] Furthermore, the improved YOLOv8 recognition network model includes a backbone network module, a neck network module, and a detection head network module, which are connected in sequence. Among them:

[0013] The backbone network module adopts a fast network backbone structure. The backbone network module specifically includes an embedding layer, a fast network module, a backbone fusion block, a global pooling layer, a backbone convolutional block, a fully connected layer, and a spatial pyramid fast pooling module. The fast network module includes a depthwise separable convolutional layer and an attention mechanism;

[0014] The neck network module adopts a bidirectional feature pyramid network structure. The neck network module specifically includes a global-to-local spatial aggregation module, an upsampling module, a neck fusion block, and a dynamic cross-stage deformable convolutional network module. The global-to-local spatial aggregation module includes a first feature segmentation module, a first two-dimensional convolutional module, a second two-dimensional convolutional module, a global-to-local spatial aggregation convolutional branch module, a context information aggregation module, a first feature splicing module, and a first convolutional block. The dynamic cross-stage deformable convolutional network module includes a second convolutional block, a second feature segmentation module, several dynamic deformable convolutional bottleneck modules, and a second feature splicing module.

[0015] Furthermore, it also includes introducing a generalized intersection over union loss function based on the intersection over union loss function, and constructing an internal generalized intersection over union loss function as the loss function of the improved YOLOv8 recognition network model. The expression of the internal generalized intersection over union loss function is specifically as follows:

[0016] ;

[0017] In the above formula, represents the internal generalized intersection over union loss function, represents the intersection over union loss function, represents the generalized intersection over union, represents the area of the smallest bounding rectangle of two bounding boxes, represents the union area.

[0018] Further, the step of performing image recognition processing on the wrist fracture image using the improved YOLOv8 recognition network model to obtain the wrist image fracture recognition result specifically includes:

[0019] Input the wrist fracture image into the improved YOLOv8 recognition network model;

[0020] Based on the backbone network module of the improved YOLOv8 recognition network model, perform feature extraction processing on the wrist fracture image to obtain a wrist fracture feature image;

[0021] Based on the neck network module of the improved YOLOv8 recognition network model, perform feature fusion processing on the wrist fracture feature image to obtain a fused wrist fracture feature image;

[0022] Based on the detection head network module of the improved YOLOv8 recognition network model, perform target recognition detection on the fused wrist fracture feature image to obtain the wrist image fracture recognition result.

[0023] Further, the step of performing feature extraction processing on the wrist fracture image based on the backbone network module of the improved YOLOv8 recognition network model to obtain a wrist fracture feature image specifically includes:

[0024] Input the wrist fracture image into the backbone network module of the improved YOLOv8 recognition network model;

[0025] Based on the embedding layer of the backbone network module, perform data dimensionality reduction processing on the wrist fracture image to obtain a dimensionally reduced wrist fracture image;

[0026] Based on the fast network module of the backbone network module, perform local detail feature and global structure feature extraction processing on the dimensionally reduced wrist fracture image to obtain a preliminary wrist fracture feature image;

[0027] Based on the backbone fusion block of the backbone network module, perform multi-resolution feature fusion processing on the preliminary wrist fracture feature image to obtain a fused wrist fracture feature image;

[0028] Based on the global pooling layer of the backbone network module, perform global pooling compression processing on the fused wrist fracture feature image to obtain a compressed wrist fracture feature image;

[0029] Based on the backbone convolution block of the backbone network module, perform convolution processing on the compressed wrist fracture feature image to obtain a convolved wrist fracture feature image;

[0030] Based on the fully connected layer of the backbone network module, perform mapping on the convolved wrist fracture feature image to obtain a mapped wrist fracture feature image;

[0031] The spatial pyramid fast pooling module based on the backbone network module performs multi-scale aggregation processing on the mapped wrist fracture feature image to obtain the wrist fracture feature image.

[0032] Furthermore, the step of the neck network module based on the improved YOLOv8 recognition network model performing feature fusion processing on the wrist fracture feature image to obtain the fused wrist fracture feature image specifically includes:

[0033] Input the wrist fracture feature image into the neck network module of the improved YOLOv8 recognition network model;

[0034] Based on the global-to-local spatial aggregation module of the neck network module, perform global attention weighted calculation and feature aggregation processing on the wrist fracture feature image to obtain the wrist fracture global feature image;

[0035] Based on the upsampling module of the neck network module, perform upsampling processing on the wrist fracture global feature image to obtain the upsampled wrist fracture global feature image;

[0036] Based on the neck fusion block of the neck network module, perform multi-scale feature fusion processing on the upsampled wrist fracture global feature image to obtain the preliminarily fused wrist fracture global feature image;

[0037] Based on the dynamic cross-stage deformable convolutional network module of the neck network module, perform cross-stage feature optimization processing on the preliminarily fused wrist fracture global feature image to obtain the fused wrist fracture feature image.

[0038] Furthermore, the step of the global-to-local spatial aggregation module based on the neck network module performing global attention weighted calculation and feature aggregation processing on the wrist fracture feature image to obtain the wrist fracture global feature image specifically includes:

[0039] Input the wrist fracture feature image into the global-to-local spatial aggregation module of the neck network module;

[0040] Based on the first feature segmentation module of the global-to-local spatial aggregation module, perform feature segmentation processing on the wrist fracture feature image to obtain the segmented wrist fracture feature image;

[0041] Based on the first-branch two-dimensional convolutional module and the second-branch two-dimensional convolutional module of the global-to-local spatial aggregation module, perform convolutional processing on the segmented wrist fracture feature image respectively to obtain the first convolutional wrist fracture feature image and the second convolutional wrist fracture feature image;

[0042] The global-to-local spatial aggregation convolution branch module based on the global-to-local spatial aggregation module performs spatial feature aggregation processing on the first convolutional wrist fracture feature image to obtain the first aggregated wrist fracture feature image;

[0043] The context information aggregation module based on the global-to-local spatial aggregation module performs context information aggregation processing on the second convolutional wrist fracture feature image to obtain the second aggregated wrist fracture feature image;

[0044] The second feature splicing module based on the global-to-local spatial aggregation module splices the first aggregated wrist fracture feature image and the second aggregated wrist fracture feature image to obtain the spliced wrist fracture feature image;

[0045] The first convolutional block based on the global-to-local spatial aggregation module performs convolution processing on the spliced wrist fracture feature image to obtain the wrist fracture global feature image.

[0046] Further, the step of the dynamic cross-stage deformable convolution network module based on the neck network module performing cross-stage feature optimization processing on the preliminarily fused wrist fracture global feature image to obtain the fused wrist fracture feature image specifically includes:

[0047] Input the preliminarily fused wrist fracture global feature image into the dynamic cross-stage deformable convolution network module of the neck network module;

[0048] The second convolutional block based on the dynamic cross-stage deformable convolution network module performs convolution processing on the preliminarily fused wrist fracture global feature image to obtain the convolved wrist fracture global feature image;

[0049] The second feature segmentation module based on the dynamic cross-stage deformable convolution network module performs feature segmentation processing on the convolved wrist fracture global feature image to obtain the segmented wrist fracture global feature image;

[0050] The dynamic deformable convolution bottleneck module based on the dynamic cross-stage deformable convolution network module performs dynamic feature optimization processing on the segmented wrist fracture global feature image to obtain the optimized wrist fracture feature image;

[0051] The second feature splicing module based on the dynamic cross-stage deformable convolution network module performs feature splicing and fusion processing on the optimized wrist fracture feature image to obtain the fused wrist fracture feature image.

[0052] The second technical solution adopted by the present invention is: a wrist image fracture recognition and detection system based on the improved YOLOv8 network, including:

[0053] The first module is used to obtain the wrist fracture image dataset to be processed and perform data preprocessing to obtain wrist fracture images;

[0054] The second module is used to introduce a bidirectional feature pyramid network structure integrating a global-to-local spatial aggregation module and a dynamic cross-stage deformable convolutional network module based on a fast network backbone structure, and construct an improved YOLOv8 recognition network model;

[0055] The third module is used to perform image recognition processing on the wrist fracture images based on the improved YOLOv8 recognition network model to obtain the wrist image fracture recognition result.

[0056] The beneficial effects of the method and system of the present invention are as follows: By obtaining the wrist fracture image dataset to be processed and performing data preprocessing, and further introducing a bidirectional feature pyramid network structure integrating a global-to-local spatial aggregation module and a dynamic cross-stage deformable convolutional network module based on a fast network backbone structure, the backbone structure of the YOLOv8 network is optimized through the fast network backbone structure, enabling the model to perform excellently in multi-scale feature extraction, significantly improving the extraction accuracy and calculation efficiency of fracture features while reducing the number of parameters. Further, through the global-to-local spatial aggregation module, while maintaining the global receptive field, it can more meticulously aggregate local spatial information, enhancing the ability to capture fracture features under complex backgrounds, thereby improving the detection accuracy of the model. The accuracy of feature extraction and fusion is further improved through the dynamic cross-stage deformable convolutional network module. Finally, image recognition processing is performed on the wrist fracture images based on the improved YOLOv8 recognition network model to improve the accuracy of wrist fracture image recognition. Description of the Drawings

[0057] Figure 1 is the step flow chart of the wrist image fracture recognition and detection method based on the improved YOLOv8 network of the present invention;

[0058] Figure 2 is the structural block diagram of the wrist image fracture recognition and detection system based on the improved YOLOv8 network of the present invention;

[0059] Figure 3 is the structural schematic diagram of the improved YOLOv8 recognition network model provided by a specific embodiment of the present invention;

[0060] Figure 4 is the structural schematic diagram of the backbone network provided by a specific embodiment of the present invention;

[0061] Figure 5 is the structural schematic diagram of the integrated bidirectional feature pyramid network provided by a specific embodiment of the present invention;

[0062] Figure 6It is a schematic structural diagram of the dynamic deformable convolution module provided by a specific embodiment of the present invention;

[0063] Figure 7 It is a schematic structural diagram of the dynamic deformable convolution bottleneck module provided by a specific embodiment of the present invention;

[0064] Figure 8 It is a schematic structural diagram of the dynamic cross-stage deformable convolution network module provided by a specific embodiment of the present invention;

[0065] Figure 9 It is a schematic structural diagram of the global-to-local spatial aggregation convolution branch module provided by a specific embodiment of the present invention;

[0066] Figure 10 It is a schematic structural diagram of the global-to-local spatial aggregation module provided by a specific embodiment of the present invention;

[0067] Figure 11 It is a schematic diagram of the change in the heat map before and after the improvement of the network provided by a specific embodiment of the present invention;

[0068] Figure 12 It is a schematic diagram of the change in detection before and after the improvement of the network provided by a specific embodiment of the present invention. Detailed implementation manners

[0069] The following further elaborates on the present invention in detail in conjunction with the accompanying drawings and specific embodiments. For the step numbers in the following embodiments, they are only set for the convenience of elaboration and explanation, and no limitation is imposed on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0070] Refer to Figure 1 , the present invention provides a method for wrist image fracture recognition and detection based on an improved YOLOv8 network, and the method includes the following steps:

[0071] S100. Obtain a wrist fracture image dataset to be processed and perform data preprocessing to obtain wrist fracture images;

[0072] Specifically, obtain a wrist fracture image dataset to be processed; perform data cleaning and annotation processing on the wrist fracture image dataset to be processed to obtain a labeled wrist fracture image dataset; perform data augmentation and division processing on the labeled wrist fracture image dataset to obtain wrist fracture images.

[0073] In this embodiment, based on the publicly available dataset GRAZPEDWRI-DX and the wrist fracture image dataset provided by a traditional Chinese medicine hospital, a wrist fracture image dataset to be processed is constructed. Then, data cleaning is performed, and annotation tools such as LabelImg are used to annotate the fracture sites, generating a YOLO format dataset containing fracture positions. The data augmentation library (Albumentations) is used to perform data augmentation operations such as random rotation, scaling, and translation on the image data to enrich the diversity of the dataset and improve the adaptability of the model to different imaging conditions. Finally, the annotated and augmented dataset is divided into a training set, a validation set, and a test set according to the ratio of 75%:15%:15% to ensure the effectiveness of model training and the accuracy of testing.

[0074] S200. Based on the fast network backbone structure, introduce a bidirectional feature pyramid network structure that integrates a global-to-local spatial aggregation module and a dynamic cross-stage deformable convolutional network module to construct an improved YOLOv8 recognition network model;

[0075] Specifically, as Figure 3 shown, the improved YOLOv8 recognition network model includes a backbone network module, a neck network module, and a detection head network module. The backbone network module, the neck network module, and the detection head network module are connected in sequence, where:

[0076] Furthermore, as Figure 4 shown, the backbone network module adopts a fast network backbone structure. The backbone network module specifically includes an embedding layer, a fast network module, a backbone fusion block, a global pooling layer, a backbone convolutional block, a fully connected layer, and a spatial pyramid fast pooling module. The fast network module includes a depthwise separable convolutional layer and an attention mechanism;

[0077] In this embodiment, in the deep learning object detection model, the backbone is responsible for extracting features from the input image, converting the original image pixel information into a multi-dimensional feature representation, and these features are then passed to the subsequent neck network and detection head for more in-depth object detection and localization. The feature extraction ability of the backbone determines the detection accuracy, speed, and generalization performance of the model, and is a key part of the entire detection process. Therefore, optimizing the backbone can significantly improve the overall performance of the model. In YOLOv8, the backbone mainly consists of multiple convolutional layers and pooling layers for extracting image features at different levels. To improve the model's performance in wrist fracture image detection, the embodiment of the present invention first refers to the design idea of FasterNet and improves it in combination with the architecture of YOLOv8. FasterNet is a lightweight and efficient backbone network architecture proposed in CVPR2023. Then, FasterNet is used to replace the backbone of YOLOv8. At the same time, to retain the multi-scale feature extraction ability of YOLOv8, the Spatial Pyramid Pooling Fast (SPPF) module in the original backbone of YOLOv8 is retained in the backbone. By combining the lightweight design of FasterNet and the multi-scale receptive field enhancement of the SPPF module, while reducing the computational overhead, it is still possible to maintain a high feature expression ability.

[0078] Further, as Figure 5 shown, the neck network module adopts a bidirectional feature pyramid network structure. The neck network module specifically includes a global-to-local spatial aggregation module, an upsampling module, a neck fusion block, and a dynamic cross-stage deformable convolutional network module. Among them, as Figure 10 shown, the global-to-local spatial aggregation module includes a first feature segmentation module, a first two-dimensional convolutional module, a second two-dimensional convolutional module, a global-to-local spatial aggregation convolutional branch module, a context information aggregation module, a first feature splicing module, and a first convolutional block. Further, as Figure 8 shown, the dynamic cross-stage deformable convolutional network module includes a second convolutional block, a second feature segmentation module, several dynamic deformable convolutional bottleneck modules, and a second feature splicing module. Among them, as Figure 7 shown, the dynamic deformable convolutional bottleneck module includes a convolutional block, a dynamic deformable convolutional module, and a skip connection block. Among them, as Figure 6 shown, the dynamic deformable convolutional module includes a two-dimensional convolutional module, a multi-scale position-sensitive cross-attention module, a feature chunking module, a feature splicing module, an activation function module, a dynamic deformable convolutional module, a two-dimensional batch normalization module, and a SiLU activation function module. Even further, as Figure 9 shown, the global-to-local spatial aggregation convolutional branch module includes several convolutional blocks and a sigmoid activation function module.

[0079] In this embodiment, in the deep learning object detection model, the Neck network is responsible for fusing features from different levels, thereby improving the model's ability to locate and classify objects. In particular, multi-scale feature fusion is crucial for accurately detecting objects of different sizes. In the task of wrist fracture image detection, the features of the fracture area are often complex and tiny. Therefore, optimizing the Neck network can significantly improve the detection accuracy and generalization performance of the model. To improve the performance of the algorithm in wrist fracture detection, in the embodiment of the present invention, the PAFPN in YOLOv8 is first replaced with BiFPN in the Neck network to improve the flexibility and efficiency of feature fusion. BiFPN allows features to flow bidirectionally between different resolutions, enabling the model to better fuse multi-scale features and enhancing the detection ability for small objects (such as fracture details). To further optimize BiFPN, in the embodiment of the present invention, some convolutional modules in BiFPN are replaced with the "Global to Local Spatial Aggregation (GLSA)" module. GLSA can aggregate local spatial information more meticulously while maintaining the global receptive field, enhancing the ability to capture fracture features under complex backgrounds, thereby improving the detection accuracy of the model. Finally, the improved MPCA enhanced deformable convolutional network (DCNv2) is used to replace the Cross Stage Partial Network (C2f) module in BiFPN. Based on the cross-attention mechanism (CA) proposed in CVPR2021, the MPCA attention mechanism is designed to enhance the ability to generate offsets and masks in DCNv2. By combining MPCA with DCNv2 and embedding it into the C2f module, a Dynamic Cross Stage Deformable Convolutional Network Module (C2f_DCNv2_Dynamic) module is formed, which further replaces the C2f module in BiFPN to further improve the accuracy of feature extraction and fusion, especially performing more excellently in complex wrist fracture image scenarios.

[0080] Furthermore, it should be noted that in the deep learning object detection model, the loss function is used to evaluate the difference between the prediction result and the true annotation, and its design directly affects the training effect of the model and the final detection performance. The localization loss of the original loss function of YOLOv8 uses , which considers the overlapping area between bounding boxes and the distance of their center points, and has a certain improvement effect on the localization accuracy. However, in the task of wrist fracture image detection, the complex fracture morphology and small target area pose higher requirements for the localization accuracy. To further improve the localization accuracy of the model, in the embodiment of the present invention, the localization loss function of YOLOv8 is improved. First, the Intersection over Union ( ) is used to replace , The loss function focuses on the overlapping region inside the bounding box and is more sensitive to small objects and subtle boundaries, making it perform better in the wrist fracture detection task. Further, an internal Generalized Intersection over Union ( ) is constructed. Based on , the Generalized Intersection over Union ( ) is introduced for improvement, enabling the loss function to more accurately measure the overlap and difference between the predicted box and the ground truth box, especially when the position and shape of the bounding box are complex. It can provide higher localization accuracy.

[0081] The loss function of the embodiment of the present invention is specifically defined as follows:

[0082] 1) Bounding box adjustment:

[0083] According to the scale parameter ( , defined as in this embodiment), the size of each bounding box is reduced, and its reduced coordinates are calculated;

[0084] For the first bounding box box1, its calculation expression is:

[0085] ;

[0086] For the second bounding box box2, its calculation expression is:

[0087] ;

[0088] 2) Intersection area calculation, and its expression is:

[0089] ;

[0090] The above formula calculates the product of the overlapping distances of the two bounding boxes in the and directions, thereby obtaining the area of the overlapping region. If the two boxes do not overlap, the overlapping area is 0.

[0091] 3) Union area calculation, and its expression is:

[0092] ;

[0093] The union area is the sum of the areas of the two boxes minus their intersection area, is a very small value used to prevent the denominator from being 0.

[0094] 4) calculation, and its expression is:

[0095] ;

[0096] 5) Introduction: is an improvement of the standard, and evaluates the positional relationship between two bounding boxes by introducing the minimum bounding rectangle. The calculation formula of

[0097] is as follows:

[0098] ;

[0099] Among them, is the area of the minimum bounding rectangle of the two bounding boxes.

[0100] The expression of

[0101] is specifically as follows:

[0102] In , is used to measure the matching degree between the shrunk core regions, and the second part , that is, is used to penalize the relative positions of the two boxes. If the two boxes are far apart, this value will increase, thereby reducing the overall value.

[0103] In summary, the improved YOLOv8 recognition network model constructed in the embodiments of the present invention can more effectively cope with the characteristics of medical images and the subtle features of wrist fractures. By combining the lightweight design concept of the FasterNet, the backbone network of YOLOv8 is optimized, making feature extraction more efficient, especially more adaptable in multi-scale feature fusion. At the same time, the bidirectional feature pyramid network (BiFPN) is adopted in the neck network to replace the original path aggregation feature pyramid network (PAFPN), and the global-to-local spatial aggregation (GLSA) module and the improved multi-scale position-sensitive cross-attention mechanism (MPCA) are used to further enhance the feature aggregation ability. These improvements have improved the detection accuracy and operation efficiency of the model, and can better support the auxiliary diagnosis requirements of wrist fractures. In addition, by using the Inner Generalized Intersection over Union (Inner_GIoU) to replace the original Complete Intersection over Union (CIOU) loss function, the positioning ability in the detection of complex shapes and small targets can be improved.

[0104] S300. Perform image recognition processing on the wrist fracture image based on the improved YOLOv8 recognition network model to obtain the wrist image fracture recognition result.

[0105] S310. Input the wrist fracture image into the improved YOLOv8 recognition network model;

[0106] S320. The backbone network module of the improved YOLOv8 recognition network model performs feature extraction processing on the wrist fracture image to obtain a wrist fracture feature image;

[0107] Specifically, the wrist fracture image is input into the backbone network module of the improved YOLOv8 recognition network model; based on the embedding layer of the backbone network module, data dimensionality reduction processing is performed on the wrist fracture image to obtain the wrist fracture image after dimensionality reduction; based on the fast network module of the backbone network module, local detail features and global structure features are extracted from the wrist fracture image after dimensionality reduction to obtain a preliminary wrist fracture feature image; based on the backbone fusion block of the backbone network module, multi-resolution feature fusion processing is performed on the preliminary wrist fracture feature image to obtain a fused wrist fracture feature image; based on the global pooling layer of the backbone network module, global pooling compression processing is performed on the fused wrist fracture feature image to obtain a compressed wrist fracture feature image; based on the backbone convolution block of the backbone network module, convolution processing is performed on the compressed wrist fracture feature image to obtain a convolved wrist fracture feature image; based on the fully connected layer of the backbone network module, mapping is performed on the convolved wrist fracture feature image to obtain a mapped wrist fracture feature image; based on the spatial pyramid fast pooling module of the backbone network module, multi-scale aggregation processing is performed on the mapped wrist fracture feature image to obtain a wrist fracture feature image.

[0108] In this embodiment, lightweight convolution modules designed by FasterNet are used to replace some convolution modules in YOLOv8 to reduce the number of parameters and computational overhead. The input image is efficiently encoded through the Embedding layer, converting low-level texture information into high-dimensional feature representations, reducing the data dimension while alleviating the subsequent computational burden; the FasterNet Block combines depthwise separable convolution and attention mechanisms to accurately extract local details and global structure features of the image, significantly enhancing the model's perception ability of the fracture area; the Merging layer integrates multi-resolution features to improve the model's adaptability to targets of different scales; the global pooling layer further integrates global features, compresses redundant information, and improves the feature expression efficiency. These optimizations not only improve the detection accuracy and feature extraction efficiency of the model, but also enhance its adaptability to complex fracture morphologies, while effectively reducing the computational complexity of fracture image diagnosis. The SPPF module in YOLOv8 is retained to enhance the multi-scale feature extraction ability, enabling the model to have higher feature expression ability while remaining lightweight. Specifically, the SPPF module can perform multi-scale aggregation on features of different scales, generating multi-layer feature representations by applying max pooling operations to the feature map, thereby enhancing the detection ability for small targets (such as fracture details) and enabling the model to have higher feature expression ability while remaining lightweight.

[0109] S330. The neck network module based on the improved YOLOv8 recognition network model performs feature fusion processing on the wrist fracture feature image to obtain the fused wrist fracture feature image.

[0110] Specifically, the wrist fracture feature image is input into the neck network module of the improved YOLOv8 recognition network model; based on the global-to-local spatial aggregation module of the neck network module, global attention weighting calculation and feature aggregation processing are performed on the wrist fracture feature image to obtain the global wrist fracture feature image; based on the upsampling module of the neck network module, upsampling processing is performed on the global wrist fracture feature image to obtain the upsampled global wrist fracture feature image; based on the neck fusion block of the neck network module, multi-scale feature fusion processing is performed on the upsampled global wrist fracture feature image to obtain the preliminarily fused global wrist fracture feature image; based on the dynamic cross-stage deformable convolutional network module of the neck network module, cross-stage feature optimization processing is performed on the preliminarily fused global wrist fracture feature image to obtain the fused wrist fracture feature image.

[0111] Among them, for the global-to-local spatial aggregation module, the wrist fracture feature image is input into the global-to-local spatial aggregation module of the neck network module; based on the first feature segmentation module of the global-to-local spatial aggregation module, feature segmentation processing is performed on the wrist fracture feature image to obtain the segmented wrist fracture feature image; based on the first branch two-dimensional convolutional module and the second branch two-dimensional convolutional module of the global-to-local spatial aggregation module, convolutional processing is respectively performed on the segmented wrist fracture feature image to obtain the first convolutional wrist fracture feature image and the second convolutional wrist fracture feature image; based on the global-to-local spatial aggregation convolutional branch module of the global-to-local spatial aggregation module, spatial feature aggregation processing is performed on the first convolutional wrist fracture feature image to obtain the first aggregated wrist fracture feature image; based on the context information aggregation module of the global-to-local spatial aggregation module, context information aggregation processing is performed on the second convolutional wrist fracture feature image to obtain the second aggregated wrist fracture feature image; based on the second feature splicing module of the global-to-local spatial aggregation module, the first aggregated wrist fracture feature image and the second aggregated wrist fracture feature image are spliced to obtain the spliced wrist fracture feature image; based on the first convolutional block of the global-to-local spatial aggregation module, convolutional processing is performed on the spliced wrist fracture feature image to obtain the global wrist fracture feature image.

[0112] Further, for the dynamic cross-stage deformable convolutional network module, the globally-featured wrist fracture image after preliminary fusion is input into the dynamic cross-stage deformable convolutional network module of the neck network module; based on the second convolutional block of the dynamic cross-stage deformable convolutional network module, convolutional processing is performed on the globally-featured wrist fracture image after preliminary fusion to obtain a convolutionally processed globally-featured wrist fracture image; based on the second feature segmentation module of the dynamic cross-stage deformable convolutional network module, feature segmentation processing is performed on the convolutionally processed globally-featured wrist fracture image to obtain a segmented globally-featured wrist fracture image; based on the dynamic deformable convolutional bottleneck module of the dynamic cross-stage deformable convolutional network module, dynamic feature optimization processing is performed on the segmented globally-featured wrist fracture image to obtain an optimized wrist fracture feature image; based on the second feature splicing module of the dynamic cross-stage deformable convolutional network module, feature splicing and fusion processing is performed on the optimized wrist fracture feature image to obtain a fused wrist fracture feature image.

[0113] In this embodiment, BiFPN is used to replace the original PAFPN. BiFPN allows features to flow bidirectionally between different resolutions. By cascading feature maps of different resolutions, the ability to fuse multi-scale features is strengthened. This improvement enables the model to effectively capture the detailed features of small targets in wrist fracture images while reducing unnecessary computational overhead. The convolutional module in BiFPN is replaced with a GLSA module to increase the fineness of feature aggregation and enhance the feature capture ability in complex backgrounds. Specifically, GLSA performs global attention weighting on feature maps from different resolutions and then performs fine-grained feature aggregation on local regions, thereby more accurately extracting the features of the fracture region in complex backgrounds. The MPCA attention mechanism is used to strengthen the generation of offsets and masks in DCNv2, improve the dynamics and pertinence of feature extraction, and embed it into the C2f module to form the C2f_DCNv2_Dynamic module. Then, by replacing the C2f module in BiFPN, the cross-stage part of the feature map can be optimized, effectively extracting the key features of the fracture region while suppressing the interference of complex backgrounds.

[0114] S340. Based on the detection head network module of the improved YOLOv8 recognition network model, target recognition and detection are performed on the fused wrist fracture feature image to obtain the wrist image fracture recognition result.

[0115] Further, the improved YOLOv8 network is trained. The number of training rounds is set to 300, the early stopping is set to 40 rounds, the initial learning rate is 0.01, the minimum learning rate is 0.01, the batch size is 64, the SGD optimizer is used, the weight decay is 0.0005. During the training process, iterate until the best effect is achieved, save the optimal weights, and use these weights to test on the test set to verify the model performance.

[0116] Finally, model evaluation is carried out, using recall (R), mean average precision (mAP50, mAP50-95) as evaluation indicators to evaluate the performance of the improved model. The experimental results show that the improved fracture detection algorithm has been improved in terms of accuracy, computational efficiency and real-time performance. Compared with the original YOLOv8 algorithm, the recall rate and mean average precision of the model have been optimized, and the model has higher accuracy in wrist fracture localization and detection. Among them, for the detection of the total category of the affected wrist, the recall rate has increased from 0.613 to 0.761, an increase of about 24%; MAP50 has increased from 0.624 to 0.784, an increase of about 26%; MAP50-100 has increased from 0.344 to 0.501, an increase of about 46%. For the detection of fracture categories, the recall rate has increased from 0.741 to 0.869, an increase of about 17%; MAP50 has increased from 0.826 to 0.928, an increase of about 12%; MAP50-100 has increased from 0.425 to 0.54, an increase of about 27%. The specific data are shown in Table 1 and Table 2, and the simulation experimental results are as Figure 11 and Figure 12 shown.

[0117] Among them, to verify the improvement of the optimized model in feature extraction, in this embodiment, the differences in the feature extraction capabilities of the models are compared through heatmaps. As Figure 11 shown, by comparing the feature activation maps of the models on the same dataset, it can be seen that the optimized model shows stronger feature responses in the fracture and joint regions of the image, and can focus more precisely on the key regions, thereby improving the accuracy and robustness of object detection.

[0118] Partial detection results of the validation set are as Figure 12 shown. By comparison, it can be obtained that the YOLOv8n model has false detection or missed detection in some samples. Through improvement, the model effectively improves this limitation, can more accurately label the fracture area, and improve the detection confidence.

[0119] Table 1 Data table of the model training results at the current stage

[0120]

[0121] Table 2 Data table of the model training results constructed by the present invention

[0122]

[0123] In summary, based on the YOLOv8 algorithm, the embodiments of the present invention optimize the backbone network by combining the design concept of FasterNet, while retaining the original SPPF module of YOLOv8, enabling the model to perform excellently in multi-scale feature extraction. Compared with the original backbone network structure, this algorithm significantly improves the extraction accuracy and calculation efficiency of fracture features while reducing the number of parameters.

[0124] In the neck network, BiFPN is used to replace PAFPN to enhance the flexibility of feature fusion, and the GLSA module is used to replace the convolutional module in BiFPN, improving the ability to capture details in the fracture area. In addition, an improved MPCA attention mechanism is designed and embedded in the offset and mask generation modules of DCNv2, combined with the C2f module to form the C2f_DCNv2_Dynamic module to further enhance the feature fusion accuracy of BiFPN. This optimization scheme makes the model more accurate in focusing on fracture features at different scales and in complex backgrounds.

[0125] In terms of localization loss, CIOU is replaced by Inner_GIoU, enabling the model to have higher localization accuracy in the detection of small targets and complex-shaped boundaries. Inner_GIoU can more effectively optimize the boundary matching of the fracture area during model training by reducing the size of the bounding box and combining the penalty mechanism of GIoU, thus significantly improving the localization accuracy.

[0126] Refer to Figure 2 , a wrist image fracture recognition and detection system based on the improved YOLOv8 network, includes:

[0127] The first module 201 is used to obtain the wrist fracture image data set to be processed and perform data preprocessing to obtain wrist fracture images;

[0128] The second module 202 is used to construct an improved YOLOv8 recognition network model based on the fast network backbone structure, introducing a bidirectional feature pyramid network structure integrating a global-to-local spatial aggregation module and a dynamic cross-stage deformable convolutional network module;

[0129] The third module 203 is used to perform image recognition processing on the wrist fracture images based on the improved YOLOv8 recognition network model to obtain the wrist image fracture recognition results.

[0130] The content in the above method embodiments is applicable to the system embodiments. The functions specifically implemented by the system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0131] The above is a specific description of the preferred embodiment of the present invention. However, the present invention is not limited to the described embodiment. Those skilled in the art can make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for wrist image fracture recognition and detection based on an improved YOLOv8 network, characterized in that, It includes the following steps: Obtain the wrist fracture image dataset to be processed and perform data preprocessing to obtain wrist fracture images; Based on the fast network backbone structure, introduce a bidirectional feature pyramid network structure that integrates a global-to-local spatial aggregation module and a dynamic cross-stage deformable convolutional network module, and construct an improved YOLOv8 recognition network model; The improved YOLOv8 recognition network model includes a backbone network module, a neck network module, and a detection head network module. The backbone network module, the neck network module, and the detection head network module are connected in sequence, where: The backbone network module adopts a fast network backbone structure. The backbone network module specifically includes an embedding layer, a fast network module, a backbone fusion block, a global pooling layer, a backbone convolutional block, a fully connected layer, and a spatial pyramid fast pooling module. The fast network module includes a depthwise separable convolutional layer and an attention mechanism; The neck network module adopts a bidirectional feature pyramid network structure. The neck network module specifically includes a global-to-local spatial aggregation module, an upsampling module, a neck fusion block, and a dynamic cross-stage deformable convolutional network module. The global-to-local spatial aggregation module includes a first feature segmentation module, a first two-dimensional convolutional module, a second two-dimensional convolutional module, a global-to-local spatial aggregation convolution branch module, a context information aggregation module, a first feature splicing module, and a first convolutional block. The dynamic cross-stage deformable convolutional network module includes a second convolutional block, a second feature segmentation module, several dynamic deformable convolutional bottleneck modules, and a second feature splicing module; Among them, for the neck network, first replace the PAFPN in YOLOv8 with BiFPN, replace the convolutional module in BiFPN with the global-to-local spatial aggregation module GLSA, design a multi-scale position-sensitive cross-attention module MPCA based on the cross-attention mechanism CA to enhance the offset and mask generation capabilities in the deformable convolutional network DCNv2. By combining the multi-scale position-sensitive cross-attention module MPCA with the deformable convolutional network DCNv2 and embedding it into the cross-stage partial network module C2f, form a dynamic cross-stage deformable convolutional network module C2f_DCNv2_Dynamic, and further replace the cross-stage partial network module C2f in BiFPN; Based on the improved YOLOv8 recognition network model, perform image recognition processing on the wrist fracture images to obtain the wrist image fracture recognition results.

2. The wrist image fracture recognition and detection method based on the improved YOLOv8 network according to claim 1, characterized in that, The step of obtaining the wrist fracture image dataset to be processed and performing data preprocessing to obtain wrist fracture images specifically includes: Obtain the wrist fracture image dataset to be processed; Perform data cleaning and annotation processing on the wrist fracture image dataset to be processed to obtain a labeled wrist fracture image dataset; Perform data augmentation and division processing on the labeled wrist fracture image dataset to obtain wrist fracture images.

3. The wrist image fracture recognition and detection method based on the improved YOLOv8 network according to claim 2, characterized in that, It also includes introducing the Generalized Intersection over Union (GIoU) loss function based on the Intersection over Union (IoU) loss function, and constructing the Internal Generalized Intersection over Union (Internal GIoU) loss function as the loss function of the improved YOLOv8 recognition network model. The expression of the Internal GIoU loss function is specifically as follows: ； In the above formula, represents the internal generalized intersection over union loss function, represents the internal intersection over union loss function, represents the generalized intersection over union, represents the area of the minimum bounding rectangle of two bounding boxes, represents the union area.

4. The wrist image fracture recognition and detection method based on the improved YOLOv8 network according to claim 3, wherein, The step of using the improved YOLOv8 recognition network model to perform image recognition processing on the wrist fracture image to obtain the wrist image fracture recognition result specifically includes: Input the wrist fracture image into the improved YOLOv8 recognition network model; Based on the backbone network module of the improved YOLOv8 recognition network model, perform feature extraction processing on the wrist fracture image to obtain the wrist fracture feature image; Based on the neck network module of the improved YOLOv8 recognition network model, perform feature fusion processing on the wrist fracture feature image to obtain the fused wrist fracture feature image; Based on the detection head network module of the improved YOLOv8 recognition network model, perform target recognition detection on the fused wrist fracture feature image to obtain the wrist image fracture recognition result.

5. The wrist image fracture recognition and detection method based on the improved YOLOv8 network according to claim 4, characterized in that, The step of using the backbone network module of the improved YOLOv8 recognition network model to perform feature extraction processing on the wrist fracture image to obtain the wrist fracture feature image specifically includes: Input the wrist fracture image into the backbone network module of the improved YOLOv8 recognition network model; Based on the embedding layer of the backbone network module, perform data dimensionality reduction processing on the wrist fracture image to obtain the dimension-reduced wrist fracture image; Based on the fast network module of the backbone network module, perform local detail feature and global structure feature extraction processing on the dimension-reduced wrist fracture image to obtain the preliminary wrist fracture feature image; Based on the backbone fusion block of the backbone network module, perform multi-resolution feature fusion processing on the preliminary wrist fracture feature image to obtain the fused wrist fracture feature image; Based on the global pooling layer of the backbone network module, perform global pooling compression processing on the fused wrist fracture feature image to obtain the compressed wrist fracture feature image; Based on the backbone convolution block of the backbone network module, perform convolution processing on the compressed wrist fracture feature image to obtain the convolved wrist fracture feature image; Based on the fully connected layer of the backbone network module, perform mapping on the convolved wrist fracture feature image to obtain the mapped wrist fracture feature image; Based on the spatial pyramid fast pooling module of the backbone network module, perform multi-scale aggregation processing on the mapped wrist fracture feature image to obtain the wrist fracture feature image.

6. The wrist image fracture recognition and detection method based on the improved YOLOv8 network according to claim 5, wherein, The step of using the neck network module of the improved YOLOv8 recognition network model to perform feature fusion processing on the wrist fracture feature image to obtain the fused wrist fracture feature image specifically includes: Input the wrist fracture feature image into the neck network module of the improved YOLOv8 recognition network model; Based on the global-to-local spatial aggregation module of the neck network module, perform global attention weighted calculation and feature aggregation processing on the wrist fracture feature image to obtain the wrist fracture global feature image; The upsampling module based on the neck network module performs upsampling processing on the global feature image of wrist fracture to obtain the upsampled global feature image of wrist fracture; The neck fusion block based on the neck network module performs multi-scale feature fusion processing on the upsampled global feature image of wrist fracture to obtain the preliminarily fused global feature image of wrist fracture; The dynamic cross-stage deformable convolutional network module based on the neck network module performs cross-stage feature optimization processing on the preliminarily fused global feature image of wrist fracture to obtain the fused wrist fracture feature image.

7. The wrist image fracture recognition and detection method based on the improved YOLOv8 network according to claim 6, characterized in that, The step of the global-to-local spatial aggregation module based on the neck network module performing global attention weighted calculation and feature aggregation processing on the wrist fracture feature image to obtain the global feature image of wrist fracture specifically includes: Input the wrist fracture feature image into the global-to-local spatial aggregation module of the neck network module; Based on the first feature segmentation module of the global-to-local spatial aggregation module, perform feature segmentation processing on the wrist fracture feature image to obtain the segmented wrist fracture feature image; Based on the first branch two-dimensional convolutional module and the second branch two-dimensional convolutional module of the global-to-local spatial aggregation module, perform convolutional processing on the segmented wrist fracture feature image respectively to obtain the first convolutional wrist fracture feature image and the second convolutional wrist fracture feature image; Based on the global-to-local spatial aggregation convolutional branch module of the global-to-local spatial aggregation module, perform spatial feature aggregation processing on the first convolutional wrist fracture feature image to obtain the first aggregated wrist fracture feature image; Based on the context information aggregation module of the global-to-local spatial aggregation module, perform context information aggregation processing on the second convolutional wrist fracture feature image to obtain the second aggregated wrist fracture feature image; Based on the second feature splicing module of the global-to-local spatial aggregation module, splice the first aggregated wrist fracture feature image and the second aggregated wrist fracture feature image to obtain the spliced wrist fracture feature image; Based on the first convolutional block of the global-to-local spatial aggregation module, perform convolutional processing on the spliced wrist fracture feature image to obtain the global feature image of wrist fracture.

8. The wrist image fracture recognition and detection method based on the improved YOLOv8 network according to claim 7, characterized in that, The step of the dynamic cross-stage deformable convolutional network module based on the neck network module performing cross-stage feature optimization processing on the preliminarily fused global feature image of wrist fracture to obtain the fused wrist fracture feature image specifically includes: Input the preliminarily fused global feature image of wrist fracture into the dynamic cross-stage deformable convolutional network module of the neck network module; Based on the second convolutional block of the dynamic cross-stage deformable convolutional network module, perform convolutional processing on the preliminarily fused global feature image of wrist fracture to obtain the convolutional global feature image of wrist fracture; Based on the second feature segmentation module of the dynamic cross-stage deformable convolutional network module, perform feature segmentation processing on the convolutional global feature image of wrist fracture to obtain the segmented global feature image of wrist fracture; The dynamic deformable convolution bottleneck module based on the dynamic cross-stage deformable convolution network module performs dynamic feature optimization processing on the segmented global feature image of the wrist fracture to obtain the optimized wrist fracture feature image; The second feature splicing module based on the dynamic cross-stage deformable convolution network module performs feature splicing and fusion processing on the optimized wrist fracture feature image to obtain the fused wrist fracture feature image.

9. A wrist image fracture recognition and detection system based on an improved YOLOv8 network, characterized in that It includes the following modules: The first module is used to obtain the wrist fracture image dataset to be processed and perform data preprocessing to obtain the wrist fracture image; The second module is used to introduce a bidirectional feature pyramid network structure integrating the global-to-local spatial aggregation module and the dynamic cross-stage deformable convolution network module based on the fast network backbone structure, and construct an improved YOLOv8 recognition network model; The improved YOLOv8 recognition network model includes a backbone network module, a neck network module, and a detection head network module. The backbone network module, the neck network module, and the detection head network module are connected in sequence, where: The backbone network module adopts a fast network backbone structure. The backbone network module specifically includes an embedding layer, a fast network module, a backbone fusion block, a global pooling layer, a backbone convolution block, a fully connected layer, and a spatial pyramid fast pooling module. The fast network module includes a depthwise separable convolution layer and an attention mechanism; The neck network module adopts a bidirectional feature pyramid network structure. The neck network module specifically includes a global-to-local spatial aggregation module, an upsampling module, a neck fusion block, and a dynamic cross-stage deformable convolution network module. The global-to-local spatial aggregation module includes a first feature segmentation module, a first two-dimensional convolution module, a second two-dimensional convolution module, a global-to-local spatial aggregation convolution branch module, a context information aggregation module, a first feature splicing module, and a first convolution block. The dynamic cross-stage deformable convolution network module includes a second convolution block, a second feature segmentation module, several dynamic deformable convolution bottleneck modules, and a second feature splicing module; Among them, for the neck network, first replace the PAFPN in YOLOv8 with BiFPN, replace the convolution module in BiFPN with the global-to-local spatial aggregation module GLSA, design a multi-scale position-sensitive cross-attention module MPCA based on the cross-attention mechanism CA to enhance the offset and mask generation capabilities in the deformable convolution network DCNv2. By combining the multi-scale position-sensitive cross-attention module MPCA with the deformable convolution network DCNv2 and embedding it into the cross-stage partial network module C2f, a dynamic cross-stage deformable convolution network module C2f_DCNv2_Dynamic is formed, and further replace the cross-stage partial network module C2f in BiFPN; The third module is used to perform image recognition processing on the wrist fracture image based on the improved YOLOv8 recognition network model to obtain the wrist image fracture recognition result.

Images