A method for constructing, training and using a segmentation model

Abstract

The application provides a use method, a construction method and a training method of a segmentation model, comprising: obtaining a radiographic image and performing pretreatment; performing feature extraction and double-dimension attention enhancement on the radiographic image; performing progressive cross-layer feature fusion and multi-scale feature extraction processing respectively to obtain multi-scale fusion feature maps; performing prediction processing of detection dimension and segmentation dimension on each fusion feature map synchronously to output segmentation feature information. The beneficial effects of the application are that the model parameter quantity is greatly compressed by eliminating redundant levels to achieve volume lightweight; the progressive cross-layer feature fusion mechanism effectively alleviates the feature loss problem of small target fractures; the deformable convolution of the double-dimension attention enhancement module is self-adaptive to irregular geometric deformation of fractures, significantly improves the segmentation edge precision of blurred and complex fractures, and reduces the risk of missed diagnosis and misdiagnosis; and the knowledge distillation strategy realizes efficient knowledge transfer, greatly improves the detection and segmentation precision of the lightweight model.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method for constructing, training, and using a segmentation model. Background Technology

[0002] This invention relates to the fields of medical image analysis and computer vision technology, specifically to X-ray image processing technology for forearm fractures in children. Children's bones are not fully developed, and X-ray images related to forearm fractures present significant processing challenges. The fracture area has a very small pixel ratio, and the fracture edges are blurred and have irregular shapes, posing a great challenge to the accurate feature extraction and segmentation of the images.

[0003] Existing deep learning-based X-ray image processing technologies mainly fall into two categories: object detection and image segmentation. Neither can adequately address the processing needs of pediatric forearm fracture images. Object detection technologies can only achieve simple localization of the fracture area, failing to extract pixel-level fracture contours, resulting in insufficient information dimensionality in the output image processing results. While image segmentation technologies can achieve fine pixel-level segmentation, their complex model architecture, large number of parameters, and high computational cost make real-time deployment on peripheral medical devices such as portable X-ray machines difficult. Therefore, there is an urgent need to develop a lightweight segmentation model for instance segmentation of pediatric forearm radiographic images. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method for using, constructing, and training a segmentation model, which is particularly suitable for instance segmentation of radiographic images of children's forearms.

[0005] The technical solution adopted in this invention is as follows: Firstly, a method for constructing a segmentation model is provided, comprising the following steps:

[0006] The YOLOv8s-seg model was selected as the baseline model.

[0007] Remove redundant downsampling layers from the backbone network of the baseline model;

[0008] A progressive cross-layer feature fusion network structure is constructed in the baseline model, specifically including: setting a progressive cross-layer skip connection module between the shallow feature layer P3 of the backbone network and the intermediate feature layer of the same scale in the neck network to concatenate the shallow features and the intermediate features; configuring a multi-scale grouped dilated convolution module at the output of the progressive cross-layer skip connection module; and connecting the output of the multi-scale grouped dilated convolution module to the deep feature layer of the same scale in the neck network.

[0009] A two-dimensional attention enhancement module is configured at the end of the feature extraction of the baseline model. The two-dimensional attention enhancement module includes a channel attention submodule and an adaptive convolutional spatial attention submodule connected in sequence.

[0010] The segmentation model is obtained after parameter initialization.

[0011] Furthermore, the redundant downsampling levels include Stage 5.

[0012] Furthermore, the dual-dimensional attention enhancement module is a convolutional block attention module that integrates deformable convolution.

[0013] Secondly, a training method for a segmentation model is provided, including the following steps:

[0014] Create a training dataset;

[0015] Select a model that is from the same source as the segmentation model architecture and use it as the parent model after pre-training;

[0016] The segmentation model is used as a sub-model, and a key intermediate layer is established with the parent model to realize the direct transfer of feature knowledge between models;

[0017] Define a composite total loss function that combines the distillation loss and the model's native loss.

[0018] An improved cosine annealing function is used to set a dynamic adjustment rule for the distillation loss weight;

[0019] Configure training hyperparameters for the segmentation model, and simultaneously input the training dataset into the parent model and the segmentation model to perform model training;

[0020] The loss ratio is adjusted in real time according to the distillation loss weight, and the network parameters of the segmentation model are continuously optimized by backpropagating the composite total loss function.

[0021] Perform a preset number of training rounds, select the network parameters with the best segmentation performance, and save them as the model weights of the segmentation model.

[0022] Furthermore, building the training dataset includes the following steps:

[0023] Acquire several radiographic images and convert them into three-channel JPG format;

[0024] The actual edges of the fracture area in the radiographic image are annotated with pixel-level polygons;

[0025] The radiographic images are normalized.

[0026] The radiographic images are divided into a training set and a test set to form the training dataset.

[0027] Furthermore, the dynamic adjustment rule for setting the distillation loss weight using the improved cosine annealing function includes the following steps:

[0028] The improved cosine annealing function is used as the dynamic adjustment function for the distillation loss weight;

[0029] Configure dedicated hyperparameters for the improved cosine annealing function;

[0030] Set the number of training cycles for the preheating phase, and keep the initial weight of the distillation loss unchanged during the preheating phase;

[0031] The distillation loss is smoothly decayed to the minimum distillation loss weight by the improved cosine annealing function as the training period increases, and is maintained until the end of training.

[0032] Thirdly, a method for using the segmentation model is provided, including the following steps:

[0033] Acquire radiographic images;

[0034] The radiographic images are preprocessed;

[0035] The radiographic image is input into the segmentation model;

[0036] The radiographic image is downsampled layer by layer to extract shallow high-resolution features, generating a high-scale core feature map, and then a medium-scale core feature map and a low-scale core feature map are derived by sampling dimension matching.

[0037] The high-scale core feature map is then weighted by channel dimension to focus on key feature regions, and the focusing range is adjusted by adaptive spatial sampling.

[0038] Multi-scale fusion processing is performed on the core feature maps at each scale to generate multi-scale fused feature maps;

[0039] Simultaneously perform prediction processing on the detection dimension and segmentation dimension of each fused feature map, and output segmentation feature information.

[0040] Furthermore, it also includes the following steps:

[0041] The segmentation feature information is then filtered based on confidence level.

[0042] Non-maximum suppression is applied to the segmentation feature information;

[0043] The segmentation feature information is upsampled and cropped using a mask to obtain the fracture region bounding box information and the instance segmentation mask image.

[0044] Furthermore, performing multi-scale fusion processing on the core feature maps at each scale includes the following steps:

[0045] The core feature maps at each scale are concatenated with the intermediate layer features at the same scale to obtain the first concatenated feature;

[0046] The first spliced ​​feature is fused by multi-scale convolution to obtain the first fused feature;

[0047] The first fusion feature is fused with the deep semantic feature to obtain the multi-scale fusion feature map.

[0048] The advantages and positive effects of this invention are as follows: By adopting the above technical solution, the number of model parameters is greatly compressed and the size is lightened by eliminating redundant layers, which solves the contradiction between high accuracy and deployment requirements in traditional instance segmentation models; the progressive cross-layer feature fusion mechanism effectively alleviates the feature loss problem of small target fractures; the deformable convolution of the dual-dimensional attention enhancement module adapts to the irregular geometric deformation of fractures, significantly improving the segmentation edge accuracy of fuzzy and complex fractures and reducing the risk of missed diagnosis and misdiagnosis; the knowledge distillation strategy realizes efficient knowledge transfer, which greatly improves the detection and segmentation accuracy of the lightweight model. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating a method for constructing a segmentation model according to an embodiment of the present invention;

[0050] Figure 2 This is a flowchart illustrating the method of using a segmentation model according to an embodiment of the present invention. Detailed Implementation

[0051] The present disclosure will now be described more fully with reference to the accompanying drawings, which illustrate exemplary embodiments of the present disclosure. The technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative effort are within the scope of protection of the present disclosure.

[0052] like Figure 1 As shown, this invention provides a method for constructing a segmentation model, comprising the following steps:

[0053] S10. Select the YOLOv8s-seg model as the baseline model;

[0054] Specifically, the YOLOv8s-seg model was selected as the baseline model, its general instance segmentation pre-trained weights were loaded, and the model input size was determined to be 640×640 pixels with 3 channels.

[0055] S11. Remove redundant downsampling layers from the backbone network in the baseline model;

[0056] S12. Construct a progressive cross-layer feature fusion network structure in the baseline model;

[0057] S13. Configure a two-dimensional attention enhancement module at the end of the feature extraction of the baseline model. The two-dimensional attention enhancement module includes a channel attention sub-module and an adaptive convolutional spatial attention sub-module connected in sequence.

[0058] Specifically, the dual-dimensional attention enhancement module (CBAM-DC module) is connected in series after the backbone network SPPF module and before the neck network, receiving the 80×80 resolution feature map output by the SPPF module. The dual-dimensional attention enhancement module includes a channel attention submodule and an adaptive convolutional spatial attention submodule connected in sequence. The channel attention submodule performs global max pooling and average pooling on the input feature map, calculates the channel weights through a shared MLP network, focuses on key feature channels related to fracture, and suppresses background noise. The adaptive convolutional spatial attention submodule pools the channel-weighted feature map, predicts the offset of deformable convolution using 3×3 lightweight convolution, and adapts to the geometric deformation of the fracture region with 9 sampling points to accurately capture the blurred fracture edge.

[0059] S14. After parameter initialization, the segmentation model is obtained.

[0060] Specifically, the pre-trained weights of Stage 1 to Stage 4 and the SPPF module in the YOLOv8s-seg baseline model are retained, only the association parameters of the pruned modules are reset, and the newly added modules are adapted and initialized to obtain the segmentation model.

[0061] Using the above method, redundant downsampling layers are eliminated, significantly reducing the number of model parameters and achieving model lightweighting; a progressive cross-layer feature fusion network structure is built to enhance the model's feature extraction capability for small target fractures and alleviate the feature loss problem caused by deep downsampling; a dual-dimensional attention enhancement module is configured to focus on key fracture features from both channel and spatial dimensions, adapt to the irregular geometric deformation of fractures, and improve the segmentation accuracy of fracture edges.

[0062] To address the issues of lack of dedicated modules for cross-layer feature fusion, unclear feature transfer paths, low efficiency of multi-scale feature fusion, and poor connection between shallow and deep feature fusion, this embodiment provides an implementation method.

[0063] In one embodiment, building a progressive cross-layer feature fusion network structure in a baseline model includes the following steps:

[0064] A progressive cross-layer skip connection module is set between the shallow feature layer P3 of the backbone network and the intermediate feature layer of the same scale in the neck network to splice the shallow features and intermediate features.

[0065] A multi-scale grouped dilated convolution module (MSGDC module) is configured at the output of the progressive cross-layer skip connection module.

[0066] The output of the multi-scale grouped dilated convolution module is connected to the same-scale deep feature layer of the neck network.

[0067] Using the above method, a cross-layer skip connection module is set up to establish a direct transmission channel between shallow and mid-layer features, reducing feature loss during layer-by-layer transmission and ensuring the effective transmission of shallow detailed features. A multi-scale convolutional fusion module is configured at the output of the connection module, enabling efficient fusion of multiple receptive fields after splicing, thus improving the model's adaptability to fractures of different sizes. The multi-scale convolutional fusion module is directly connected to the deep feature layer, allowing seamless integration of fused features with deep semantic features, achieving progressive feature fusion, and improving the overall feature fusion effect of the model.

[0068] In one embodiment, the redundant downsampling level includes Stage 5.

[0069] Preferably, Stage 5 and related modules used to generate P5 feature maps in the backbone network of the baseline model are removed, while Stage 1 to Stage 4, the basic convolutional module (CBS), and the spatial pyramid pooling module (SPPF) are retained to achieve lightweighting. The CBS module processes the input image layer by layer at a downsampling rate of 2x, and finally generates an 80×80 resolution P3 shallow feature map. The SPPF module performs multi-receptive field aggregation on the P3 features to retain fracture detail information.

[0070] In one embodiment, the two-dimensional attention enhancement module is a convolutional block attention module that integrates deformable convolution.

[0071] Preferably, after the SPPF module at the end of the backbone network of the baseline model, a CBAM-DC module (two-dimensional attention enhancement module) is integrated. The CBAM-DC module retains the standard channel attention mechanism of CBAM and replaces the standard 7×7 convolution in its spatial attention module with deformable convolution. The offset of the deformable convolution is predicted by a lightweight 3×3 convolution layer with a sampling number of N=9, so that the convolution kernel adapts to the irregular geometry of the fracture region.

[0072] To train a segmentation model in a segmentation model construction method, this disclosure also provides a training method for a segmentation model, comprising the following steps:

[0073] Create a training dataset;

[0074] Select a model that is from the same architecture as the segmentation model and use it as the parent model after pre-training;

[0075] Preferably, the YOLOv8s-seg model pre-trained on images from the COCO dataset is selected as the parent model, and its pre-trained weights are loaded to provide a foundation for knowledge distillation for subsequent model training.

[0076] By using the segmentation model as a sub-model and establishing a key intermediate layer with the parent model, the direct transfer of feature knowledge between models can be achieved.

[0077] Specifically, in the backbone and neck networks of the parent and child models (segmentation models), four key intermediate layers with the same feature processing function and similar feature output dimensions are located. These are the core nodes in the model feature extraction / fusion process. At the same time, the output attributes such as feature map resolution and number of channels of each key intermediate layer are confirmed to ensure that the feature dimensions of the mapping layers at the same position in the parent and child models match.

[0078] Preferably, the mapping relationship between the parent model and the child model is as follows: the SPPF module of the parent model corresponds to the CBAM-DC module of the child model, the first C2f module of the parent model corresponds to the MSGDC module of the child model, the second C2f module of the parent model corresponds to the first C2f module of the child model, and the third C2f module of the parent model corresponds to the second C2f module of the child model.

[0079] Define a composite total loss function, which combines the distillation loss and the model's native loss.

[0080] Specifically, we define the composite total loss function. The equation is ,in For the model's native loss, For distillation losses, The weights for the distillation loss are: the model's native loss (object detection and segmentation loss), and the distillation loss (feature distillation loss, prototype distillation loss, and mask coefficient distillation loss). Feature distillation loss calculates the mean squared error between corresponding intermediate layer feature maps; prototype distillation loss calculates the mean squared error between prototype feature maps output by the segmentation head; and mask coefficient distillation loss uses temperature scaling (T = temperature hyperparameter) and KL divergence to calculate the difference between the probability distributions of the mask coefficient outputs.

[0081] An improved cosine annealing function is used to set a dynamic adjustment rule for the distillation loss weight.

[0082] Configure training hyperparameters for the segmentation model, and simultaneously input the training dataset into both the parent model and the segmentation model to perform model training;

[0083] Preferably, the training hyperparameters are set as follows: initial learning rate of 0.01, batch size of 16, and training for 200 epochs.

[0084] The loss ratio is adjusted in real time based on the distillation loss weight, and the network parameters of the segmentation model are continuously optimized through backpropagation of the composite total loss function.

[0085] Specifically, before each epoch of model training, the current training round is determined. Based on the improved cosine annealing function, the distillation loss weight corresponding to the current epoch is calculated and obtained in real time. After the training dataset is input into the parent model and child model and forward propagation is performed, the total loss value of distillation loss and the model's native loss is calculated respectively, providing a basis for adjusting the loss ratio. Using the real-time obtained distillation loss weight as the weight coefficient, the ratio of the two types of loss is dynamically adjusted and the composite total loss function is calculated. The gradient is initialized, the backpropagation of the composite total loss function is performed, the optimizer is called, the sub-model network parameters are updated based on the gradient value, and batch iteration is performed to continuously optimize the network parameters.

[0086] Perform a preset number of training rounds, select the network parameters with the best segmentation performance, and save them as the model weights of the segmentation model.

[0087] Using the above method, a pre-trained parent model with the same architecture as the segmentation model is selected. Leveraging pre-trained knowledge from a general dataset, knowledge is transferred to small-sample medical scenarios, improving the model's basic performance. A key intermediate layer is established to directly transfer feature knowledge, reducing information loss during knowledge distillation and significantly improving the efficiency of knowledge transfer. Dynamically adjusting the distillation loss weights allows the model to smoothly transition from imitating the parent model to autonomous optimization, better adapting to the specific task of detecting pediatric forearm fracture images. A composite total loss function integrates distillation loss and native loss, achieving knowledge transfer from multiple levels and dimensions, effectively improving the model's detection and segmentation accuracy. Standardized training procedures and parameter optimization ensure the stability of model training, allowing the model's network parameters to be fully optimized, thus improving the final segmentation performance.

[0088] To address the issues of inconsistent medical radiological image formats, low annotation accuracy, lack of standardized data processing, unreasonable dataset partitioning leading to data leakage, and poor quality of model training samples, this embodiment provides an implementation method.

[0089] In one embodiment, building a training dataset includes the following steps:

[0090] Acquire several radiographic images and convert them into three-channel JPG format;

[0091] Pixel-level polygon annotations are performed on the actual edges of the fracture area in the radiographic images;

[0092] Normalize the radiographic images;

[0093] The radiographic images are divided into a training set and a test set to form a training dataset.

[0094] Using the above method, pixel-level polygon annotation provides accurate pixel-level supervision signals for the instance segmentation model, enabling the model to learn the precise edge features of fractures and improve the segmentation accuracy of the model. Normalizing the images can accelerate the training convergence speed of the model, ensure the stability of the model training process, and reduce the training bias caused by pixel value differences.

[0095] To address the issues of insufficient initial knowledge learning in the model due to the lack of a clear execution process for adjusting distillation loss weights, the absence of a preheating stage, and the irregular weight decay leading to poor integration between knowledge transfer and autonomous optimization, this embodiment provides an implementation method.

[0096] In one embodiment, dynamically adjusting the distillation loss weights using an improved cosine annealing function includes the following steps:

[0097] An improved cosine annealing function is used as the dynamic adjustment function for the distillation loss weight;

[0098] Configure dedicated hyperparameters for the improved cosine annealing function;

[0099] Preferably, the specific hyperparameters of the cosine annealing function are: initial weight 0.8, minimum weight 0.3, number of warm-up rounds 10, total number of training rounds 100, and decaying shaping parameter 0.8.

[0100] Set the number of training cycles for the warm-up phase, and keep the initial weight of the distillation loss unchanged during the warm-up phase;

[0101] The distillation loss is smoothly decayed to the minimum distillation loss weight by an improved cosine annealing function as the training cycle increases, and is maintained until the end of training.

[0102] Specifically, in the warm-up phase: for the first 10 epochs of training, the distillation loss weight remains unchanged at its initial value of 0.8, forcing the sub-model to strongly imitate the feature output of the parent model; in the decay phase: after the warm-up epochs, from the 11th epoch to the 100th epoch, the distillation loss weight is gradually and smoothly decayed from 0.8 to 0.3 using an improved cosine annealing function, gradually reducing the intensity of imitation of the parent model and encouraging the sub-model to autonomously optimize and adapt to the pediatric forearm fracture detection task; in the subsequent maintenance phase: if the training cycle exceeds 100 epochs, the distillation loss weight remains unchanged at its minimum weight of 0.3, allowing the sub-model to continue learning autonomously.

[0103] Using the above method, an improved cosine annealing function is employed as the weight adjustment function to provide a scientific basis for the dynamic changes of the distillation loss weights, ensuring the smoothness of weight adjustment and avoiding model training oscillations caused by sudden weight changes. Dedicated hyperparameters are configured for the function to make the rhythm and magnitude of weight adjustment more closely match the training task requirements for pediatric forearm fracture detection, thus improving the targeting of weight adjustment. A warm-up phase is set up while keeping the initial weights unchanged, allowing the model to intensively imitate the parent model in the early stages of training, fully learning general knowledge and solidifying the model's knowledge base. After warm-up, the weights smoothly decay with the training cycle and remain until the end of training, achieving a smooth transition from imitation learning to autonomous optimization, improving knowledge transfer efficiency, and allowing the model to better adapt to the target detection task.

[0104] like Figure 2 As shown, in order to better utilize the segmentation model in a segmentation model construction method, this invention also provides a method for using the segmentation model, including the following steps:

[0105] S20. Acquire radiographic images;

[0106] Specifically, X-ray images of fractures in the limbs are acquired. Preferably, X-ray images of fractures in the forearm of a child are acquired.

[0107] S21. Preprocess the radiographic images;

[0108] Specifically, the radiographic images are converted to three-channel JPG format and normalized to a uniform resolution. Preferably, the radiographic images are normalized and uniformly scaled to a resolution of 640×640 pixels.

[0109] S22. Input the radiographic image into the segmentation model;

[0110] Specifically, the radiographic images are input into the segmentation model, and a single forward propagation is performed.

[0111] S23. Perform layer-by-layer downsampling on the radiographic image to extract shallow high-resolution features, generate a high-scale core feature map, and derive a medium-scale core feature map and a low-scale core feature map by sampling dimension matching.

[0112] Specifically, the radiographic images are input into the backbone network for layer-by-layer downsampling feature extraction. After 2x downsampling, an 80×80 resolution P3 shallow feature map is generated. The SPPF module performs multi-receptive field aggregation on the P3 features to preserve fracture details. By matching upsampling / downsampling dimensions, a 40×40 and 20×20 three-scale feature pyramid is derived to capture image features of different receptive fields.

[0113] S24. Focus on key feature regions by channel dimension weighting of the high-scale core feature map, and then adjust the focusing range through adaptive spatial sampling;

[0114] Specifically, the high-scale core feature map is input into the dual-dimensional attention enhancement module. First, global max pooling and average pooling are performed on the input feature map through the channel attention submodule. Then, the channel-weighted feature map is pooled through adaptive convolutional spatial attention. The offset of deformable convolution is predicted by 3×3 lightweight convolution. Nine sampling points are used to adapt to the geometric deformation of the fracture region and accurately capture the blurred fracture edge. Finally, the high-scale core feature map enhanced by dual dimensions is output.

[0115] S25. Perform multi-scale fusion processing on the core feature maps at each scale to generate multi-scale fused feature maps;

[0116] S26. Simultaneously perform prediction processing on each fused feature map for both detection and segmentation dimensions, and output segmentation feature information. Segmentation feature information includes bounding box coordinates, feature confidence scores, mask prototype coefficients, and mask prototype maps.

[0117] Specifically, the fused feature maps at various scales are processed in parallel. First, detection dimension features are extracted, and the bounding box coordinates and fracture feature confidence information of the fracture region are extracted for each feature map grid to characterize the location and feature saliency of the fracture region in the image. Then, segmentation dimension features are extracted, and mask prototype coefficients and mask prototype maps are generated for each extracted bounding box.

[0118] Using the above methods, a single forward propagation simplifies the inference process, significantly improving image processing efficiency and adapting to the deployment needs of edge devices with limited computing resources. Feature extraction combined with dual-dimensional attention enhancement can accurately focus on key fracture features, effectively capture the geometric deformation features of irregular fractures, and reduce the missed detection of small target fractures. Multi-scale feature fusion and extraction are adapted to pediatric forearm fractures of different sizes, giving the model good feature processing capabilities for small, conventional, and complex fractures. Detection and segmentation dimensions are processed simultaneously, outputting segmentation feature information in one go, enriching the information dimensions of the image processing results and providing comprehensive support for subsequent image analysis.

[0119] To address the issues of redundant segmentation feature information, low mask resolution, and overlapping fracture region localization in the original output of the model, which prevent it from directly adapting to the needs of clinical radiological image analysis, this embodiment provides an implementation method.

[0120] In one embodiment, the following steps are also included:

[0121] Confidence filtering is performed on the segmentation feature information;

[0122] Specifically, low-quality feature information with a confidence level below a preset threshold is removed to reduce interference from invalid information.

[0123] Non-maximum suppression is applied to the segmentation feature information;

[0124] Specifically, for the retained feature information, for the parts that predict the same fracture and whose bounding boxes highly overlap, only the bounding box information with the highest confidence is retained, and redundant localization information is eliminated.

[0125] The segmentation feature information is upsampled and cropped using a mask to obtain the bounding box information of the fracture region and the instance segmentation mask image.

[0126] Specifically, a coarse segmentation mask is obtained by linearly combining the low-resolution mask prototype image with the mask prototype coefficients; the coarse segmentation mask is upsampled to the corresponding bounding box size, then cropped and mapped back to the coordinate space of the original resolution radiographic image to generate a pixel-level fracture feature segmentation mask image with the same resolution as the original radiographic image.

[0127] Using the above methods, confidence filtering can quickly remove low-quality segmentation feature information, reduce the interference of invalid information on subsequent image processing, and improve the effectiveness of processing results; non-maximum suppression can eliminate overlapping bounding box information in the fracture area, making the fracture area more accurately located and avoiding processing errors such as repeated annotation; mask upsampling and cropping restore the low-resolution mask to an instance segmentation mask image with the same resolution as the original image, accurately delineating the fracture contour and meeting the clinical need for detailed anatomical information.

[0128] To address the issues that direct feature fusion can lead to a surge in the number of channels, feature loss, and an inability to smoothly transition between shallow detail features and deep semantic features in small target fractures, this embodiment provides an implementation method.

[0129] In one embodiment, performing multi-scale fusion processing on core feature maps at each scale includes the following steps:

[0130] The core feature maps at each scale are concatenated with the intermediate layer features at the same scale to obtain the first concatenated feature;

[0131] The first concatenated feature is fused using multi-scale convolution to obtain the first fused feature;

[0132] The first fusion feature is fused with the deep semantic feature to obtain a multi-scale fusion feature map.

[0133] Specifically, the shallow high-resolution features of layer P3 in the high-scale core feature map are concatenated with the intermediate feature map output from the first layer of the neck network to form the first concatenated feature. The first concatenated feature is input into the MSGDC module (multi-scale grouped dilated convolution module), and features of different receptive fields are extracted through three sets of parallel grouped convolutions with dilation rates of 1, 3, and 5 (small dilation rate captures details, large dilation rate covers a wide range of features). Then, the features are fused through shortcut connections and forward propagation to output the first fused feature, avoiding channel redundancy caused by direct concatenation. The first fused feature is then fused element-wise with the deep semantic features (including fracture category-related information) of the same scale as the neck network to complete the progressive feature processing of "shallow details, multi-scale enhancement, and deep semantics" and generate a fused feature map. The intermediate-scale core feature map and the low-scale core feature map are also progressively fused at the corresponding scales according to the above logic, and finally, a multi-scale fused feature map adapted to fractures of different sizes is generated.

[0134] Using the above method, shallow high-resolution features are concatenated with intermediate layer features. This preserves the superficial details of the fracture while incorporating basic semantic features, avoiding the information deficiencies of a single feature dimension and laying the foundation for subsequent fusion. Multi-scale convolutional fusion of the concatenated features can capture fracture features from different receptive fields, better adapting to the irregular and diverse morphology of children's forearm fractures and improving the comprehensiveness of feature fusion. Secondary fusion with deep semantic features achieves a smooth transition from shallow details to deep semantics, effectively alleviating the problem of loss of small target fracture features during the fusion process and improving the effectiveness of feature fusion.

[0135] Based on embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0136] An electronic device includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the usage, construction, and training methods of the segmentation model provided in this disclosure.

[0137] Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0138] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the usage, construction, and training methods of the segmentation model provided in this disclosure.

[0139] The various embodiments of this disclosure can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] A computer program product includes a computer program / instructions, which are executed by a processor, and the methods for using, constructing, and training the segmentation model provided in this disclosure are also included.

[0141] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0142] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0143] The embodiments of the present invention have been described in detail above, but the content described is only a preferred embodiment of the present invention and should not be considered as limiting the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the patent coverage of the present invention.

Claims

1. A method for constructing a segmentation model, characterized in that, Includes the following steps: The YOLOv8s-seg model was selected as the baseline model. Redundant downsampling layers in the backbone network of the baseline model are removed, including Stage 5 layers; A progressive cross-layer feature fusion network structure is constructed in the baseline model, specifically including: setting a progressive cross-layer skip connection module between the shallow feature layer P3 of the backbone network and the intermediate feature layer of the same scale in the neck network to concatenate the shallow features and the intermediate features; configuring a multi-scale grouped dilated convolution module at the output of the progressive cross-layer skip connection module; and connecting the output of the multi-scale grouped dilated convolution module to the deep feature layer of the same scale in the neck network. A two-dimensional attention enhancement module is configured at the end of the feature extraction of the baseline model. The two-dimensional attention enhancement module includes a channel attention sub-module and an adaptive convolutional spatial attention sub-module connected in sequence. The two-dimensional attention enhancement module is a convolutional block attention module that integrates deformable convolution. The segmentation model is obtained after parameter initialization; Training a segmentation model involves the following steps: Create a training dataset; Select a model that is from the same source as the segmentation model architecture and use it as the parent model after pre-training; The segmentation model is used as a sub-model, and a key intermediate layer is established with the parent model to realize the direct transfer of feature knowledge between models; Define a composite total loss function that combines the distillation loss and the model's native loss. An improved cosine annealing function is used to set a dynamic adjustment rule for the distillation loss weight; Configure training hyperparameters for the segmentation model, and simultaneously input the training dataset into the parent model and the segmentation model to perform model training; The loss ratio is adjusted in real time according to the distillation loss weight, and the network parameters of the segmentation model are continuously optimized by backpropagating the composite total loss function. Perform a preset number of training rounds, select the network parameters with the best segmentation performance, and save them as the model weights of the segmentation model; The use of segmentation models includes the following steps: Acquire radiographic images; The radiographic images are preprocessed; The radiographic image is input into the segmentation model; The radiographic image is downsampled layer by layer to extract shallow high-resolution features, generating a high-scale core feature map, and then a medium-scale core feature map and a low-scale core feature map are derived by sampling dimension matching. The high-scale core feature map is then weighted by channel dimension to focus on key feature regions, and the focusing range is adjusted by adaptive spatial sampling. Multi-scale fusion processing is performed on the core feature maps at each scale to generate multi-scale fused feature maps. Specifically, this includes: concatenating the shallow high-resolution features of layer P3 in the high-scale core feature map with the intermediate feature map output from the first layer of the neck network to form a first concatenated feature; inputting the first concatenated feature into the multi-scale grouped dilated convolution module, extracting features from different receptive fields through three sets of parallel grouped convolutions with dilation rates of 1, 3, and 5, and then fusing them through shortcut connections and forward propagation to output the first fused feature, avoiding channel redundancy caused by direct concatenation; fusing the first fused feature with the deep semantic features of the same scale in the neck network element-wise to complete the progressive feature processing of "shallow details, multi-scale enhancement, and deep semantics" to generate a fused feature map; the mid-scale core feature map and the low-scale core feature map are also progressively fused at their corresponding scales according to the above logic to finally generate the multi-scale fused feature map. Simultaneously perform prediction processing on the detection dimension and segmentation dimension of each fused feature map, and output segmentation feature information.

2. The method for constructing a segmentation model according to claim 1, characterized in that, Creating a training dataset involves the following steps: Acquire several radiographic images and convert them into three-channel JPG format; The actual edges of the fracture area in the radiographic image are annotated with pixel-level polygons; The radiographic images are normalized. The radiographic images are divided into a training set and a test set to form the training dataset.

3. The method for constructing a segmentation model according to claim 1 or 2, characterized in that, The dynamic adjustment rule for setting the distillation loss weight using an improved cosine annealing function includes the following steps: The improved cosine annealing function is used as the dynamic adjustment function for the distillation loss weight; Configure dedicated hyperparameters for the improved cosine annealing function; Set the number of training cycles for the preheating phase, and keep the initial weight of the distillation loss unchanged during the preheating phase; The distillation loss is smoothly decayed to the minimum distillation loss weight by the improved cosine annealing function as the training period increases, and is maintained until the end of training.

4. The method for constructing a segmentation model according to claim 1, characterized in that, It also includes the following steps: The segmentation feature information is then filtered based on confidence level. Non-maximum suppression is applied to the segmentation feature information; The segmentation feature information is upsampled and cropped using a mask to obtain the fracture region bounding box information and the instance segmentation mask image.

5. The method for constructing a segmentation model according to claim 1 or 4, characterized in that, Performing multi-scale fusion processing on the core feature maps at each scale includes the following steps: The core feature maps at each scale are concatenated with the intermediate layer features at the same scale to obtain the first concatenated feature; The first spliced ​​feature is fused by multi-scale convolution to obtain the first fused feature; The first fusion feature is fused with the deep semantic feature to obtain the multi-scale fusion feature map.

Images